STRUMPACK vs EIGEN performance
The goal of this notebook is to allow the documentation of STRUMPACK vs EIGEN performance to be maintained in a single accessible location. The environment within which this notebook is run follows the standard cctbx conda build instructions available here. For this instance, we are using the STRUMPACK-enabled build of cctbx located at ExaFEL:cctbx_project(str_merge). STRUMPACK is currently built using the installation script STRUMPACK_installer_shared.sh, and if the installation takes place within the same directory as moddules and build, the cctbx build process can make use of it as a backend. After the STRUMPACK solver finishes, enter the build directory, run libtbx.refresh and make. The STRUMPACK-supported modules should now build and link with the new backend.
The solution of the below systems assume that we have some A and b data already, and we can provide these in CSV file format to the solver (A is in “row col value” format). We can begin by importing all of the required modules. We make use of Numpy’s ability to parse CSV files into numeric format, and SciPy’s sparse storage format to format the matrices into the required CSR sparse storage format for solving.
%matplotlib inline
%config InlineBackend.figure_format = 'retina'
import matplotlib
import matplotlib.pyplot as plt
font = {'family' : 'serif',
#'weight' : 'bold',
'size' : 12}
matplotlib.rc('font', **font)
matplotlib.rcParams['figure.figsize'] = (16,10)
matplotlib.rcParams['figure.dpi']= 150
import matplotlib.gridspec as gridspec
from __future__ import division
from cctbx.array_family import flex
from libtbx.test_utils import approx_equal
from libtbx.development.timers import Profiler
import sys
import numpy as np
import scipy.sparse as sps
%env BOOST_ADAPTBX_FPE_DEFAULT=1
%env BOOST_ADAPTBX_SIGNALS_DEFAULT=1
env: BOOST_ADAPTBX_FPE_DEFAULT=1
env: BOOST_ADAPTBX_SIGNALS_DEFAULT=1
We load the A and b data set for solving from the locations specified below. Ideally, we can loop over and load all different data sets and process each individually. For a working example, we begin by processing a single data set, comparing Eigen and Strumpack for varying levels of shared parallelism.
A_path="/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv"
A_mat = np.loadtxt(A_path,dtype={'names':('rows','cols','vals'),'formats':('i8','i8','f8')})
b_path="/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv"
b_vec = np.loadtxt(b_path)
We are solving system of normal equations, and so we can safely assume that we have a square matrix. This ensures the number of rows and colums will be identical, which can be read from the length of the numpy array b_vec.
n_rows = len(b_vec)
n_cols = n_rows
We next wish to create a SciPy CSR sparse matrix type using the data from A_mat.
A_sp = sps.csr_matrix((A_mat['vals'],(A_mat['rows'],A_mat['cols'])))
With this, we can now create the appropriate flex data types for solving the linear system. While the operations performed upto and including this point involve many copy steps, it can be stated that the time to solve the system is much greater than the time to perform internal copies, and so we can ignore the times required for these steps for any realistic datasets.
For the solvers we require the row index pointers, the column indices, and sparse matrix values. However, it is import to first ensure that our A matrix is symmetric, and not the upper or lower triangle only. While Eigen’s Cholesky deocmposition can use the upper or lower triangle, other algorithms and solvers tend to expect a full matrix. We can create this as follows.
tu=sps.triu(A_sp)
tl=sps.tril(A_sp)
sd=sps.diags(A_sp.diagonal())
if tu.nnz == sd.getnnz() or tl.nnz == sd.getnnz():
A_sp = A_sp + A_sp.transpose() - sd
plt.spy(A_sp, marker='.',markersize=1.0)
plt.show()
 |
The resulting data sets can be read directly and converted to flex types as follows.
A_indptr = flex.int(A_sp.indptr)
A_indices = flex.int(A_sp.indices)
A_values = flex.double(A_sp.data)
b = flex.double(b_vec)
With the data in the required format, we can now create a solver object and solve the system. We also include a built-in profiler object to time this solver step for both the STRUMPACK and EIGEN backends.
Next, we load the OpenMP-enabled STRUMPACK solver module and the Eigen based solver. These are contained within “modules/cctbx_project/scitbx/examples/bevington/strumpack_solver_ext.cpp” and where a Boost.Python wrapper for both an EIGEN and STRUMPACK Ax=b solver object is provided.
A non-notebook script exists to test this functionality at “modules/cctbx_project/scitbx/examples/bevington/strumpack_eigen_solver.py”, which can be called as:
OMP_NUM_THREADS=x libtbx.python strumpack_eigen_solver.py A_mat.csv b_vec.csv
where A_mat.csv and b_vec.csv are the CSV files with the linear system to be solved, and x specifies the number of OpenMP threads to spawn. Additionally, the MPI-enabled STRUMPACK backend is provided by “modules/cctbx_project/scitbx/examples/bevington/strumpack_solver_ext_mpi_dist.cpp”, with the Python script to test the result located at “modules/cctbx_project/scitbx/examples/bevington/strumpack_eigen_solver_mpi_dist.py”. This is called as
OMP_NUM_THREADS=x mpirun -n y libtbx.python strumpack_eigen_solver.py A_mat.csv b_vec.csv
As an example, the previously loaded can be provided to the solvers as follows:
import boost.python
ext_omp = boost.python.import_ext("scitbx_examples_strumpack_solver_ext")
ext_mpi = boost.python.import_ext("scitbx_examples_strumpack_mpi_dist_solver_ext")
es = ext_omp.eigen_solver
ss = ext_omp.strumpack_solver
ssmd = ext_mpi.strumpack_mpi_dist_solver
def run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b):
P = Profiler("STRUMPACK_SCOTCH")
res_strum_sc = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.scotch, ext_omp.auto)
del P
P = Profiler("STRUMPACK_METIS")
res_strum_me = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.metis, ext_omp.auto)
del P
P = Profiler("EIGEN_LDLT")
res_eig_ldlt = es(1, n_rows, n_cols, A_indptr, A_indices, A_values, b)
del P
P = Profiler("EIGEN_BICGSTAB")
res_eig_bicgstab = es(2, n_rows, n_cols, A_indptr, A_indices, A_values, b)
del P
return res_strum_sc, res_strum_me, res_eig_ldlt, res_eig_bicgstab
r_s_sc, r_s_me, r_e_ldlt, r_e_bicgstab = run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b)
for i in xrange(len(r_s_sc.x)):
assert( approx_equal(r_s_sc.x[i], r_s_me.x[i]) )
assert( approx_equal(r_s_sc.x[i], r_e_ldlt.x[i]) )
assert( approx_equal(r_s_sc.x[i], r_e_bicgstab.x[i]) )
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.
try: mod = __import__(name)
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.
try: mod = __import__(name)
individual call time for STRUMPACK_SCOTCH: CPU, 19.210s; elapsed, 0.424s
individual call time for STRUMPACK_METIS: CPU, 176.190s; elapsed, 2.805s
individual call time for EIGEN_LDLT: CPU, 3.940s; elapsed, 0.063s
individual call time for EIGEN_BICGSTAB: CPU, 1.400s; elapsed, 0.023s
While we can use the above method to process the data, we must respecify the OMP_NUM_THREADS variable to observe different levels of scalability. %env OMP_NUM_THREADS is sufficient to test this for the first run, however for an undetermined reason the notebook must be started to allow the value to be changed. There, we can run the above tasks from a !<command> cell, specifying the thread count here instead.
OMP_SOLVER='''
from __future__ import division
from cctbx.array_family import flex
from libtbx.test_utils import approx_equal
from libtbx.development.timers import Profiler
import sys
import numpy as np
import scipy.sparse as sps
import matplotlib
import matplotlib.pyplot as plt
import matplotlib
import matplotlib.pyplot as plt
matplotlib.rcParams['figure.figsize'] = (16,10)
A_path=sys.argv[1]
A_mat = np.loadtxt(A_path,dtype={'names':('rows','cols','vals'),'formats':('i8','i8','f8')})
b_path=sys.argv[2]
b_vec = np.loadtxt(b_path)
n_rows = len(b_vec)
n_cols = n_rows
A_sp = sps.csr_matrix((A_mat['vals'],(A_mat['rows'],A_mat['cols'])))
#Check for triangular matrix. If so, A_sp := A+A^T - diag(A)
tu=sps.triu(A_sp)
tl=sps.tril(A_sp)
sd=sps.diags(A_sp.diagonal())
A_spS = A_sp
if tu.nnz == sd.getnnz() or tl.nnz == sd.getnnz():
A_spS = A_sp + A_sp.transpose() - sd
A_indptr = flex.int(A_sp.indptr)
A_indices = flex.int(A_sp.indices)
A_values = flex.double(A_sp.data)
b = flex.double(b_vec)
#import time
#timing_dict = {"strum":0, "eigen":0}
import boost.python
ext_omp = boost.python.import_ext("scitbx_examples_strumpack_solver_ext")
ext_mpi = boost.python.import_ext("scitbx_examples_strumpack_mpi_dist_solver_ext")
es = ext_omp.eigen_solver
ss = ext_omp.strumpack_solver
def run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS):
P = Profiler("EIGEN_LLT_CHOL")
res_eig_llt_chol = es(0, n_rows, n_cols, A_indptr, A_indices, A_values, b)
del P
P = Profiler("EIGEN_LDLT_CHOL")
res_eig_ldlt_chol = es(1, n_rows, n_cols, A_indptr, A_indices, A_values, b)
del P
A_indptr = flex.int(A_spS.indptr)
A_indices = flex.int(A_spS.indices)
A_values = flex.double(A_spS.data)
P = Profiler("EIGEN_BICGSTAB")
res_eig_bicgstab = es(2, n_rows, n_cols, A_indptr, A_indices, A_values, b)
del P
P = Profiler("STRUMPACK_SCOTCH_AUTO")
res_strum_sc_a = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.scotch, ext_omp.auto)
del P
P = Profiler("STRUMPACK_METIS_AUTO")
res_strum_mt_a = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.metis, ext_omp.auto)
del P
P = Profiler("STRUMPACK_SCOTCH_BICGSTAB")
res_strum_sc_bi = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.scotch, ext_omp.bicgstab)
del P
P = Profiler("STRUMPACK_METIS_BICGSTAB")
res_strum_mt_bi = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.metis, ext_omp.bicgstab)
del P
P = Profiler("STRUMPACK_SCOTCH_PRECBICGSTAB")
res_strum_sc_pr = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.scotch, ext_omp.prec_bicgstab)
del P
P = Profiler("STRUMPACK_METIS_PRECBICGSTAB")
res_strum_mt_pr = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.metis, ext_omp.prec_bicgstab)
del P
for i in xrange(len(res_strum_sc.x)):
assert( approx_equal(res_eig_llt_chol.x[i], res_eig_ldlt_chol.x[i]) )
assert( approx_equal(res_strum_sc_a.x[i], res_eig_bicgstab.x[i]) )
assert( approx_equal(res_strum_sc_a.x[i], res_eig_ldlt_chol.x[i]) )
assert( approx_equal(res_strum_mt_a.x[i], res_strum_sc_a.x[i]) )
assert( approx_equal(res_strum_mt_a.x[i], res_strum_sc_bi.x[i]) )
assert( approx_equal(res_strum_mt_a.x[i], res_strum_mt_bi.x[i]) )
assert( approx_equal(res_strum_mt_a.x[i], res_strum_sc_pr.x[i]) )
assert( approx_equal(res_strum_mt_a.x[i], res_strum_mt_pr.x[i]) )
run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)
'''
OMP_SOLVER = OMP_SOLVER
OMP_SOLVER_FILE = open("OMP_SOLVER.py", "w")
OMP_SOLVER_FILE.write(OMP_SOLVER)
OMP_SOLVER_FILE.close()
DATAPATH="/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/"
A_LIST = !find {DATAPATH} -iname "A*.csv"
B_LIST = [ii.replace('/A_','/b_') for ii in A_LIST]
We now record the indices of the data with a specific number of images. We can use these indices to later submit jobs with a given number of frames, and hence resulting matrix size.
list_idx={}
for imgs in ['1k','5k','10k','32k']:
list_idx.update({imgs:[i for i, j in enumerate(A_LIST) if imgs in j]})
list_idx
{'10k': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14],
'1k': [15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29],
'32k': [30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44],
'5k': [45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59]}
With the file writtent to disk, we now create a loop in mixed Python/Bash to call the solver, then extract the standard output from the profiler to examine the OpenMP scalability of STRUMPACK. The profiler code wraps each solver, and does not take into account the load-times for the data.
str_out={}
import os
threads_list = [16]
for imgs_size in list_idx:
print "Data Set Size:=%s"%imgs_size
#Subselect the smallest data size for now
if imgs_size != "1k":
print "Skipping %s"%imgs_size
continue
for imgs_idx in list_idx[imgs_size]:
A_path = A_LIST[imgs_idx]; b_path = B_LIST[imgs_idx]
dat_name = A_path.split('/')[-1][2:-4]
print "Data Set Name:=%s"%(dat_name)
#Ensure the A and b data are matched correctly
assert(os.path.dirname(A_path) == os.path.dirname(b_path))
print {A_path}, {b_path}
for threads in threads_list:
print "OMP_NUM_THREADS:=%d"%threads
val = !OMP_NUM_THREADS={threads} libtbx.python ./OMP_SOLVER.py {A_path} {b_path}
print val
for s in val:
if "assert" in s:
raise Exception("Solutions not equal! Halting")
key = 'omp' + str(threads) + '_' + dat_name
str_out.update({key:val})
—CLICK FOR OUTPUT—
Data Set Size:=10k
Skipping 10k
Data Set Size:=1k
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.810s; elapsed, 0.815s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.870s; elapsed, 0.863s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.025s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 4.280s; elapsed, 0.271s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 31.080s; elapsed, 1.960s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.930s; elapsed, 0.121s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.880s; elapsed, 0.117s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.920s; elapsed, 0.182s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 33.550s; elapsed, 2.101s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 33.550s; elapsed, 2.101s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.810s; elapsed, 0.815s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.930s; elapsed, 0.121s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.025s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 4.280s; elapsed, 0.271s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.920s; elapsed, 0.182s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.880s; elapsed, 0.117s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.870s; elapsed, 0.863s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 31.080s; elapsed, 1.960s, #calls: 1', 'TOTAL : CPU, 77.570s; elapsed, 6.455s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.220s; elapsed, 0.232s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.238s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.570s; elapsed, 0.049s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 7.060s; elapsed, 0.447s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 28.970s; elapsed, 1.831s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.720s; elapsed, 0.169s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.830s; elapsed, 0.177s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.750s; elapsed, 0.422s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 31.770s; elapsed, 2.001s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 31.770s; elapsed, 2.001s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.220s; elapsed, 0.232s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.720s; elapsed, 0.169s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.570s; elapsed, 0.049s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 7.060s; elapsed, 0.447s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.750s; elapsed, 0.422s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.830s; elapsed, 0.177s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.238s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 28.970s; elapsed, 1.831s, #calls: 1', 'TOTAL : CPU, 81.130s; elapsed, 5.566s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Deff-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.240s; elapsed, 0.237s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.230s; elapsed, 0.232s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.024s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.400s; elapsed, 0.215s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 28.710s; elapsed, 1.812s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.750s; elapsed, 0.109s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.740s; elapsed, 0.109s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.070s; elapsed, 0.192s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.190s; elapsed, 1.893s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.190s; elapsed, 1.893s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.240s; elapsed, 0.237s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.750s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.024s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.400s; elapsed, 0.215s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.070s; elapsed, 0.192s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.740s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.230s; elapsed, 0.232s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 28.710s; elapsed, 1.812s, #calls: 1', 'TOTAL : CPU, 69.570s; elapsed, 4.823s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.070s; elapsed, 0.064s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.050s; elapsed, 0.052s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.025s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.350s; elapsed, 0.212s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 28.510s; elapsed, 1.799s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.670s; elapsed, 0.105s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.760s; elapsed, 0.110s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.310s; elapsed, 0.207s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 26.660s; elapsed, 1.673s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 26.660s; elapsed, 1.673s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.070s; elapsed, 0.064s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.670s; elapsed, 0.105s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.025s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.350s; elapsed, 0.212s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.310s; elapsed, 0.207s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.760s; elapsed, 0.110s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.050s; elapsed, 0.052s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 28.510s; elapsed, 1.799s, #calls: 1', 'TOTAL : CPU, 65.610s; elapsed, 4.247s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 1.990s; elapsed, 1.995s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 2.110s; elapsed, 2.120s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.048s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 6.590s; elapsed, 0.417s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 34.870s; elapsed, 2.191s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.440s; elapsed, 0.153s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.620s; elapsed, 0.163s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.920s; elapsed, 0.433s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 38.160s; elapsed, 2.391s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 38.160s; elapsed, 2.391s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 1.990s; elapsed, 1.995s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.440s; elapsed, 0.153s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.048s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 6.590s; elapsed, 0.417s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.920s; elapsed, 0.433s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.620s; elapsed, 0.163s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 2.110s; elapsed, 2.120s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 34.870s; elapsed, 2.191s, #calls: 1', 'TOTAL : CPU, 96.220s; elapsed, 9.909s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 2.040s; elapsed, 2.044s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 2.090s; elapsed, 2.091s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.024s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.470s; elapsed, 0.221s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 35.250s; elapsed, 2.219s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.910s; elapsed, 0.118s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.840s; elapsed, 0.115s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.990s; elapsed, 0.188s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 38.560s; elapsed, 2.412s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 38.560s; elapsed, 2.412s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 2.040s; elapsed, 2.044s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.910s; elapsed, 0.118s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.024s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.470s; elapsed, 0.221s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.990s; elapsed, 0.188s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.840s; elapsed, 0.115s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 2.090s; elapsed, 2.091s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 35.250s; elapsed, 2.219s, #calls: 1', 'TOTAL : CPU, 88.400s; elapsed, 9.434s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.830s; elapsed, 0.819s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.860s; elapsed, 0.858s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.030s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.520s; elapsed, 0.223s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 30.270s; elapsed, 1.907s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.000s; elapsed, 0.125s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.070s; elapsed, 0.130s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.260s; elapsed, 0.204s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.920s; elapsed, 1.944s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.920s; elapsed, 1.944s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.830s; elapsed, 0.819s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.000s; elapsed, 0.125s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.030s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.520s; elapsed, 0.223s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.260s; elapsed, 0.204s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.070s; elapsed, 0.130s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.860s; elapsed, 0.858s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 30.270s; elapsed, 1.907s, #calls: 1', 'TOTAL : CPU, 74.030s; elapsed, 6.238s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.850s; elapsed, 0.843s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.860s; elapsed, 0.864s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.050s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 7.220s; elapsed, 0.459s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 31.850s; elapsed, 2.009s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.370s; elapsed, 0.148s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.580s; elapsed, 0.162s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.480s; elapsed, 0.406s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 34.160s; elapsed, 2.146s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 34.160s; elapsed, 2.146s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.850s; elapsed, 0.843s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.370s; elapsed, 0.148s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.050s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 7.220s; elapsed, 0.459s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.480s; elapsed, 0.406s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.580s; elapsed, 0.162s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.860s; elapsed, 0.864s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 31.850s; elapsed, 2.009s, #calls: 1', 'TOTAL : CPU, 86.900s; elapsed, 7.087s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.220s; elapsed, 0.231s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.236s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.046s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 7.200s; elapsed, 0.455s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 28.890s; elapsed, 1.823s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.610s; elapsed, 0.163s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.930s; elapsed, 0.183s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 7.140s; elapsed, 0.447s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.950s; elapsed, 1.941s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.950s; elapsed, 1.941s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.220s; elapsed, 0.231s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.610s; elapsed, 0.163s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.046s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 7.200s; elapsed, 0.455s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 7.140s; elapsed, 0.447s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.930s; elapsed, 0.183s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.236s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 28.890s; elapsed, 1.823s, #calls: 1', 'TOTAL : CPU, 80.710s; elapsed, 5.525s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.240s; elapsed, 0.238s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.232s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.023s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.330s; elapsed, 0.212s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 29.220s; elapsed, 1.835s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.750s; elapsed, 0.109s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.750s; elapsed, 0.109s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.090s; elapsed, 0.194s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 28.990s; elapsed, 1.830s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 28.990s; elapsed, 1.830s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.240s; elapsed, 0.238s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.750s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.023s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.330s; elapsed, 0.212s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.090s; elapsed, 0.194s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.750s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.232s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 29.220s; elapsed, 1.835s, #calls: 1', 'TOTAL : CPU, 68.820s; elapsed, 4.782s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 3.880s; elapsed, 3.883s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 4.090s; elapsed, 4.098s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.050s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 6.870s; elapsed, 0.435s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 37.710s; elapsed, 2.365s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.460s; elapsed, 0.154s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.680s; elapsed, 0.167s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 7.180s; elapsed, 0.450s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 40.580s; elapsed, 2.539s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 40.580s; elapsed, 2.539s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 3.880s; elapsed, 3.883s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.460s; elapsed, 0.154s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.050s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 6.870s; elapsed, 0.435s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 7.180s; elapsed, 0.450s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.680s; elapsed, 0.167s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 4.090s; elapsed, 4.098s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 37.710s; elapsed, 2.365s, #calls: 1', 'TOTAL : CPU, 106.000s; elapsed, 14.140s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 2.020s; elapsed, 2.023s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 2.110s; elapsed, 2.119s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.048s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 6.330s; elapsed, 0.401s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 38.360s; elapsed, 2.411s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.420s; elapsed, 0.151s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.580s; elapsed, 0.161s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.920s; elapsed, 0.434s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 37.560s; elapsed, 2.354s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 37.560s; elapsed, 2.354s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 2.020s; elapsed, 2.023s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.420s; elapsed, 0.151s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.048s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 6.330s; elapsed, 0.401s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.920s; elapsed, 0.434s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.580s; elapsed, 0.161s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 2.110s; elapsed, 2.119s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 38.360s; elapsed, 2.411s, #calls: 1', 'TOTAL : CPU, 98.830s; elapsed, 10.102s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 1.990s; elapsed, 1.994s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 2.120s; elapsed, 2.119s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.024s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.440s; elapsed, 0.220s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 34.260s; elapsed, 2.157s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.770s; elapsed, 0.111s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.670s; elapsed, 0.104s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.980s; elapsed, 0.187s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 37.390s; elapsed, 2.350s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 37.390s; elapsed, 2.350s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 1.990s; elapsed, 1.994s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.770s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.024s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.440s; elapsed, 0.220s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.980s; elapsed, 0.187s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.670s; elapsed, 0.104s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 2.120s; elapsed, 2.119s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 34.260s; elapsed, 2.157s, #calls: 1', 'TOTAL : CPU, 85.860s; elapsed, 9.266s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.830s; elapsed, 0.835s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.870s; elapsed, 0.863s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.540s; elapsed, 0.051s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 7.370s; elapsed, 0.466s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 32.030s; elapsed, 2.025s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.370s; elapsed, 0.148s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.660s; elapsed, 0.166s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.810s; elapsed, 0.426s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 33.250s; elapsed, 2.094s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 33.250s; elapsed, 2.094s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.830s; elapsed, 0.835s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.370s; elapsed, 0.148s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.540s; elapsed, 0.051s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 7.370s; elapsed, 0.466s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.810s; elapsed, 0.426s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.660s; elapsed, 0.166s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.870s; elapsed, 0.863s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 32.030s; elapsed, 2.025s, #calls: 1', 'TOTAL : CPU, 86.730s; elapsed, 7.075s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 3.890s; elapsed, 3.903s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 4.070s; elapsed, 4.073s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.047s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 7.190s; elapsed, 0.454s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 38.310s; elapsed, 2.408s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.510s; elapsed, 0.157s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.690s; elapsed, 0.168s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.390s; elapsed, 0.401s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 41.380s; elapsed, 2.597s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 41.380s; elapsed, 2.597s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 3.890s; elapsed, 3.903s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.510s; elapsed, 0.157s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.047s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 7.190s; elapsed, 0.454s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.390s; elapsed, 0.401s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.690s; elapsed, 0.168s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 4.070s; elapsed, 4.073s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 38.310s; elapsed, 2.408s, #calls: 1', 'TOTAL : CPU, 106.950s; elapsed, 14.208s']
Data Set Size:=5k
Skipping 5k
Data Set Size:=32k
Skipping 32k
In the event of a crash, it is wise to save the resulting data (JSON and Pickle) to disk…
import json, cPickle
with open("OMP_SOLVER_TIME_1k_all.json", "w") as OMP_SOLVER_TIME_FILE:
OMP_SOLVER_TIME_FILE.write(json.dumps(str_out))
with open("OMP_SOLVER_TIME_1k_all.pickle", "w") as OMP_SOLVER_TIME_FILE:
OMP_SOLVER_TIME_FILE.write(cPickle.dumps(str_out))
…which can then be easily reloaded later. Now, we parse the string, extract the timing data, and add it to the python object at the specified thread number index. This allows for easily sorting and plot the data later.
import cPickle
str_out=cPickle.load(open('OMP_SOLVER_TIME_1k_all.pickle', 'rb'))
To simplify indexing later for plots, we create shortened keys. Additionally, we ca also separate the data based upon unique index set and thread numbers.
str_out_sm={}
for k in str_out.keys():
str_out_sm.update({k.replace('levmar.parameter_flags=','').replace('_strum_1k_omp1_params','_'):str_out[k]})
import pandas as pd
u_dat={}
threads_list=[16]
uniq_ref = set([k.split('_')[1] for k in str_out_sm.keys()])
df_list=[]
for u in uniq_ref:
same_t = lambda: None #Functions are objects, so legit
same_t.strum_scotch_a = []
same_t.strum_metis_a = []
same_t.strum_scotch_bi = []
same_t.strum_metis_bi = []
same_t.strum_scotch_pr = []
same_t.strum_metis_pr = []
same_t.eig_llt = []
same_t.eig_ldlt = []
same_t.eig_bicgstab = []
for t in threads_list:
same_t.strum_metis_a.append(str_out_sm["omp%d_%s"%(t,u)][-2].rsplit(',')[-2].strip()[0:-1])
same_t.eig_ldlt.append(str_out_sm["omp%d_%s"%(t,u)][-3].rsplit(',')[-2].strip()[0:-1])
same_t.strum_metis_bi.append(str_out_sm["omp%d_%s"%(t,u)][-4].rsplit(',')[-2].strip()[0:-1])
same_t.strum_scotch_pr.append(str_out_sm["omp%d_%s"%(t,u)][-5].rsplit(',')[-2].strip()[0:-1])
same_t.strum_scotch_a.append(str_out_sm["omp%d_%s"%(t,u)][-6].rsplit(',')[-2].strip()[0:-1])
same_t.eig_bicgstab.append(str_out_sm["omp%d_%s"%(t,u)][-7].rsplit(',')[-2].strip()[0:-1])
same_t.strum_scotch_bi.append(str_out_sm["omp%d_%s"%(t,u)][-8].rsplit(',')[-2].strip()[0:-1])
same_t.eig_llt.append(str_out_sm["omp%d_%s"%(t,u)][-9].rsplit(',')[-2].strip()[0:-1])
same_t.strum_metis_pr.append(str_out_sm["omp%d_%s"%(t,u)][-10].rsplit(',')[-2].strip()[0:-1])
u_dat.update({u:same_t})
str_E_LLT = "EIG_LLT_%s"%u
str_E_LDLT = "EIG_LDLT_%s"%u
str_E_BICGSTAB = "EIG_BICGSTAB_%s"%u
str_S_SCOTCH_A = "STRUM_SCOTCHA_%s"%u
str_S_METIS_A = "STRUM_METISA_%s"%u
str_S_SCOTCH_BI = "STRUM_SCOTCHBI_%s"%u
str_S_METIS_BI = "STRUM_METISBI_%s"%u
str_S_SCOTCH_PR = "STRUM_SCOTCHPR_%s"%u
str_S_METIS_PR = "STRUM_METISPR_%s"%u
df_list.append( pd.DataFrame({str_E_LLT:same_t.eig_llt, str_E_LDLT:same_t.eig_ldlt, str_E_BICGSTAB:same_t.eig_bicgstab, str_S_SCOTCH_A:same_t.strum_scotch_a, str_S_METIS_A:same_t.strum_metis_a, str_S_SCOTCH_BI:same_t.strum_scotch_bi, str_S_METIS_BI:same_t.strum_metis_bi, str_S_SCOTCH_PR:same_t.strum_scotch_pr, str_S_METIS_PR:same_t.strum_metis_pr }, index=threads_list).transpose() )
We now have a list of Pandas Dataframes, with which we can combine to a single entity. We can then proceed to plot the resulting columns, comparing both Eigen and STRUMPACK for each specific set of refined parameters.
perf_s = pd.concat(df_list).transpose()
perf_s.transpose()
—CLICK FOR OUTPUT—
| 16 | |
|---|---|
| EIG_BICGSTAB_Rxy-Deff-Eta- | 0.024 |
| EIG_LDLT_Rxy-Deff-Eta- | 2.119 |
| EIG_LLT_Rxy-Deff-Eta- | 1.994 |
| STRUM_METISA_Rxy-Deff-Eta- | 2.157 |
| STRUM_METISBI_Rxy-Deff-Eta- | 0.104 |
| STRUM_METISPR_Rxy-Deff-Eta- | 2.350 |
| STRUM_SCOTCHA_Rxy-Deff-Eta- | 0.220 |
| STRUM_SCOTCHBI_Rxy-Deff-Eta- | 0.111 |
| STRUM_SCOTCHPR_Rxy-Deff-Eta- | 0.187 |
| EIG_BICGSTAB_Bfactor- | 0.049 |
| EIG_LDLT_Bfactor- | 0.238 |
| EIG_LLT_Bfactor- | 0.232 |
| STRUM_METISA_Bfactor- | 1.831 |
| STRUM_METISBI_Bfactor- | 0.177 |
| STRUM_METISPR_Bfactor- | 2.001 |
| STRUM_SCOTCHA_Bfactor- | 0.447 |
| STRUM_SCOTCHBI_Bfactor- | 0.169 |
| STRUM_SCOTCHPR_Bfactor- | 0.422 |
| EIG_BICGSTAB_Bfactor-Deff-Eta- | 0.051 |
| EIG_LDLT_Bfactor-Deff-Eta- | 0.863 |
| EIG_LLT_Bfactor-Deff-Eta- | 0.835 |
| STRUM_METISA_Bfactor-Deff-Eta- | 2.025 |
| STRUM_METISBI_Bfactor-Deff-Eta- | 0.166 |
| STRUM_METISPR_Bfactor-Deff-Eta- | 2.094 |
| STRUM_SCOTCHA_Bfactor-Deff-Eta- | 0.466 |
| STRUM_SCOTCHBI_Bfactor-Deff-Eta- | 0.148 |
| STRUM_SCOTCHPR_Bfactor-Deff-Eta- | 0.426 |
| EIG_BICGSTAB_Rxy-Bfactor-Deff-Eta- | 0.047 |
| EIG_LDLT_Rxy-Bfactor-Deff-Eta- | 4.073 |
| EIG_LLT_Rxy-Bfactor-Deff-Eta- | 3.903 |
| ... | ... |
| STRUM_SCOTCHA_Deff- | 0.215 |
| STRUM_SCOTCHBI_Deff- | 0.109 |
| STRUM_SCOTCHPR_Deff- | 0.192 |
| EIG_BICGSTAB_Rxy-Bfactor- | 0.048 |
| EIG_LDLT_Rxy-Bfactor- | 2.120 |
| EIG_LLT_Rxy-Bfactor- | 1.995 |
| STRUM_METISA_Rxy-Bfactor- | 2.191 |
| STRUM_METISBI_Rxy-Bfactor- | 0.163 |
| STRUM_METISPR_Rxy-Bfactor- | 2.391 |
| STRUM_SCOTCHA_Rxy-Bfactor- | 0.417 |
| STRUM_SCOTCHBI_Rxy-Bfactor- | 0.153 |
| STRUM_SCOTCHPR_Rxy-Bfactor- | 0.433 |
| EIG_BICGSTAB_Rxy-Bfactor-Eta- | 0.048 |
| EIG_LDLT_Rxy-Bfactor-Eta- | 2.119 |
| EIG_LLT_Rxy-Bfactor-Eta- | 2.023 |
| STRUM_METISA_Rxy-Bfactor-Eta- | 2.411 |
| STRUM_METISBI_Rxy-Bfactor-Eta- | 0.161 |
| STRUM_METISPR_Rxy-Bfactor-Eta- | 2.354 |
| STRUM_SCOTCHA_Rxy-Bfactor-Eta- | 0.401 |
| STRUM_SCOTCHBI_Rxy-Bfactor-Eta- | 0.151 |
| STRUM_SCOTCHPR_Rxy-Bfactor-Eta- | 0.434 |
| EIG_BICGSTAB_Rxy-Bfactor-Deff- | 0.050 |
| EIG_LDLT_Rxy-Bfactor-Deff- | 4.098 |
| EIG_LLT_Rxy-Bfactor-Deff- | 3.883 |
| STRUM_METISA_Rxy-Bfactor-Deff- | 2.365 |
| STRUM_METISBI_Rxy-Bfactor-Deff- | 0.167 |
| STRUM_METISPR_Rxy-Bfactor-Deff- | 2.539 |
| STRUM_SCOTCHA_Rxy-Bfactor-Deff- | 0.435 |
| STRUM_SCOTCHBI_Rxy-Bfactor-Deff- | 0.154 |
| STRUM_SCOTCHPR_Rxy-Bfactor-Deff- | 0.450 |
135 rows × 1 columns
With the performance data in an easily accessible format, we can plot each respective data set for all solvers. Each bar corresponds to the parameters being refined, where the colour indicates the respective solver used, and the y-axis is the time to solve the system in seconds.
# define the figure size and grid layout properties
matplotlib.rcParams.update({'font.size': 22})
figsize = (20, 16)
cols = 4
fig3 = plt.figure(num=3, figsize=figsize)
ax3 = plt.axes()
ii = []
uu=[]
keys=['EIG_LLT','EIG_LDLT','STRUM_METISA','STRUM_SCOTCHA','STRUM_METISBI','STRUM_SCOTCHBI','STRUM_METISPR','STRUM_SCOTCHPR','EIG_BICGSTAB']
colours = ['#a6cee3','#1f78b4','#b2df8a','#33a02c','#fb9a99','#e31a1c','#fdbf6f','#ff7f00','#cab2d6']
key_col = {k:v for (k,v) in zip(keys,colours)}
hatch_pattern = [ '/' , '\\' , '|' , '-' , '+' , 'x' ,'o' ,'O' , '.', '*']
for i, u in enumerate(uniq_ref):
p_list=[]
ax3.set_ylabel('time [s]')
keys_df = [k+'_'+u for k in keys]
dfl=perf_s[keys_df]
m = zip(dfl.as_matrix().astype(np.float)[0], dfl.keys().tolist() )
m = sorted(m,key=lambda x: x[0])
for p in reversed(xrange(len(m))):
c_label = "_".join(m[p][1].split('_')[0:2])
p_list.append( plt.bar(i, m[p][0], label=m[p][1], color=key_col[c_label], edgecolor='k') )
ii.append(i)
uu.append(u)
ax3.set_yscale('log',basey=10)
plt.title('1k frames OMP_NUM_THREADS=16', fontsize=24)
plt.xticks(ii, uu)
plt.setp(ax3.get_xticklabels(), rotation=90, ha='left')
ax3.legend(p_list,keys,bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.,)
plt.savefig('omp_1kframes_allsolvers_omp16.pdf', pad_inches=10)
 |
So far we have examined the single threaded and OpenMP enabled solver performance. Strumpack also allows us to use the distributed MPI backend. We can set this up as follows using the ipyparallel backend, or invoke an mpirun call using the cell bash bash environment. For completeness, the ipyparallel environment can be set up as follows:
conda install ipyparallel
jupyter serverextension enable --py ipyparallel --user
jupyter nbextension install --py ipyparallel --user
jupyter nbextension enable --py ipyparallel --user
ipcluster start --profile=mpi -n 4
Take note of the controller client json file <HOME>/.ipython/profile_<PROFILE_NAME>/security/ipcontroller-client.json. This will be used to allow the notebook to connect to the ipyparallel cluster profile mpi. We can then return the handle of the cluster environment with:
import ipyparallel as ipp
#c = ipp.Client('/net/cci-filer2/raid1/home/mlxd/.ipython/profile_mpi/security/ipcontroller-client.json')
c = ipp.Client()
rc=c[:]
c.ids #Instance IDs
[0]
For our purposes it can be easier to call the solver using the ! bash environment, as was used for the OpenMP studies earlier. A simple test script can be performed as follows, which tests the listed solvers against the MPI backend.
!OMP_NUM_THREADS=1 mpirun -np 2 libtbx.python \
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/\
modules/cctbx_project/scitbx/examples/bevington/strumpack_eigen_solver_mpi_dist.py \
/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/\
out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv \
/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/\
out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.
try: mod = __import__(name)
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.
try: mod = __import__(name)
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.
try: mod = __import__(name)
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.
try: mod = __import__(name)
14151 7075.5
# ***** WARNING ****************************************************
# Detected a large number of levels in the frontal/elimination tree.
# STRUMPACK currently does not handle this safely, which
# could lead to segmentation faults due to stack overflows.
# As a remedy, you can try to increase the stack size,
# or try a different ordering (metis, scotch, ..).
# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,
# (enabled by default) iso --sp_enable_METIS_NodeND.
# ******************************************************************
individual call time for STRUMPACK_OMP: CPU, 0.350s; elapsed, 0.344s
individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.050s
individual call time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.027s
CPP -2
# Initializing STRUMPACK
# using 1 OpenMP thread
CPP -2
# using 2 MPI processes
# matching job: maximum matching with row and column scaling
# initial matrix:
# - number of unknowns = 14,151
# - number of nonzeros = 72,305
# nested dissection reordering:
# - Scotch reordering
# - strategy parameter = 8
# - number of separators = 4,075
# - number of levels = 68
# ***** WARNING ****************************************************
# Detected a large number of levels in the frontal/elimination tree.
# STRUMPACK currently does not handle this safely, which
# could lead to segmentation faults due to stack overflows.
# As a remedy, you can try to increase the stack size,
# or try a different ordering (metis, scotch, ..).
# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,
# (enabled by default) iso --sp_enable_METIS_NodeND.
# ******************************************************************
# - nd time = 0.0810261
# - matching time = 0.0116782
# - symmetrization time = 0.00337219
# symbolic factorization:
# - nr of dense Frontal matrices = 4,075
# - nr of HSS Frontal matrices = 0
# - symb-factor time = 0.021524
# multifrontal factorization:
# - estimated memory usage (exact solver) = 7.70359 MB
# - factor time = 0.223322
# - factor nonzeros = 962,949
# - factor memory = 7.70359 MB
# - total flops = 2.49112e+08, min = 3.46562e+07, max = 2.14456e+08
# - flop rate = 1.11548 GFlop/s
# - factor memory/nonzeros = 100 % of multifrontal
# - maximum HSS rank = 0
# - HSS compression = false
# - relative compression tolerance = 0.01
# - absolute compression tolerance = 1e-08
# - normal(0,1) distribution with minstd_rand engine
# ----- FLOP BREAKDOWN ---------------------
# compression = 0
# random = 0
# ID = 0
# QR = 0
# ortho = 0
# reduce_samples = 0
# update_samples = 0
# extraction = 0
# sampling = 0
# CB_sample = 0
# sparse_sampling = 0
# ULV_factor = 0
# Schur = 0
# full_rank = 2.63271e+08
# --------------------------------------------
# total = 2.63271e+08
# --------------------------------------------
REFINEMENT it. 0 res = 15322 rel.res = 1 bw.error = 1
REFINEMENT it. 1 res = 1.1079e-11 rel.res = 7.23077e-16 bw.error = 1.25632e-15
# DIRECT/GMRES solve:
# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000
# - number of Krylov iterations = 1
# - solve time = 0.011821
# - total flops = 2.47602e+06, min = 838849, max = 1.63717e+06
# - flop rate = 0.209459 GFlop/s
# - bytes moved = 21.8361 MB, min = 0 MB, max = 0 MB
# - byte rate = 1.84722 GByte/s
# - solve arithmetic intensity = 0.113391 flop/byte
individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.350s; elapsed, 0.359s
individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.370s; elapsed, 0.365s
Exiting profiler
time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.350s; elapsed, 0.359s, #calls: 1
TOTAL : CPU, 0.350s; elapsed, 0.359s
Strumpack solutions agree with Eigen
Exiting profiler
time for STRUMPACK_OMP: CPU, 0.350s; elapsed, 0.344s, #calls: 1
time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.027s, #calls: 1
time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.050s, #calls: 1
time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.370s; elapsed, 0.365s, #calls: 1
TOTAL : CPU, 0.790s; elapsed, 0.786s
We now use the same procedure of creating a solver file, write to disk, and call with the respective matrix arguments with OMP_NUM_THREADS and mpirun -n. For simplicity, we examine the OpenMP STRUM_SCOTCHA, EIGEN_LDLT and EIGEN_BICGSTAB, compared with the MPI-enabled STRUM_SCOTCHA (we may also examine the full range of algorithms as used in the previous example).
MPI_SOLVER='''
from __future__ import division
import mpi4py
mpi4py.rc.threads = True
mpi4py.rc.thread_level = "funneled"
from mpi4py import MPI
assert MPI.Is_initialized()
assert MPI.Query_thread() == MPI.THREAD_FUNNELED
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size()
from scitbx.matrix import sqr,col
from cctbx.array_family import flex
from libtbx.test_utils import approx_equal
from libtbx.development.timers import Profiler
import boost.python
ext_omp = boost.python.import_ext("scitbx_examples_strumpack_solver_ext")
ext_mpi = boost.python.import_ext("scitbx_examples_strumpack_mpi_dist_solver_ext")
import sys
import numpy as np
import scipy.sparse as sps
if rank==0:
A_mat = np.loadtxt(sys.argv[1],dtype={'names':('rows','cols','vals'),'formats':('i8','i8','f8')})
b_vec = np.loadtxt(sys.argv[2])
n_rows = len(b_vec)
n_cols = n_rows
nnz = len(A_mat['vals'])
print n_rows, n_rows/size
#Convert the sparse CSR to flex doubles, then use them to solve using the implemented framework
A_sp = sps.csr_matrix((A_mat['vals'],(A_mat['rows'],A_mat['cols'])))
#Check if upper/lower triangular only, and generate full if so
tu=sps.triu(A_sp)
tl=sps.tril(A_sp)
sd=sps.diags(A_sp.diagonal())
A_spS = A_sp
if tu.nnz == sd.getnnz() or tl.nnz == sd.getnnz():
A_spS = A_sp + A_sp.transpose() - sd
import numpy as np
row_idx_split = np.array_split(np.arange(n_rows),size)
len_row_idx_split = flex.int( np.cumsum( np.append([0], [len(i) for i in row_idx_split]) ).tolist() )
P = Profiler("STRUMPACK_OMP")
A_indptr = flex.int( A_spS.indptr )
A_indices = flex.int( A_spS.indices )
A_data = flex.double( A_spS.data )
b = flex.double( b_vec )
res_strum_omp = ext_omp.strumpack_solver(
n_rows, n_cols,
A_indptr, A_indices,
A_data, b,
ext_omp.scotch,
ext_omp.auto
)
del P
P = Profiler("EIGEN_LDLT")
res_eig_ldlt = ext_omp.eigen_solver(1, n_rows, n_cols, A_indptr, A_indices, A_data, b)
del P
P = Profiler("EIGEN_BICGSTAB")
res_eig_bicgstab = ext_omp.eigen_solver(2, n_rows, n_cols, A_indptr, A_indices, A_data, b)
del P
else:
A_spS=None
row_idx_split=None
len_row_idx_split=None
b_vec=None
n_cols=None
if size>1:
#Broadcast data to each rank
A_spS = comm.bcast(A_spS, root=0)
row_idx_split = comm.bcast(row_idx_split, root=0)
len_row_idx_split = comm.bcast(len_row_idx_split, root=0)
b_vec = comm.bcast(b_vec, root=0)
n_cols = comm.bcast(n_cols, root=0)
#Take subset of data for each rank
A_row_offset = flex.int(A_spS[row_idx_split[rank],:].indptr)
A_col_offset = flex.int(A_spS[row_idx_split[rank],:].indices)
A_values = flex.double(A_spS[row_idx_split[rank],:].data)
b = flex.double(b_vec[row_idx_split[rank]])
P = Profiler("STRUMPACK_MPI_DIST_RANK=%d"%rank)
res_strum_mpi_local = ext_mpi.strumpack_mpi_dist_solver(len(row_idx_split[rank]), n_cols, comm, A_row_offset, A_col_offset, A_values, b, len_row_idx_split, ext_mpi.scotch, ext_mpi.auto)
strum_result_mpi_list = comm.gather(res_strum_mpi_local.x, root=0)
del P
if rank==0:
strum_result_mpi = flex.double()
for l in strum_result_mpi_list:
strum_result_mpi.extend(l)
#MPI.Finalize()
if rank==0:
strum_result_omp = res_strum_omp.x
eig_result_ldlt = res_eig_ldlt.x
eig_result_bicgstab = res_eig_bicgstab.x
for ii in xrange(len(strum_result_omp)):
assert approx_equal( strum_result_omp[ii], eig_result_ldlt[ii] )
assert approx_equal( strum_result_omp[ii], eig_result_bicgstab[ii] )
if size>1:
assert approx_equal( strum_result_omp[ii], strum_result_mpi[ii] )
print "Strumpack solutions agree with Eigen"
'''
MPI_SOLVER_FILE = open("MPI_SOLVER.py", "w")
MPI_SOLVER_FILE.write(MPI_SOLVER)
MPI_SOLVER_FILE.close()
DATAPATH="/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/"
A_LIST = !find {DATAPATH} -iname "A*.csv"
B_LIST = [ii.replace('/A_','/b_') for ii in A_LIST]
We intend to examine performance while varying both OMP_NUM_THREADS and the MPI process number to determine an optimal set of parameters for the distributed backend. In the following example we test between 1 to 16 OpenMP threads per process, and from 1 to 16 MPI processes, assuming a total of threads*processes <= 32 cores. The resulting data set will be a scalar field of time as a function of both variables, which we can plot as a heatmap. We can begin by running the analysis on a 1000 image dataset.
str_out={}
import os
threads_list = [1,2,4,8,16]
mpi_proc_list = [1,2,4,8,16]
for imgs_size in list_idx:
print "Data Set Size:=%s"%imgs_size
#Subselect the smallest data size for now
if imgs_size != "10k":
print "Skipping %s"%imgs_size
continue
for imgs_idx in list_idx[imgs_size]:
A_path = A_LIST[imgs_idx]; b_path = B_LIST[imgs_idx]
dat_name = A_path.split('/')[-1][2:-4]
print "Data Set Name:=%s"%(dat_name)
#Ensure the A and b data are matched correctly
assert(os.path.dirname(A_path) == os.path.dirname(b_path))
print {A_path}, {b_path}
for procs in mpi_proc_list:
for threads in threads_list:
if threads*procs <= 32:
print "mpirun -n %d"%procs
print "OMP_NUM_THREADS:=%d"%threads
val = !OMP_NUM_THREADS={threads} mpirun -n {procs} libtbx.python ./MPI_SOLVER.py {A_path} {b_path}
print val
for s in val:
if "assert" in s:
raise Exception("Solutions not equal! Halting with log: %s"%val)
key = 'mpi' + str(procs) + 'omp' + str(threads) + '_' + dat_name
str_out.update({key:val})
—CLICK FOR OUTPUT—
Data Set Size:=10k
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Rxy-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-/A_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-/b_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 31.130s; elapsed, 31.154s', 'individual call time for EIGEN_LDLT: CPU, 37.630s; elapsed, 37.656s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.117s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 31.130s; elapsed, 31.154s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.117s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.630s; elapsed, 37.656s, #calls: 1', 'TOTAL : CPU, 68.880s; elapsed, 68.927s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 35.010s; elapsed, 17.742s', 'individual call time for EIGEN_LDLT: CPU, 37.800s; elapsed, 37.629s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.120s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 35.010s; elapsed, 17.742s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.120s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.800s; elapsed, 37.629s, #calls: 1', 'TOTAL : CPU, 72.980s; elapsed, 55.492s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 58.500s; elapsed, 14.890s', 'individual call time for EIGEN_LDLT: CPU, 38.240s; elapsed, 37.722s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.099s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 58.500s; elapsed, 14.890s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.099s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.240s; elapsed, 37.722s, #calls: 1', 'TOTAL : CPU, 96.920s; elapsed, 52.711s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 114.020s; elapsed, 14.496s', 'individual call time for EIGEN_LDLT: CPU, 38.940s; elapsed, 37.725s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.094s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 114.020s; elapsed, 14.496s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.094s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.940s; elapsed, 37.725s, #calls: 1', 'TOTAL : CPU, 153.230s; elapsed, 52.315s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 305.110s; elapsed, 19.364s', 'individual call time for EIGEN_LDLT: CPU, 40.400s; elapsed, 37.744s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.440s; elapsed, 0.098s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 305.110s; elapsed, 19.364s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.440s; elapsed, 0.098s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.400s; elapsed, 37.744s, #calls: 1', 'TOTAL : CPU, 345.950s; elapsed, 57.206s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 31.410s; elapsed, 31.429s', 'individual call time for EIGEN_LDLT: CPU, 37.660s; elapsed, 37.677s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.116s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.423636', '# - matching time = 0.0671489', '# - symmetrization time = 0.051945', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.084671', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 30.2248', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35496e+10, min = 41770, max = 9.35496e+10', '# - flop rate = 3.09512 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.60414e+10', '# --------------------------------------------', '# total = 9.60414e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.73537e-11\trel.res = 1.86806e-15\tbw.error = 5.7415e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.132668', '# - total flops = 6.72147e+07, min = 165052, max = 6.70497e+07', '# - flop rate = 0.506638 GFlop/s', '# - bytes moved = 408.126 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.07629 GByte/s', '# - solve arithmetic intensity = 0.164691 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 31.050s; elapsed, 31.068s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 31.070s; elapsed, 31.079s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 31.050s; elapsed, 31.068s, #calls: 1', 'TOTAL : CPU, 31.050s; elapsed, 31.068s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 31.410s; elapsed, 31.429s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.116s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.660s; elapsed, 37.677s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 31.070s; elapsed, 31.079s, #calls: 1', 'TOTAL : CPU, 100.260s; elapsed, 100.301s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 35.110s; elapsed, 17.808s', 'individual call time for EIGEN_LDLT: CPU, 37.740s; elapsed, 37.569s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.105s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 130', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.420951', '# - matching time = 0.068316', '# - symmetrization time = 0.051847', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0771451', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 16.5968', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35498e+10, min = 41770, max = 9.35498e+10', '# - flop rate = 5.63661 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.60414e+10', '# --------------------------------------------', '# total = 9.60414e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.80099e-11\trel.res = 1.88391e-15\tbw.error = 5.74116e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.116206', '# - total flops = 6.72328e+07, min = 165052, max = 6.70678e+07', '# - flop rate = 0.578566 GFlop/s', '# - bytes moved = 408.883 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.5186 GByte/s', '# - solve arithmetic intensity = 0.164431 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 34.470s; elapsed, 17.417s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.180s; elapsed, 17.429s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 34.470s; elapsed, 17.417s, #calls: 1', 'TOTAL : CPU, 34.470s; elapsed, 17.417s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 35.110s; elapsed, 17.808s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.105s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.740s; elapsed, 37.569s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.180s; elapsed, 17.429s, #calls: 1', 'TOTAL : CPU, 91.170s; elapsed, 72.911s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 63.980s; elapsed, 16.291s', 'individual call time for EIGEN_LDLT: CPU, 39.450s; elapsed, 38.939s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.106s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.463055', '# - matching time = 0.0712121', '# - symmetrization time = 0.0550501', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0739958', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 16.3268', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35498e+10, min = 41770, max = 9.35498e+10', '# - flop rate = 5.72984 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.60414e+10', '# --------------------------------------------', '# total = 9.60414e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.79581e-11\trel.res = 1.88266e-15\tbw.error = 5.74116e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.197363', '# - total flops = 6.72328e+07, min = 165052, max = 6.70678e+07', '# - flop rate = 0.340656 GFlop/s', '# - bytes moved = 408.902 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.07183 GByte/s', '# - solve arithmetic intensity = 0.164423 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 67.860s; elapsed, 17.288s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.790s; elapsed, 17.298s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 67.860s; elapsed, 17.288s, #calls: 1', 'TOTAL : CPU, 67.860s; elapsed, 17.288s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 63.980s; elapsed, 16.291s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.106s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.450s; elapsed, 38.939s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.790s; elapsed, 17.298s, #calls: 1', 'TOTAL : CPU, 123.440s; elapsed, 72.635s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 120.850s; elapsed, 15.440s', 'individual call time for EIGEN_LDLT: CPU, 59.540s; elapsed, 58.326s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.113s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.44408', '# - matching time = 0.087352', '# - symmetrization time = 0.0665729', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0964291', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 14.9592', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35498e+10, min = 42535, max = 9.35498e+10', '# - flop rate = 6.25367 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.60414e+10', '# --------------------------------------------', '# total = 9.60414e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.79316e-11\trel.res = 1.88202e-15\tbw.error = 5.74116e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.198861', '# - total flops = 6.72328e+07, min = 165052, max = 6.70678e+07', '# - flop rate = 0.338089 GFlop/s', '# - bytes moved = 408.94 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.05641 GByte/s', '# - solve arithmetic intensity = 0.164408 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 125.210s; elapsed, 15.962s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.160s; elapsed, 15.972s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 125.210s; elapsed, 15.962s, #calls: 1', 'TOTAL : CPU, 125.210s; elapsed, 15.962s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 120.850s; elapsed, 15.440s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.113s, #calls: 1', 'time for EIGEN_LDLT: CPU, 59.540s; elapsed, 58.326s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.160s; elapsed, 15.972s, #calls: 1', 'TOTAL : CPU, 202.850s; elapsed, 89.851s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 250.560s; elapsed, 15.947s', 'individual call time for EIGEN_LDLT: CPU, 40.520s; elapsed, 37.817s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.420s; elapsed, 0.094s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7CPP -2', ' = log_2(#threads) + 3', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.445719', '# - matching time = 0.0899839', '# - symmetrization time = 0.0637009', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0908601', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 16.9512', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35508e+10, min = 43208, max = 9.35507e+10', '# - flop rate = 5.51884 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.60414e+10', '# --------------------------------------------', '# total = 9.60414e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.68634e-11\trel.res = 1.85622e-15\tbw.error = 5.74638e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.198701', '# - total flops = 6.72375e+07, min = 165052, max = 6.70724e+07', '# - flop rate = 0.338385 GFlop/s', '# - bytes moved = 409.091 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.05883 GByte/s', '# - solve arithmetic intensity = 0.164358 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 281.170s; elapsed, 17.946s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 31.240s; elapsed, 17.957s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 281.170s; elapsed, 17.946s, #calls: 1', 'TOTAL : CPU, 281.170s; elapsed, 17.946s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 250.560s; elapsed, 15.947s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.420s; elapsed, 0.094s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.520s; elapsed, 37.817s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 31.240s; elapsed, 17.957s, #calls: 1', 'TOTAL : CPU, 322.740s; elapsed, 71.815s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 37.500s; elapsed, 37.561s', 'individual call time for EIGEN_LDLT: CPU, 37.950s; elapsed, 37.984s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.122s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.452391', '# - matching time = 0.070359', '# - symmetrization time = 0.0538969', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0570049', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 17.0081', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35496e+10, min = 41770, max = 7.04191e+10', '# - flop rate = 5.5003 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.59508e+10', '# --------------------------------------------', '# total = 9.59508e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.73296e-11\trel.res = 1.86748e-15\tbw.error = 5.73834e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.10983', '# - total flops = 7.09444e+07, min = 127142, max = 5.09433e+07', '# - flop rate = 0.645948 GFlop/s', '# - bytes moved = 217.757 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.98267 GByte/s', '# - solve arithmetic intensity = 0.325796 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.800s; elapsed, 17.815s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.780s; elapsed, 17.793s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.780s; elapsed, 17.807s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.810s; elapsed, 17.814s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.780s; elapsed, 17.807s, #calls: 1', 'TOTAL : CPU, 17.780s; elapsed, 17.807s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.780s; elapsed, 17.793s, #calls: 1', 'TOTAL : CPU, 17.780s; elapsed, 17.793s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.800s; elapsed, 17.815s, #calls: 1', 'TOTAL : CPU, 17.800s; elapsed, 17.815s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 37.500s; elapsed, 37.561s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.122s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.950s; elapsed, 37.984s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.810s; elapsed, 17.814s, #calls: 1', 'TOTAL : CPU, 93.380s; elapsed, 93.481s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 36.260s; elapsed, 18.367s', 'individual call time for EIGEN_LDLT: CPU, 38.970s; elapsed, 38.812s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.109s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.445381', '# - matching time = 0.073508', '# - symmetrization time = 0.0528131', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.059128', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 17.3075', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35496e+10, min = 41770, max = 7.04191e+10', '# - flop rate = 5.40515 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.59508e+10', '# --------------------------------------------', '# total = 9.59508e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.71926e-11\trel.res = 1.86417e-15\tbw.error = 5.74381e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.107487', '# - total flops = 7.09444e+07, min = 127142, max = 5.09433e+07', '# - flop rate = 0.660028 GFlop/s', '# - bytes moved = 217.769 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.02601 GByte/s', '# - solve arithmetic intensity = 0.325778 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 25.390s; elapsed, 18.116s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 27.570s; elapsed, 18.094s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 35.550s; elapsed, 18.109s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.810s; elapsed, 18.114s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 35.550s; elapsed, 18.109s, #calls: 1', 'TOTAL : CPU, 35.550s; elapsed, 18.109s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 25.390s; elapsed, 18.116s, #calls: 1', 'TOTAL : CPU, 25.390s; elapsed, 18.116s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 27.570s; elapsed, 18.094s, #calls: 1', 'TOTAL : CPU, 27.570s; elapsed, 18.094s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 36.260s; elapsed, 18.367s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.970s; elapsed, 38.812s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.810s; elapsed, 18.114s, #calls: 1', 'TOTAL : CPU, 94.180s; elapsed, 75.402s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 69.500s; elapsed, 17.692s', 'individual call time for EIGEN_LDLT: CPU, 38.230s; elapsed, 37.698s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.093s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.460802', '# - matching time = 0.0799248', '# - symmetrization time = 0.0598788', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0779431', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 16.5414', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35496e+10, min = 41770, max = 7.04191e+10', '# - flop rate = 5.65547 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.59508e+10', '# --------------------------------------------', '# total = 9.59508e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.7193e-11\trel.res = 1.86418e-15\tbw.error = 5.74381e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.112011', '# - total flops = 7.09444e+07, min = 127142, max = 5.09433e+07', '# - flop rate = 0.633368 GFlop/s', '# - bytes moved = 217.784 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.94431 GByte/s', '# - solve arithmetic intensity = 0.325755 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 37.940s; elapsed, 17.404s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 48.640s; elapsed, 17.384s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 67.640s; elapsed, 17.396s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.600s; elapsed, 17.403s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 67.640s; elapsed, 17.396s, #calls: 1', 'TOTAL : CPU, 67.640s; elapsed, 17.396s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 48.640s; elapsed, 17.384s, #calls: 1', 'TOTAL : CPU, 48.640s; elapsed, 17.384s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 37.940s; elapsed, 17.404s, #calls: 1', 'TOTAL : CPU, 37.940s; elapsed, 17.404s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 69.500s; elapsed, 17.692s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.093s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.230s; elapsed, 37.698s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.600s; elapsed, 17.403s, #calls: 1', 'TOTAL : CPU, 127.500s; elapsed, 72.886s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 111.720s; elapsed, 14.239s', 'individual call time for EIGEN_LDLT: CPU, 40.080s; elapsed, 38.861s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.094s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.491927', '# - matching time = 0.0973151', '# - symmetrization time = 0.0595419', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0807052', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 15.3578', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35496e+10, min = 42535, max = 7.04191e+10', '# - flop rate = 6.09135 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.59508e+10', '# --------------------------------------------', '# total = 9.59508e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.71813e-11\trel.res = 1.8639e-15\tbw.error = 5.74928e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.136156', '# - total flops = 7.09444e+07, min = 127142, max = 5.09433e+07', '# - flop rate = 0.521053 GFlop/s', '# - bytes moved = 217.815 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.59975 GByte/s', '# - solve arithmetic intensity = 0.325709 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 60.840s; elapsed, 16.297s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 91.810s; elapsed, 16.276s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 125.580s; elapsed, 16.290s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 21.650s; elapsed, 16.296s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 125.580s; elapsed, 16.290s, #calls: 1', 'TOTAL : CPU, 125.580s; elapsed, 16.290s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 91.810s; elapsed, 16.276s, #calls: 1', 'TOTAL : CPU, 91.810s; elapsed, 16.276s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 60.840s; elapsed, 16.297s, #calls: 1', 'TOTAL : CPU, 60.840s; elapsed, 16.297s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 111.720s; elapsed, 14.239s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.094s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.080s; elapsed, 38.861s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 21.650s; elapsed, 16.296s, #calls: 1', 'TOTAL : CPU, 173.710s; elapsed, 69.489s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 3790.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.540s; elapsed, 32.559s', 'individual call time for EIGEN_LDLT: CPU, 38.210s; elapsed, 38.240s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.120s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,171', '# - number of levels = 128', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.443409', '# - matching time = 0.0673399', '# - symmetrization time = 0.0424738', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0652878', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.518 MB', '# - factor time = 17.2776', '# - factor nonzeros = 31,314,729', '# - factor memory = 250.518 MB', '# - total flops = 9.5061e+10, min = 41770, max = 7.01469e+10', '# - flop rate = 5.50197 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.72559e+10', '# --------------------------------------------', '# total = 9.72559e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.01663e-10\trel.res = 2.45512e-15\tbw.error = 4.56971e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.106128', '# - total flops = 8.12073e+07, min = 108187, max = 5.0687e+07', '# - flop rate = 0.765183 GFlop/s', '# - bytes moved = 211.022 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.98838 GByte/s', '# - solve arithmetic intensity = 0.384828 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 18.020s; elapsed, 18.062s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 18.030s; elapsed, 18.064s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.040s; elapsed, 18.059s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 18.040s; elapsed, 18.056s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.990s; elapsed, 18.046s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.030s; elapsed, 18.057s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 18.030s; elapsed, 18.058s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.070s; elapsed, 18.065s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 18.030s; elapsed, 18.064s, #calls: 1', 'TOTAL : CPU, 18.030s; elapsed, 18.064s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 18.020s; elapsed, 18.062s, #calls: 1', 'TOTAL : CPU, 18.020s; elapsed, 18.062s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.990s; elapsed, 18.046s, #calls: 1', 'TOTAL : CPU, 17.990s; elapsed, 18.046s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.030s; elapsed, 18.057s, #calls: 1', 'TOTAL : CPU, 18.030s; elapsed, 18.057s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 18.040s; elapsed, 18.056s, #calls: 1', 'TOTAL : CPU, 18.040s; elapsed, 18.056s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.040s; elapsed, 18.059s, #calls: 1', 'TOTAL : CPU, 18.040s; elapsed, 18.059s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 18.030s; elapsed, 18.058s, #calls: 1', 'TOTAL : CPU, 18.030s; elapsed, 18.058s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.540s; elapsed, 32.559s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.120s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.210s; elapsed, 38.240s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.070s; elapsed, 18.065s, #calls: 1', 'TOTAL : CPU, 88.940s; elapsed, 88.984s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 3790.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 40.480s; elapsed, 20.502s', 'individual call time for EIGEN_LDLT: CPU, 37.870s; elapsed, 37.694s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.108s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,171', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.465065', '# - matching time = 0.079277', '# - symmetrization time = 0.039073', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.08726', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.518 MB', '# - factor time = 14.0964', '# - factor nonzeros = 31,314,729', '# - factor memory = 250.518 MB', '# - total flops = 9.5061e+10, min = 41770, max = 7.01469e+10', '# - flop rate = 6.74364 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.72559e+10', '# --------------------------------------------', '# total = 9.72559e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.01627e-10\trel.res = 2.45425e-15\tbw.error = 4.56971e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0849791', '# - total flops = 8.12073e+07, min = 108187, max = 5.0687e+07', '# - flop rate = 0.955615 GFlop/s', '# - bytes moved = 211.031 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.48333 GByte/s', '# - solve arithmetic intensity = 0.384812 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 21.130s; elapsed, 14.914s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 19.200s; elapsed, 14.914s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.150s; elapsed, 14.908s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 28.870s; elapsed, 14.911s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.120s; elapsed, 14.909s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.760s; elapsed, 14.910s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 23.480s; elapsed, 14.899s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 15.620s; elapsed, 14.917s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 28.870s; elapsed, 14.911s, #calls: 1', 'TOTAL : CPU, 28.870s; elapsed, 14.911s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.120s; elapsed, 14.909s, #calls: 1', 'TOTAL : CPU, 17.120s; elapsed, 14.909s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 23.480s; elapsed, 14.899s, #calls: 1', 'TOTAL : CPU, 23.480s; elapsed, 14.899s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 19.200s; elapsed, 14.914s, #calls: 1', 'TOTAL : CPU, 19.200s; elapsed, 14.914s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.150s; elapsed, 14.908s, #calls: 1', 'TOTAL : CPU, 17.150s; elapsed, 14.908s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 21.130s; elapsed, 14.914s, #calls: 1', 'TOTAL : CPU, 21.130s; elapsed, 14.914s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.760s; elapsed, 14.910s, #calls: 1', 'TOTAL : CPU, 16.760s; elapsed, 14.910s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 40.480s; elapsed, 20.502s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.108s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.870s; elapsed, 37.694s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 15.620s; elapsed, 14.917s, #calls: 1', 'TOTAL : CPU, 94.120s; elapsed, 73.220s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 3790.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 59.340s; elapsed, 15.228s', 'individual call time for EIGEN_LDLT: CPU, 40.550s; elapsed, 40.039s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.119s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,171', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.521897', '# - matching time = 0.0839229', '# - symmetrization time = 0.0437839', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.072571', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.518 MB', '# - factor time = 18.7376', '# - factor nonzeros = 31,314,729', '# - factor memory = 250.518 MB', '# - total flops = 9.5061e+10, min = 41770, max = 7.01469e+10', '# - flop rate = 5.07328 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.72559e+10', '# --------------------------------------------', '# total = 9.72559e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.0162e-10\trel.res = 2.45408e-15\tbw.error = 4.56971e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.102959', '# - total flops = 8.12073e+07, min = 108187, max = 5.0687e+07', '# - flop rate = 0.788733 GFlop/s', '# - bytes moved = 211.042 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.04976 GByte/s', '# - solve arithmetic intensity = 0.384792 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 26.920s; elapsed, 19.628s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 38.880s; elapsed, 19.634s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 30.440s; elapsed, 19.635s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 51.330s; elapsed, 19.616s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 26.080s; elapsed, 19.630s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 75.560s; elapsed, 19.629s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 25.390s; elapsed, 19.632s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 21.770s; elapsed, 19.636s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 26.920s; elapsed, 19.628s, #calls: 1', 'TOTAL : CPU, 26.920s; elapsed, 19.628s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 26.080s; elapsed, 19.630s, #calls: 1', 'TOTAL : CPU, 26.080s; elapsed, 19.630s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 51.330s; elapsed, 19.616s, #calls: 1', 'TOTAL : CPU, 51.330s; elapsed, 19.616s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 38.880s; elapsed, 19.634s, #calls: 1', 'TOTAL : CPU, 38.880s; elapsed, 19.634s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 75.560s; elapsed, 19.629s, #calls: 1', 'TOTAL : CPU, 75.560s; elapsed, 19.629s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 25.390s; elapsed, 19.632s, #calls: 1', 'TOTAL : CPU, 25.390s; elapsed, 19.632s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 30.440s; elapsed, 19.635s, #calls: 1', 'TOTAL : CPU, 30.440s; elapsed, 19.635s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 59.340s; elapsed, 15.228s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.119s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.550s; elapsed, 40.039s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 21.770s; elapsed, 19.636s, #calls: 1', 'TOTAL : CPU, 121.880s; elapsed, 75.021s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 1895.4375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.680s; elapsed, 32.706s', 'individual call time for EIGEN_LDLT: CPU, 39.000s; elapsed, 39.039s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.119s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,171', '# - number of levels = 127', '# - nd time = 0.449232', '# - matching time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.070698', '# - symmetrization time = 0.0213499', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.066813', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.518 MB', '# - factor time = 17.1123', '# - factor nonzeros = 31,314,729', '# - factor memory = 250.518 MB', '# - total flops = 9.5061e+10, min = 41770, max = 7.00432e+10', '# - flop rate = 5.55511 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.68706e+10', '# --------------------------------------------', '# total = 9.68706e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.56982e-11\trel.res = 2.31108e-15\tbw.error = 3.94337e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0819831', '# - total flops = 1.01113e+08, min = 98712, max = 5.0209e+07', '# - flop rate = 1.23334 GFlop/s', '# - bytes moved = 203.539 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.4827 GByte/s', '# - solve arithmetic intensity = 0.496775 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.760s; elapsed, 17.855s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 17.820s; elapsed, 17.854s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 17.830s; elapsed, 17.854s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.750s; elapsed, 17.853s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.840s; elapsed, 17.854s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 17.750s; elapsed, 17.855s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 17.740s; elapsed, 17.854s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.750s; elapsed, 17.854s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.820s; elapsed, 17.851s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 17.750s; elapsed, 17.848s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.730s; elapsed, 17.851s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.830s; elapsed, 17.852s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 17.750s; elapsed, 17.849s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 17.730s; elapsed, 17.848s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.720s; elapsed, 17.852s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.850s; elapsed, 17.859s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.750s; elapsed, 17.854s, #calls: 1', 'TOTAL : CPU, 17.750s; elapsed, 17.854s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 17.750s; elapsed, 17.848s, #calls: 1', 'TOTAL : CPU, 17.750s; elapsed, 17.848s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.750s; elapsed, 17.853s, #calls: 1', 'TOTAL : CPU, 17.750s; elapsed, 17.853s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.840s; elapsed, 17.854s, #calls: 1', 'TOTAL : CPU, 17.840s; elapsed, 17.854s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 17.820s; elapsed, 17.854s, #calls: 1', 'TOTAL : CPU, 17.820s; elapsed, 17.854s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.760s; elapsed, 17.855s, #calls: 1', 'TOTAL : CPU, 17.760s; elapsed, 17.855s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 17.740s; elapsed, 17.854s, #calls: 1', 'TOTAL : CPU, 17.740s; elapsed, 17.854s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.830s; elapsed, 17.852s, #calls: 1', 'TOTAL : CPU, 17.830s; elapsed, 17.852s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 17.750s; elapsed, 17.855s, #calls: 1', 'TOTAL : CPU, 17.750s; elapsed, 17.855s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 17.750s; elapsed, 17.849s, #calls: 1', 'TOTAL : CPU, 17.750s; elapsed, 17.849s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.720s; elapsed, 17.852s, #calls: 1', 'TOTAL : CPU, 17.720s; elapsed, 17.852s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 17.830s; elapsed, 17.854s, #calls: 1', 'TOTAL : CPU, 17.830s; elapsed, 17.854s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.730s; elapsed, 17.851s, #calls: 1', 'TOTAL : CPU, 17.730s; elapsed, 17.851s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 17.730s; elapsed, 17.848s, #calls: 1', 'TOTAL : CPU, 17.730s; elapsed, 17.848s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.820s; elapsed, 17.851s, #calls: 1', 'TOTAL : CPU, 17.820s; elapsed, 17.851s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.680s; elapsed, 32.706s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.119s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.000s; elapsed, 39.039s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.850s; elapsed, 17.859s, #calls: 1', 'TOTAL : CPU, 89.650s; elapsed, 89.723s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 1895.4375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 38.100s; elapsed, 19.255s', 'individual call time for EIGEN_LDLT: CPU, 38.600s; elapsed, 38.444s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.120s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 16 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,171', '# - number of levels = 127', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.506749', '# - matching time = 0.079515', '# - symmetrization time = 0.0236838', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.081279', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.518 MB', '# - factor time = 18.2385', '# - factor nonzeros = 31,314,729', '# - factor memory = 250.518 MB', '# - total flops = 9.5061e+10, min = 41770, max = 7.00432e+10', '# - flop rate = 5.21211 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.68706e+10', '# --------------------------------------------', '# total = 9.68706e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.56833e-11\trel.res = 2.31072e-15\tbw.error = 3.94337e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.079212', '# - total flops = 1.01113e+08, min = 98712, max = 5.0209e+07', '# - flop rate = 1.27649 GFlop/s', '# - bytes moved = 203.544 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.56961 GByte/s', '# - solve arithmetic intensity = 0.496763 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 20.820s; elapsed, 19.068s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 21.580s; elapsed, 19.070s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 21.380s; elapsed, 19.069s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 21.140s; elapsed, 19.070s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 25.440s; elapsed, 19.073s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 23.180s; elapsed, 19.071s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 23.760s; elapsed, 19.070s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 21.410s; elapsed, 19.065s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 20.780s; elapsed, 19.069s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 20.660s; elapsed, 19.068s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 21.700s; elapsed, 19.061s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 37.170s; elapsed, 19.069s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 21.620s; elapsed, 19.067s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 21.190s; elapsed, 19.068s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 28.200s; elapsed, 19.062s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.750s; elapsed, 19.075s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 20.820s; elapsed, 19.068s, #calls: 1', 'TOTAL : CPU, 20.820s; elapsed, 19.068s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 20.780s; elapsed, 19.069s, #calls: 1', 'TOTAL : CPU, 20.780s; elapsed, 19.069s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 21.380s; elapsed, 19.069s, #calls: 1', 'TOTAL : CPU, 21.380s; elapsed, 19.069s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 21.410s; elapsed, 19.065s, #calls: 1', 'TOTAL : CPU, 21.410s; elapsed, 19.065s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 23.180s; elapsed, 19.071s, #calls: 1', 'TOTAL : CPU, 23.180s; elapsed, 19.071s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 25.440s; elapsed, 19.073s, #calls: 1', 'TOTAL : CPU, 25.440s; elapsed, 19.073s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 20.660s; elapsed, 19.068s, #calls: 1', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 21.580s; elapsed, 19.070s, #calls: 1', 'TOTAL : CPU, 21.580s; elapsed, 19.070s', 'TOTAL : CPU, 20.660s; elapsed, 19.068s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 37.170s; elapsed, 19.069s, #calls: 1', 'TOTAL : CPU, 37.170s; elapsed, 19.069s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 21.140s; elapsed, 19.070s, #calls: 1', 'Exiting profiler', 'TOTAL : CPU, 21.140s; elapsed, 19.070s', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 28.200s; elapsed, 19.062s, #calls: 1', 'TOTAL : CPU, 28.200s; elapsed, 19.062s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 23.760s; elapsed, 19.070s, #calls: 1', 'TOTAL : CPU, 23.760s; elapsed, 19.070s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 21.190s; elapsed, 19.068s, #calls: 1', 'TOTAL : CPU, 21.190s; elapsed, 19.068s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 21.620s; elapsed, 19.067s, #calls: 1', 'TOTAL : CPU, 21.620s; elapsed, 19.067s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 21.700s; elapsed, 19.061s, #calls: 1', 'TOTAL : CPU, 21.700s; elapsed, 19.061s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 38.100s; elapsed, 19.255s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.120s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.600s; elapsed, 38.444s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.750s; elapsed, 19.075s, #calls: 1', 'TOTAL : CPU, 96.620s; elapsed, 76.893s']
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Bfactor-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Bfactor-/A_strum_10k_omp1_paramslevmar.parameter_flags=Bfactor-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Bfactor-/b_strum_10k_omp1_paramslevmar.parameter_flags=Bfactor-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 141.730s; elapsed, 141.848s', 'individual call time for EIGEN_LDLT: CPU, 292.340s; elapsed, 292.520s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.279s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 141.730s; elapsed, 141.848s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.279s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.340s; elapsed, 292.520s, #calls: 1', 'TOTAL : CPU, 434.360s; elapsed, 434.647s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 164.130s; elapsed, 83.129s', 'individual call time for EIGEN_LDLT: CPU, 292.820s; elapsed, 292.863s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.239s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 164.130s; elapsed, 83.129s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.239s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.820s; elapsed, 292.863s, #calls: 1', 'TOTAL : CPU, 457.310s; elapsed, 376.231s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 246.360s; elapsed, 63.096s', 'individual call time for EIGEN_LDLT: CPU, 293.190s; elapsed, 292.847s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.239s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 246.360s; elapsed, 63.096s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.239s, #calls: 1', 'time for EIGEN_LDLT: CPU, 293.190s; elapsed, 292.847s, #calls: 1', 'TOTAL : CPU, 540.000s; elapsed, 356.183s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 538.270s; elapsed, 68.635s', 'individual call time for EIGEN_LDLT: CPU, 293.590s; elapsed, 293.120s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.178s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 538.270s; elapsed, 68.635s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.178s, #calls: 1', 'time for EIGEN_LDLT: CPU, 293.590s; elapsed, 293.120s, #calls: 1', 'TOTAL : CPU, 832.410s; elapsed, 361.933s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1020.030s; elapsed, 64.684s', 'individual call time for EIGEN_LDLT: CPU, 295.470s; elapsed, 293.099s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.920s; elapsed, 0.177s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1020.030s; elapsed, 64.684s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.920s; elapsed, 0.177s, #calls: 1', 'time for EIGEN_LDLT: CPU, 295.470s; elapsed, 293.099s, #calls: 1', 'TOTAL : CPU, 1316.420s; elapsed, 357.961s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 144.920s; elapsed, 145.000s', 'individual call time for EIGEN_LDLT: CPU, 293.070s; elapsed, 293.189s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.268s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 68', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.563893', '# - matching time = 0.121963', '# - symmetrization time = 0.1143', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.173911', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.182 MB', '# - factor time = 83.9234', '# - factor nonzeros = 113,147,790', '# - factor memory = 905.182 MB', '# - total flops = 7.19514e+11, min = 1.6676e+11, max = 5.52754e+11', '# - flop rate = 8.57346 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.29274e+11', '# --------------------------------------------', '# total = 7.29274e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.12487e-10\trel.res = 2.68831e-15\tbw.error = 6.87881e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.193389', '# - total flops = 2.36268e+08, min = 4.69568e+07, max = 1.89311e+08', '# - flop rate = 1.22172 GFlop/s', '# - bytes moved = 525.018 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.71483 GByte/s', '# - solve arithmetic intensity = 0.450019 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 85.030s; elapsed, 85.109s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 85.090s; elapsed, 85.129s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 85.030s; elapsed, 85.109s, #calls: 1', 'TOTAL : CPU, 85.030s; elapsed, 85.109s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 144.920s; elapsed, 145.000s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.268s, #calls: 1', 'time for EIGEN_LDLT: CPU, 293.070s; elapsed, 293.189s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 85.090s; elapsed, 85.129s, #calls: 1', 'TOTAL : CPU, 523.340s; elapsed, 523.585s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 156.990s; elapsed, 78.931s', 'individual call time for EIGEN_LDLT: CPU, 292.820s; elapsed, 292.806s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.242s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.593915', '# - matching time = 0.124004', '# - symmetrization time = 0.118403', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.190537', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.182 MB', '# - factor time = 82.3973', '# - factor nonzeros = 113,147,790', '# - factor memory = 905.182 MB', '# - total flops = 7.19514e+11, min = 1.6676e+11, max = 5.52754e+11', '# - flop rate = 8.73226 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.29274e+11', '# --------------------------------------------', '# total = 7.29274e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.12473e-10\trel.res = 2.68797e-15\tbw.error = 6.87847e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.332306', '# - total flops = 2.36317e+08, min = 4.70056e+07, max = 1.89311e+08', '# - flop rate = 0.711141 GFlop/s', '# - bytes moved = 526.314 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.58382 GByte/s', '# - solve arithmetic intensity = 0.449003 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 146.320s; elapsed, 83.786s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 165.160s; elapsed, 83.810s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 146.320s; elapsed, 83.786s, #calls: 1', 'TOTAL : CPU, 146.320s; elapsed, 83.786s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 156.990s; elapsed, 78.931s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.242s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.820s; elapsed, 292.806s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 165.160s; elapsed, 83.810s, #calls: 1', 'TOTAL : CPU, 615.330s; elapsed, 455.788s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 326.460s; elapsed, 83.440s', 'individual call time for EIGEN_LDLT: CPU, 301.500s; elapsed, 301.186s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.233s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.56676', '# - matching time = 0.132276', '# - symmetrization time = 0.122117', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.196164', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.182 MB', '# - factor time = 76.0822', '# - factor nonzeros = 113,147,790', '# - factor memory = 905.182 MB', '# - total flops = 7.19514e+11, min = 1.6676e+11, max = 5.52754e+11', '# - flop rate = 9.45707 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.29274e+11', '# --------------------------------------------', '# total = 7.29274e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.12402e-10\trel.res = 2.68628e-15\tbw.error = 6.87847e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.201955', '# - total flops = 2.36317e+08, min = 4.70056e+07, max = 1.89311e+08', '# - flop rate = 1.17015 GFlop/s', '# - bytes moved = 526.329 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.60617 GByte/s', '# - solve arithmetic intensity = 0.44899 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 229.530s; elapsed, 77.330s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 299.970s; elapsed, 77.347s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 229.530s; elapsed, 77.330s, #calls: 1', 'TOTAL : CPU, 229.530s; elapsed, 77.330s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 326.460s; elapsed, 83.440s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.233s, #calls: 1', 'time for EIGEN_LDLT: CPU, 301.500s; elapsed, 301.186s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 299.970s; elapsed, 77.347s, #calls: 1', 'TOTAL : CPU, 928.430s; elapsed, 462.206s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 656.220s; elapsed, 83.746s', 'individual call time for EIGEN_LDLT: CPU, 350.840s; elapsed, 350.455s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.600s; elapsed, 0.204s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.601008', '# - matching time = 0.149213', '# - symmetrization time = 0.134562', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.236634', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.182 MB', '# - factor time = 88.7679', '# - factor nonzeros = 113,147,790', '# - factor memory = 905.182 MB', '# - total flops = 7.19514e+11, min = 1.6676e+11, max = 5.52754e+11', '# - flop rate = 8.10557 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.29274e+11', '# --------------------------------------------', '# total = 7.29274e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.12404e-10\trel.res = 2.68632e-15\tbw.error = 6.87887e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.266323', '# - total flops = 2.36317e+08, min = 4.70056e+07, max = 1.89311e+08', '# - flop rate = 0.88733 GFlop/s', '# - bytes moved = 526.362 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.9764 GByte/s', '# - solve arithmetic intensity = 0.448962 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 400.040s; elapsed, 90.179s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 704.490s; elapsed, 90.196s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 400.040s; elapsed, 90.179s, #calls: 1', 'TOTAL : CPU, 400.040s; elapsed, 90.179s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 656.220s; elapsed, 83.746s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.600s; elapsed, 0.204s, #calls: 1', 'time for EIGEN_LDLT: CPU, 350.840s; elapsed, 350.455s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 704.490s; elapsed, 90.196s, #calls: 1', 'TOTAL : CPU, 1712.150s; elapsed, 524.602s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1449.460s; elapsed, 91.683s', 'individual call time for EIGEN_LDLT: CPU, 302.120s; elapsed, 299.595s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.920s; elapsed, 0.175s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.608151', '# - matching time = 0.156646', '# - symmetrization time = 0.18053', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.229522', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.182 MB', '# - factor time = 100.584', '# - factor nonzeros = 113,147,790', '# - factor memory = 905.182 MB', '# - total flops = 7.19514e+11, min = 1.6676e+11, max = 5.52754e+11', '# - flop rate = 7.15334 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.29274e+11', '# --------------------------------------------', '# total = 7.29274e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.12389e-10\trel.res = 2.68597e-15\tbw.error = 6.87887e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.391923', '# - total flops = 2.36317e+08, min = 4.70056e+07, max = 1.89311e+08', '# - flop rate = 0.602967 GFlop/s', '# - bytes moved = 526.428 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.34319 GByte/s', '# - solve arithmetic intensity = 0.448906 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 996.030s; elapsed, 102.181s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1587.170s; elapsed, 102.199s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 996.030s; elapsed, 102.181s, #calls: 1', 'TOTAL : CPU, 996.030s; elapsed, 102.181s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1449.460s; elapsed, 91.683s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.920s; elapsed, 0.175s, #calls: 1', 'time for EIGEN_LDLT: CPU, 302.120s; elapsed, 299.595s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1587.170s; elapsed, 102.199s, #calls: 1', 'TOTAL : CPU, 3339.670s; elapsed, 493.651s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 147.950s; elapsed, 148.084s', 'individual call time for EIGEN_LDLT: CPU, 302.140s; elapsed, 302.364s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.292s', 'CPP -2', '# Initializing STRUMPACK', 'CPP -2', 'CPP -2', 'CPP -2', '# using 1 OpenMP thread', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,139', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.607033', '# - matching time = 0.134257', '# - symmetrization time = 0.0776892', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.159855', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 901.01 MB', '# - factor time = 69.9338', '# - factor nonzeros = 112,626,254', '# - factor memory = 901.01 MB', '# - total flops = 7.15057e+11, min = 1.16766e+08, max = 5.51962e+11', '# - flop rate = 10.2248 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2444e+11', '# --------------------------------------------', '# total = 7.2444e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.75852e-11\trel.res = 2.09319e-15\tbw.error = 7.0214e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.241741', '# - total flops = 2.50759e+08, min = 255585, max = 1.8896e+08', '# - flop rate = 1.0373 GFlop/s', '# - bytes moved = 456.661 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.88905 GByte/s', '# - solve arithmetic intensity = 0.549114 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 71.150s; elapsed, 71.179s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 71.140s; elapsed, 71.175s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 71.020s; elapsed, 71.168s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 71.110s; elapsed, 71.188s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 71.150s; elapsed, 71.179s, #calls: 1', 'TOTAL : CPU, 71.150s; elapsed, 71.179s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 71.140s; elapsed, 71.175s, #calls: 1', 'TOTAL : CPU, 71.140s; elapsed, 71.175s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 71.020s; elapsed, 71.168s, #calls: 1', 'TOTAL : CPU, 71.020s; elapsed, 71.168s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 147.950s; elapsed, 148.084s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.292s, #calls: 1', 'time for EIGEN_LDLT: CPU, 302.140s; elapsed, 302.364s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 71.110s; elapsed, 71.188s, #calls: 1', 'TOTAL : CPU, 521.490s; elapsed, 521.927s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 188.090s; elapsed, 95.490s', 'individual call time for EIGEN_LDLT: CPU, 302.140s; elapsed, 302.181s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.244s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,139', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.593759', '# - matching time = 0.136821', '# - symmetrization time = 0.078804', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.169685', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 901.01 MB', '# - factor time = 86.2361', '# - factor nonzeros = 112,626,254', '# - factor memory = 901.01 MB', '# - total flops = 7.15057e+11, min = 1.16766e+08, max = 5.51962e+11', '# - flop rate = 8.29186 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2444e+11', '# --------------------------------------------', '# total = 7.2444e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.40934e-11\trel.res = 2.00974e-15\tbw.error = 7.0214e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.220926', '# - total flops = 2.5077e+08, min = 255585, max = 1.8896e+08', '# - flop rate = 1.13509 GFlop/s', '# - bytes moved = 457.114 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.06908 GByte/s', '# - solve arithmetic intensity = 0.548595 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 97.270s; elapsed, 87.456s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 117.890s; elapsed, 87.449s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 127.830s; elapsed, 87.458s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 170.670s; elapsed, 87.472s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 117.890s; elapsed, 87.449s, #calls: 1', 'TOTAL : CPU, 117.890s; elapsed, 87.449s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 97.270s; elapsed, 87.456s, #calls: 1', 'TOTAL : CPU, 97.270s; elapsed, 87.456s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 127.830s; elapsed, 87.458s, #calls: 1', 'TOTAL : CPU, 127.830s; elapsed, 87.458s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 188.090s; elapsed, 95.490s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.244s, #calls: 1', 'time for EIGEN_LDLT: CPU, 302.140s; elapsed, 302.181s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 170.670s; elapsed, 87.472s, #calls: 1', 'TOTAL : CPU, 661.260s; elapsed, 485.387s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 254.190s; elapsed, 64.677s', 'individual call time for EIGEN_LDLT: CPU, 304.590s; elapsed, 304.241s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.215s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,139', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.598257', '# - matching time = 0.135279', '# - symmetrization time = 0.0794539', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.170847', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 901.01 MB', '# - factor time = 82.2278', '# - factor nonzeros = 112,626,254', '# - factor memory = 901.01 MB', '# - total flops = 7.15057e+11, min = 1.16767e+08, max = 5.51962e+11', '# - flop rate = 8.69606 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2444e+11', '# --------------------------------------------', '# total = 7.2444e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.40844e-11\trel.res = 2.00952e-15\tbw.error = 7.0214e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.198475', '# - total flops = 2.5077e+08, min = 255585, max = 1.8896e+08', '# - flop rate = 1.26349 GFlop/s', '# - bytes moved = 457.126 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.30319 GByte/s', '# - solve arithmetic intensity = 0.54858 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 111.970s; elapsed, 83.431s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 201.980s; elapsed, 83.427s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 188.080s; elapsed, 83.423s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 323.740s; elapsed, 83.442s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 201.980s; elapsed, 83.427s, #calls: 1', 'TOTAL : CPU, 201.980s; elapsed, 83.427s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 111.970s; elapsed, 83.431s, #calls: 1', 'TOTAL : CPU, 111.970s; elapsed, 83.431s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 188.080s; elapsed, 83.423s, #calls: 1', 'TOTAL : CPU, 188.080s; elapsed, 83.423s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 254.190s; elapsed, 64.677s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.215s, #calls: 1', 'time for EIGEN_LDLT: CPU, 304.590s; elapsed, 304.241s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 323.740s; elapsed, 83.442s, #calls: 1', 'TOTAL : CPU, 882.990s; elapsed, 452.575s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 703.410s; elapsed, 88.912s', 'individual call time for EIGEN_LDLT: CPU, 304.160s; elapsed, 303.644s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.630s; elapsed, 0.197s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,139', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# - nd time = # ******************************************************************', '0.610624', '# - matching time = 0.153965', '# - symmetrization time = 0.0895329', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.198788', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 901.01 MB', '# - factor time = 69.6335', '# - factor nonzeros = 112,626,254', '# - factor memory = 901.01 MB', '# - total flops = 7.15057e+11, min = 1.16767e+08, max = 5.51962e+11', '# - flop rate = 10.2689 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2444e+11', '# --------------------------------------------', '# total = 7.2444e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.40817e-11\trel.res = 2.00946e-15\tbw.error = 7.0214e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.260917', '# - total flops = 2.5077e+08, min = 255585, max = 1.8896e+08', '# - flop rate = 0.961111 GFlop/s', '# - bytes moved = 457.151 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.75209 GByte/s', '# - solve arithmetic intensity = 0.54855 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 128.120s; elapsed, 70.975s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 369.910s; elapsed, 70.971s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 310.540s; elapsed, 70.966s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 541.220s; elapsed, 70.987s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 369.910s; elapsed, 70.971s, #calls: 1', 'TOTAL : CPU, 369.910s; elapsed, 70.971s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 128.120s; elapsed, 70.975s, #calls: 1', 'TOTAL : CPU, 128.120s; elapsed, 70.975s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 310.540s; elapsed, 70.966s, #calls: 1', 'TOTAL : CPU, 310.540s; elapsed, 70.966s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 703.410s; elapsed, 88.912s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.630s; elapsed, 0.197s, #calls: 1', 'time for EIGEN_LDLT: CPU, 304.160s; elapsed, 303.644s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 541.220s; elapsed, 70.987s, #calls: 1', 'TOTAL : CPU, 1549.420s; elapsed, 463.740s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 3166.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 151.980s; elapsed, 152.089s', 'individual call time for EIGEN_LDLT: CPU, 306.960s; elapsed, 307.079s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.296s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,127', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.598349', '# - matching time = 0.133743', '# - symmetrization time = 0.0565939', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,127', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.157678', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.692 MB', '# - factor time = 67.1467', '# - factor nonzeros = 112,961,510', '# - factor memory = 903.692 MB', '# - total flops = 7.18008e+11, min = 9.84773e+07, max = 5.53144e+11', '# - flop rate = 10.6931 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2662e+11', '# --------------------------------------------', '# total = 7.2662e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.52645e-11\trel.res = 1.79874e-15\tbw.error = 6.61787e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.147024', '# - total flops = 2.86389e+08, min = 226800, max = 1.88719e+08', '# - flop rate = 1.94791 GFlop/s', '# - bytes moved = 432.794 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.94369 GByte/s', '# - solve arithmetic intensity = 0.661723 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 68.220s; elapsed, 68.262s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 68.200s; elapsed, 68.257s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 68.190s; elapsed, 68.259s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 68.200s; elapsed, 68.262s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 68.220s; elapsed, 68.263s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 68.030s; elapsed, 68.263s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 68.180s; elapsed, 68.260s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 68.230s; elapsed, 68.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 68.200s; elapsed, 68.257s, #calls: 1', 'TOTAL : CPU, 68.200s; elapsed, 68.257s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 68.190s; elapsed, 68.259s, #calls: 1', 'TOTAL : CPU, 68.190s; elapsed, 68.259s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 68.200s; elapsed, 68.262s, #calls: 1', 'TOTAL : CPU, 68.200s; elapsed, 68.262s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 68.180s; elapsed, 68.260s, #calls: 1', 'TOTAL : CPU, 68.180s; elapsed, 68.260s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 68.220s; elapsed, 68.262s, #calls: 1', 'TOTAL : CPU, 68.220s; elapsed, 68.262s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 68.030s; elapsed, 68.263s, #calls: 1', 'TOTAL : CPU, 68.030s; elapsed, 68.263s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 68.220s; elapsed, 68.263s, #calls: 1', 'TOTAL : CPU, 68.220s; elapsed, 68.263s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 151.980s; elapsed, 152.089s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.296s, #calls: 1', 'time for EIGEN_LDLT: CPU, 306.960s; elapsed, 307.079s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 68.230s; elapsed, 68.270s, #calls: 1', 'TOTAL : CPU, 527.460s; elapsed, 527.734s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 3166.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 237.010s; elapsed, 119.786s', 'individual call time for EIGEN_LDLT: CPU, 334.340s; elapsed, 334.452s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.410s; elapsed, 0.283s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,127', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.697687', '# - matching time = 0.156106', '# - symmetrization time = 0.065562', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,127', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.181438', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.692 MB', '# - factor time = 60.0713', '# - factor nonzeros = 112,961,510', '# - factor memory = 903.692 MB', '# - total flops = 7.18008e+11, min = 9.84773e+07, max = 5.53144e+11', '# - flop rate = 11.9526 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2662e+11', '# --------------------------------------------', '# total = 7.2662e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.49551e-11\trel.res = 1.79134e-15\tbw.error = 6.61787e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.124838', '# - total flops = 2.86389e+08, min = 226800, max = 1.88719e+08', '# - flop rate = 2.29409 GFlop/s', '# - bytes moved = 432.807 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.46694 GByte/s', '# - solve arithmetic intensity = 0.661703 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 68.110s; elapsed, 61.316s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 68.880s; elapsed, 61.314s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 68.700s; elapsed, 61.316s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 111.080s; elapsed, 61.317s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 70.060s; elapsed, 61.308s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 85.820s; elapsed, 61.308s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 83.090s; elapsed, 61.308s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 116.870s; elapsed, 61.324s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 70.060s; elapsed, 61.308s, #calls: 1', 'TOTAL : CPU, 70.060s; elapsed, 61.308s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 68.110s; elapsed, 61.316s, #calls: 1', 'TOTAL : CPU, 68.110s; elapsed, 61.316s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 83.090s; elapsed, 61.308s, #calls: 1', 'TOTAL : CPU, 83.090s; elapsed, 61.308s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 85.820s; elapsed, 61.308s, #calls: 1', 'TOTAL : CPU, 85.820s; elapsed, 61.308s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 68.880s; elapsed, 61.314s, #calls: 1', 'TOTAL : CPU, 68.880s; elapsed, 61.314s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 68.700s; elapsed, 61.316s, #calls: 1', 'TOTAL : CPU, 68.700s; elapsed, 61.316s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 111.080s; elapsed, 61.317s, #calls: 1', 'TOTAL : CPU, 111.080s; elapsed, 61.317s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 237.010s; elapsed, 119.786s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.410s; elapsed, 0.283s, #calls: 1', 'time for EIGEN_LDLT: CPU, 334.340s; elapsed, 334.452s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 116.870s; elapsed, 61.324s, #calls: 1', 'TOTAL : CPU, 688.630s; elapsed, 515.845s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 3166.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 310.150s; elapsed, 78.638s', 'individual call time for EIGEN_LDLT: CPU, 313.270s; elapsed, 313.058s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.253s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,127', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.613572', '# - matching time = 0.150232', '# - symmetrization time = 0.0639539', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,127', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.203158', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 903.692 MB', '# - factor time = 60.7165', '# - factor nonzeros = 112,961,510', '# - factor memory = 903.692 MB', '# - total flops = 7.18008e+11, min = 9.84791e+07, max = 5.53144e+11', '# - flop rate = 11.8256 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2662e+11', '# --------------------------------------------', '# total = 7.2662e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.49343e-11\trel.res = 1.79085e-15\tbw.error = 6.61787e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.135319', '# - total flops = 2.86389e+08, min = 226800, max = 1.88719e+08', '# - flop rate = 2.1164 GFlop/s', '# - bytes moved = 432.824 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.19854 GByte/s', '# - solve arithmetic intensity = 0.661677 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 81.900s; elapsed, 61.908s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 84.150s; elapsed, 61.908s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 87.470s; elapsed, 61.903s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 83.740s; elapsed, 61.907s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 225.110s; elapsed, 61.909s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 133.980s; elapsed, 61.903s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 133.120s; elapsed, 61.905s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 233.260s; elapsed, 61.916s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 81.900s; elapsed, 61.908s, #calls: 1', 'TOTAL : CPU, 81.900s; elapsed, 61.908s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 133.120s; elapsed, 61.905s, #calls: 1', 'TOTAL : CPU, 133.120s; elapsed, 61.905s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 83.740s; elapsed, 61.907s, #calls: 1', 'TOTAL : CPU, 83.740s; elapsed, 61.907s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 225.110s; elapsed, 61.909s, #calls: 1', 'TOTAL : CPU, 225.110s; elapsed, 61.909s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 87.470s; elapsed, 61.903s, #calls: 1', 'TOTAL : CPU, 87.470s; elapsed, 61.903s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 84.150s; elapsed, 61.908s, #calls: 1', 'TOTAL : CPU, 84.150s; elapsed, 61.908s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 133.980s; elapsed, 61.903s, #calls: 1', 'TOTAL : CPU, 133.980s; elapsed, 61.903s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 310.150s; elapsed, 78.638s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.253s, #calls: 1', 'time for EIGEN_LDLT: CPU, 313.270s; elapsed, 313.058s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 233.260s; elapsed, 61.916s, #calls: 1', 'TOTAL : CPU, 857.210s; elapsed, 453.864s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 1583.125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 205.250s; elapsed, 205.762s', 'individual call time for EIGEN_LDLT: CPU, 328.620s; elapsed, 328.831s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.305s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,149', '# - number of levels = 65', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.640875', '# - matching time = 0.141697', '# - symmetrization time = 0.0501812', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,149', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.173143', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 902.044 MB', '# - factor time = 85.2134', '# - factor nonzeros = 112,755,472', '# - factor memory = 902.044 MB', '# - total flops = 7.1691e+11, min = 9.76849e+07, max = 5.51315e+11', '# - flop rate = 8.41311 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2397e+11', '# --------------------------------------------', '# total = 7.2397e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.99952e-11\trel.res = 1.91179e-15\tbw.error = 5.68886e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.10624', '# - total flops = 3.67095e+08, min = 217887, max = 1.87487e+08', '# - flop rate = 3.45533 GFlop/s', '# - bytes moved = 405.893 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.82053 GByte/s', '# - solve arithmetic intensity = 0.904413 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 86.250s; elapsed, 86.334s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 86.270s; elapsed, 86.335s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 86.270s; elapsed, 86.336s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 86.240s; elapsed, 86.335s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 86.310s; elapsed, 86.335s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 86.230s; elapsed, 86.333s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 86.210s; elapsed, 86.332s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 86.290s; elapsed, 86.335s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 86.280s; elapsed, 86.334s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 86.300s; elapsed, 86.333s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 86.300s; elapsed, 86.333s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 86.240s; elapsed, 86.332s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 86.240s; elapsed, 86.332s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 86.230s; elapsed, 86.336s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 86.280s; elapsed, 86.336s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 86.100s; elapsed, 86.342s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 86.230s; elapsed, 86.333s, #calls: 1', 'TOTAL : CPU, 86.230s; elapsed, 86.333s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 86.290s; elapsed, 86.335s, #calls: 1', 'TOTAL : CPU, 86.290s; elapsed, 86.335s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 86.270s; elapsed, 86.335s, #calls: 1', 'TOTAL : CPU, 86.270s; elapsed, 86.335s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 86.310s; elapsed, 86.335s, #calls: 1', 'TOTAL : CPU, 86.310s; elapsed, 86.335s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 86.240s; elapsed, 86.335s, #calls: 1', 'TOTAL : CPU, 86.240s; elapsed, 86.335s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 86.250s; elapsed, 86.334s, #calls: 1', 'TOTAL : CPU, 86.250s; elapsed, 86.334s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 86.270s; elapsed, 86.336s, #calls: 1', 'TOTAL : CPU, 86.270s; elapsed, 86.336sExiting profiler', '', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 86.300s; elapsed, 86.333s, #calls: 1', 'TOTAL : CPU, 86.300s; elapsed, 86.333s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 86.240s; elapsed, 86.332s, #calls: 1', 'TOTAL : CPU, 86.240s; elapsed, 86.332s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 86.280s; elapsed, 86.334s, #calls: 1', 'TOTAL : CPU, 86.280s; elapsed, 86.334s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 86.300s; elapsed, 86.333s, #calls: 1', 'TOTAL : CPU, 86.300s; elapsed, 86.333s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 86.240s; elapsed, 86.332s, #calls: 1', 'TOTAL : CPU, 86.240s; elapsed, 86.332s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 86.230s; elapsed, 86.336s, #calls: 1', 'TOTAL : CPU, 86.230s; elapsed, 86.336s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 86.210s; elapsed, 86.332s, #calls: 1', 'TOTAL : CPU, 86.210s; elapsed, 86.332s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 86.280s; elapsed, 86.336s, #calls: 1', 'TOTAL : CPU, 86.280s; elapsed, 86.336s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 205.250s; elapsed, 205.762s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.305s, #calls: 1', 'time for EIGEN_LDLT: CPU, 328.620s; elapsed, 328.831s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 86.100s; elapsed, 86.342s, #calls: 1', 'TOTAL : CPU, 620.280s; elapsed, 621.240s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 1583.125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 187.000s; elapsed, 94.271s', 'individual call time for EIGEN_LDLT: CPU, 303.190s; elapsed, 303.165s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.240s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 1,201,280', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,149', '# - number of levels = 65', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.632178', '# - matching time = 0.149364', '# - symmetrization time = 0.053596', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,149', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.184642', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 902.044 MB', '# - factor time = 59.7917', '# - factor nonzeros = 112,755,472', '# - factor memory = 902.044 MB', '# - total flops = 7.1691e+11, min = 9.76849e+07, max = 5.51315e+11', '# - flop rate = 11.9901 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2397e+11', '# --------------------------------------------', '# total = 7.2397e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.98842e-11\trel.res = 1.90914e-15\tbw.error = 5.67767e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.097754', '# - total flops = 3.67095e+08, min = 217887, max = 1.87487e+08', '# - flop rate = 3.75529 GFlop/s', '# - bytes moved = 405.902 MB, min = 0 MB, max = 0 MB', '# - byte rate = 4.15228 GByte/s', '# - solve arithmetic intensity = 0.904391 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 67.350s; elapsed, 60.928s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 67.360s; elapsed, 60.925s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 67.150s; elapsed, 60.926s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 67.490s; elapsed, 60.924s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 66.760s; elapsed, 60.923s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 65.530s; elapsed, 60.927s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 68.260s; elapsed, 60.922s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 66.220s; elapsed, 60.926s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 66.400s; elapsed, 60.926s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 65.540s; elapsed, 60.926s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 68.350s; elapsed, 60.927s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 106.640s; elapsed, 60.926s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 82.150s; elapsed, 60.922s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 73.550s; elapsed, 60.924s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 78.890s; elapsed, 60.923s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 115.100s; elapsed, 60.932s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 67.350s; elapsed, 60.928s, #calls: 1', 'TOTAL : CPU, 67.350s; elapsed, 60.928s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 67.150s; elapsed, 60.926s, #calls: 1', 'TOTAL : CPU, 67.150s; elapsed, 60.926s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 67.490s; elapsed, 60.924s, #calls: 1', 'TOTAL : CPU, 67.490s; elapsed, 60.924s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 66.220s; elapsed, 60.926s, #calls: 1', 'TOTAL : CPU, 66.220s; elapsed, 60.926s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 65.530s; elapsed, 60.927s, #calls: 1', 'TOTAL : CPU, 65.530s; elapsed, 60.927s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 73.550s; elapsed, 60.924s, #calls: 1', 'TOTAL : CPU, 73.550s; elapsed, 60.924s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 67.360s; elapsed, 60.925s, #calls: 1', 'TOTAL : CPU, 67.360s; elapsed, 60.925s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 68.260s; elapsed, 60.922s, #calls: 1', 'TOTAL : CPU, 68.260s; elapsed, 60.922s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 65.540s; elapsed, 60.926s, #calls: 1', 'TOTAL : CPU, 65.540s; elapsed, 60.926s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 66.760s; elapsed, 60.923s, #calls: 1', 'TOTAL : CPU, 66.760s; elapsed, 60.923s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 66.400s; elapsed, 60.926s, #calls: 1', 'TOTAL : CPU, 66.400s; elapsed, 60.926s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 82.150s; elapsed, 60.922s, #calls: 1', 'TOTAL : CPU, 82.150s; elapsed, 60.922s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 106.640s; elapsed, 60.926s, #calls: 1', 'TOTAL : CPU, 106.640s; elapsed, 60.926s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 78.890s; elapsed, 60.923s, #calls: 1', 'TOTAL : CPU, 78.890s; elapsed, 60.923s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 68.350s; elapsed, 60.927s, #calls: 1', 'TOTAL : CPU, 68.350s; elapsed, 60.927s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 187.000s; elapsed, 94.271s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.360s; elapsed, 0.240s, #calls: 1', 'time for EIGEN_LDLT: CPU, 303.190s; elapsed, 303.165s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 115.100s; elapsed, 60.932s, #calls: 1', 'TOTAL : CPU, 605.650s; elapsed, 458.608s']
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Deff-/A_strum_10k_omp1_paramslevmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Deff-/b_strum_10k_omp1_paramslevmar.parameter_flags=Deff-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 42.720s; elapsed, 42.753s', 'individual call time for EIGEN_LDLT: CPU, 37.930s; elapsed, 37.930s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.115s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 42.720s; elapsed, 42.753s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.115s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.930s; elapsed, 37.930s, #calls: 1', 'TOTAL : CPU, 80.760s; elapsed, 80.798s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 50.110s; elapsed, 25.125s', 'individual call time for EIGEN_LDLT: CPU, 37.790s; elapsed, 37.624s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.111s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 50.110s; elapsed, 25.125s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.790s; elapsed, 37.624s, #calls: 1', 'TOTAL : CPU, 88.060s; elapsed, 62.860s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 70.530s; elapsed, 17.760s', 'individual call time for EIGEN_LDLT: CPU, 38.120s; elapsed, 37.624s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.104s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 70.530s; elapsed, 17.760s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.104s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.120s; elapsed, 37.624s, #calls: 1', 'TOTAL : CPU, 108.860s; elapsed, 55.488s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 141.210s; elapsed, 17.809s', 'individual call time for EIGEN_LDLT: CPU, 38.910s; elapsed, 37.721s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.090s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 141.210s; elapsed, 17.809s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.090s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.910s; elapsed, 37.721s, #calls: 1', 'TOTAL : CPU, 180.380s; elapsed, 55.620s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 25330.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 296.500s; elapsed, 18.642s', 'individual call time for EIGEN_LDLT: CPU, 40.330s; elapsed, 37.716s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.420s; elapsed, 0.088s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 296.500s; elapsed, 18.642s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.420s; elapsed, 0.088s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.330s; elapsed, 37.716s, #calls: 1', 'TOTAL : CPU, 337.250s; elapsed, 56.446s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 44.830s; elapsed, 44.868s', 'individual call time for EIGEN_LDLT: CPU, 39.250s; elapsed, 39.298s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.123s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.260154', '# - matching time = 0.0706661', '# - symmetrization time = 0.047965', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.08951', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 29.5005', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74476e+10, min = 3.84903e+10, max = 5.89573e+10', '# - flop rate = 3.30325 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.02063e+11', '# --------------------------------------------', '# total = 1.02063e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.93647e-11\trel.res = 2.39962e-15\tbw.error = 4.56252e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.111981', '# - total flops = 6.69055e+07, min = 2.31678e+07, max = 4.37377e+07', '# - flop rate = 0.597472 GFlop/s', '# - bytes moved = 286.981 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.56277 GByte/s', '# - solve arithmetic intensity = 0.233135 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 30.080s; elapsed, 30.117s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 30.110s; elapsed, 30.133s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 30.080s; elapsed, 30.117s, #calls: 1', 'TOTAL : CPU, 30.080s; elapsed, 30.117s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 44.830s; elapsed, 44.868s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.123s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.250s; elapsed, 39.298s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 30.110s; elapsed, 30.133s, #calls: 1', 'TOTAL : CPU, 114.310s; elapsed, 114.421s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 52.280s; elapsed, 26.230s', 'individual call time for EIGEN_LDLT: CPU, 38.910s; elapsed, 38.767s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.113s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.245893', '# - matching time = 0.0730791', '# - symmetrization time = 0.0460291', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0892651', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 17.9068', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74477e+10, min = 3.84903e+10, max = 5.89573e+10', '# - flop rate = 5.44195 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.02063e+11', '# --------------------------------------------', '# total = 1.02063e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.91496e-11\trel.res = 2.39443e-15\tbw.error = 4.6452e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.120273', '# - total flops = 6.69125e+07, min = 2.31748e+07, max = 4.37377e+07', '# - flop rate = 0.556339 GFlop/s', '# - bytes moved = 287.3 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.38874 GByte/s', '# - solve arithmetic intensity = 0.232901 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 32.740s; elapsed, 18.523s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 36.750s; elapsed, 18.536s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 32.740s; elapsed, 18.523s, #calls: 1', 'TOTAL : CPU, 32.740s; elapsed, 18.523s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 52.280s; elapsed, 26.230s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.113s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.910s; elapsed, 38.767s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 36.750s; elapsed, 18.536s, #calls: 1', 'TOTAL : CPU, 128.100s; elapsed, 83.645s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 71.530s; elapsed, 18.036s', 'individual call time for EIGEN_LDLT: CPU, 38.140s; elapsed, 37.618s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.101s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.240442', '# - matching time = 0.0652699', '# - symmetrization time = 0.0480769', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0959878', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 18.0631', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74477e+10, min = 3.84903e+10, max = 5.89573e+10', '# - flop rate = 5.39485 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.02063e+11', '# --------------------------------------------', '# total = 1.02063e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.90918e-11\trel.res = 2.39303e-15\tbw.error = 4.6452e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.127596', '# - total flops = 6.69125e+07, min = 2.31748e+07, max = 4.37377e+07', '# - flop rate = 0.524409 GFlop/s', '# - bytes moved = 287.316 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.25177 GByte/s', '# - solve arithmetic intensity = 0.232888 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 66.010s; elapsed, 18.685s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 73.740s; elapsed, 18.700s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 66.010s; elapsed, 18.685s, #calls: 1', 'TOTAL : CPU, 66.010s; elapsed, 18.685s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 71.530s; elapsed, 18.036s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.101s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.140s; elapsed, 37.618s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 73.740s; elapsed, 18.700s, #calls: 1', 'TOTAL : CPU, 183.590s; elapsed, 74.455s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 143.700s; elapsed, 18.136s', 'individual call time for EIGEN_LDLT: CPU, 40.210s; elapsed, 39.005s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.105s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.269401', '# - matching time = 0.0709729', '# - symmetrization time = 0.049592', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.102719', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 18.6305', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74477e+10, min = 3.84903e+10, max = 5.89573e+10', '# - flop rate = 5.23055 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.02063e+11', '# --------------------------------------------', '# total = 1.02063e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.9097e-11\trel.res = 2.39316e-15\tbw.error = 4.64314e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.118326', '# - total flops = 6.69125e+07, min = 2.31748e+07, max = 4.37377e+07', '# - flop rate = 0.565493 GFlop/s', '# - bytes moved = 287.348 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.42845 GByte/s', '# - solve arithmetic intensity = 0.232862 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 120.660s; elapsed, 19.284s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 151.700s; elapsed, 19.299s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 120.660s; elapsed, 19.284s, #calls: 1', 'TOTAL : CPU, 120.660s; elapsed, 19.284s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 143.700s; elapsed, 18.136s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.105s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.210s; elapsed, 39.005s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 151.700s; elapsed, 19.299s, #calls: 1', 'TOTAL : CPU, 335.850s; elapsed, 76.544s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 12665.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 324.760s; elapsed, 20.429s', 'individual call time for EIGEN_LDLT: CPU, 48.880s; elapsed, 46.209s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.400s; elapsed, 0.099s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.272035', '# - matching time = 0.0876808', '# - symmetrization time = 0.0576181', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.11199', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 19.994', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74477e+10, min = 3.84903e+10, max = 5.89573e+10', '# - flop rate = 4.87385 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.02063e+11', '# --------------------------------------------', '# total = 1.02063e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.90639e-11\trel.res = 2.39236e-15\tbw.error = 4.64314e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.137957', '# - total flops = 6.69125e+07, min = 2.31748e+07, max = 4.37377e+07', '# - flop rate = 0.485024 GFlop/s', '# - bytes moved = 287.413 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.08335 GByte/s', '# - solve arithmetic intensity = 0.232809 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 243.560s; elapsed, 20.713s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 325.340s; elapsed, 20.726s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 243.560s; elapsed, 20.713s, #calls: 1', 'TOTAL : CPU, 243.560s; elapsed, 20.713s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 324.760s; elapsed, 20.429s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.400s; elapsed, 0.099s, #calls: 1', 'time for EIGEN_LDLT: CPU, 48.880s; elapsed, 46.209s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 325.340s; elapsed, 20.726s, #calls: 1', 'TOTAL : CPU, 699.380s; elapsed, 87.463s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 52.530s; elapsed, 52.625s', 'individual call time for EIGEN_LDLT: CPU, 38.690s; elapsed, 38.712s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.128s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,101', '# - number of levels = 131', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.258646', '# - matching time = 0.0747569', '# - symmetrization time = 0.037302', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,101', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.077363', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 240.946 MB', '# - factor time = 18.0914', '# - factor nonzeros = 30,118,212', '# - factor memory = 240.946 MB', '# - total flops = 9.63072e+10, min = 3.73096e+06, max = 5.90189e+10', '# - flop rate = 5.32337 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.00843e+11', '# --------------------------------------------', '# total = 1.00843e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.49226e-11\trel.res = 1.80935e-15\tbw.error = 4.2417e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0825031', '# - total flops = 7.3905e+07, min = 65585, max = 4.36844e+07', '# - flop rate = 0.895785 GFlop/s', '# - bytes moved = 252.69 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.06279 GByte/s', '# - solve arithmetic intensity = 0.292473 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 18.610s; elapsed, 18.647s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.610s; elapsed, 18.633s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.630s; elapsed, 18.645s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.630s; elapsed, 18.652s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 18.610s; elapsed, 18.647s, #calls: 1', 'TOTAL : CPU, 18.610s; elapsed, 18.647s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.610s; elapsed, 18.633s, #calls: 1', 'TOTAL : CPU, 18.610s; elapsed, 18.633s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.630s; elapsed, 18.645s, #calls: 1', 'TOTAL : CPU, 18.630s; elapsed, 18.645s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 52.530s; elapsed, 52.625s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.128s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.690s; elapsed, 38.712s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.630s; elapsed, 18.652s, #calls: 1', 'TOTAL : CPU, 109.980s; elapsed, 110.116s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 55.490s; elapsed, 27.841s', 'individual call time for EIGEN_LDLT: CPU, 39.860s; elapsed, 39.709s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.111s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 4CPP -2', 'CPP -2', 'CPP -2', ' MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,101', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# - nd time = # As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.300268', '# - matching time = 0.079905', '# - symmetrization time = 0.0366862', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,101', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.091177', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 240.946 MB', '# - factor time = 17.6616', '# - factor nonzeros = 30,118,212', '# - factor memory = 240.946 MB', '# - total flops = 9.63072e+10, min = 3.73096e+06, max = 5.90189e+10', '# - flop rate = 5.45292 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.00843e+11', '# --------------------------------------------', '# total = 1.00843e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.47973e-11\trel.res = 1.80633e-15\tbw.error = 4.24305e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0819921', '# - total flops = 7.39141e+07, min = 65585, max = 4.36844e+07', '# - flop rate = 0.901478 GFlop/s', '# - bytes moved = 252.882 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.08422 GByte/s', '# - solve arithmetic intensity = 0.292287 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 27.570s; elapsed, 18.282s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 20.520s; elapsed, 18.278s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 32.340s; elapsed, 18.270s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 35.960s; elapsed, 18.286s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 27.570s; elapsed, 18.282s, #calls: 1', 'TOTAL : CPU, 27.570s; elapsed, 18.282s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 20.520s; elapsed, 18.278s, #calls: 1', 'TOTAL : CPU, 20.520s; elapsed, 18.278s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 32.340s; elapsed, 18.270s, #calls: 1', 'TOTAL : CPU, 32.340s; elapsed, 18.270s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 55.490s; elapsed, 27.841s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.860s; elapsed, 39.709s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 35.960s; elapsed, 18.286s, #calls: 1', 'TOTAL : CPU, 131.460s; elapsed, 85.947s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 71.350s; elapsed, 17.954s', 'individual call time for EIGEN_LDLT: CPU, 38.790s; elapsed, 38.277s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.096s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5CPP -2', 'CPP -2', 'CPP -2', ' = log_2(#threads) + 3', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,101', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.272886', '# - matching time = 0.078949', '# - symmetrization time = 0.0410912', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,101', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0950689', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 240.946 MB', '# - factor time = 17.7718', '# - factor nonzeros = 30,118,212', '# - factor memory = 240.946 MB', '# - total flops = 9.63072e+10, min = 3.73175e+06, max = 5.90189e+10', '# - flop rate = 5.4191 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.00843e+11', '# --------------------------------------------', '# total = 1.00843e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.47955e-11\trel.res = 1.80628e-15\tbw.error = 4.24305e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0979948', '# - total flops = 7.39141e+07, min = 65585, max = 4.36844e+07', '# - flop rate = 0.754266 GFlop/s', '# - bytes moved = 252.89 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.58065 GByte/s', '# - solve arithmetic intensity = 0.292277 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 47.850s; elapsed, 18.389s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 23.760s; elapsed, 18.388s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 60.490s; elapsed, 18.377s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 72.150s; elapsed, 18.394s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 23.760s; elapsed, 18.388s, #calls: 1', 'TOTAL : CPU, 23.760s; elapsed, 18.388s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 47.850s; elapsed, 18.389s, #calls: 1', 'TOTAL : CPU, 47.850s; elapsed, 18.389s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 60.490s; elapsed, 18.377s, #calls: 1', 'TOTAL : CPU, 60.490s; elapsed, 18.377s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 71.350s; elapsed, 17.954s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.096s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.790s; elapsed, 38.277s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 72.150s; elapsed, 18.394s, #calls: 1', 'TOTAL : CPU, 182.470s; elapsed, 74.721s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 6332.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 150.310s; elapsed, 18.945s', 'individual call time for EIGEN_LDLT: CPU, 39.170s; elapsed, 37.967s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.119s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,101', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.283675', '# - matching time = 0.0831909', '# - symmetrization time = 0.0405788', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,101', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.102354', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 240.946 MB', '# - factor time = 18.8719', '# - factor nonzeros = 30,118,212', '# - factor memory = 240.946 MB', '# - total flops = 9.63072e+10, min = 3.73175e+06, max = 5.90189e+10', '# - flop rate = 5.1032 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.00843e+11', '# --------------------------------------------', '# total = 1.00843e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.47952e-11\trel.res = 1.80628e-15\tbw.error = 4.24305e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.101359', '# - total flops = 7.39141e+07, min = 65585, max = 4.36844e+07', '# - flop rate = 0.72923 GFlop/s', '# - bytes moved = 252.909 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.49517 GByte/s', '# - solve arithmetic intensity = 0.292256 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 89.210s; elapsed, 19.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 31.930s; elapsed, 19.511s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 120.870s; elapsed, 19.502s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 151.950s; elapsed, 19.519s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 31.930s; elapsed, 19.511s, #calls: 1', 'TOTAL : CPU, 31.930s; elapsed, 19.511s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 120.870s; elapsed, 19.502s, #calls: 1', 'TOTAL : CPU, 120.870s; elapsed, 19.502s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 89.210s; elapsed, 19.513s, #calls: 1', 'TOTAL : CPU, 89.210s; elapsed, 19.513s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 150.310s; elapsed, 18.945s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.119s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.170s; elapsed, 37.967s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 151.950s; elapsed, 19.519s, #calls: 1', 'TOTAL : CPU, 341.700s; elapsed, 76.550s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 3166.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 45.960s; elapsed, 45.975s', 'individual call time for EIGEN_LDLT: CPU, 39.940s; elapsed, 39.948s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.128s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,053', '# - number of levels = 130', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.266578', '# - matching time = 0.0708241', '# - symmetrization time = 0.037334', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,053', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0776591', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 243.179 MB', '# - factor time = 19.8297', '# - factor nonzeros = 30,397,416', '# - factor memory = 243.179 MB', '# - total flops = 9.76253e+10, min = 3.33333e+06, max = 5.89486e+10', '# - flop rate = 4.92318 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.01977e+11', '# --------------------------------------------', '# total = 1.01977e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 6.77384e-11\trel.res = 1.63586e-15\tbw.error = 4.40647e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.079818', '# - total flops = 9.18847e+07, min = 46449, max = 4.35803e+07', '# - flop rate = 1.15118 GFlop/s', '# - bytes moved = 243.937 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.05617 GByte/s', '# - solve arithmetic intensity = 0.376674 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 20.340s; elapsed, 20.385s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 20.340s; elapsed, 20.381s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 20.370s; elapsed, 20.382s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 20.360s; elapsed, 20.382s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 20.340s; elapsed, 20.380s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 20.360s; elapsed, 20.380s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 20.350s; elapsed, 20.370s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.370s; elapsed, 20.388s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 20.360s; elapsed, 20.382s, #calls: 1', 'TOTAL : CPU, 20.360s; elapsed, 20.382s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 20.340s; elapsed, 20.381s, #calls: 1', 'TOTAL : CPU, 20.340s; elapsed, 20.381s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 20.370s; elapsed, 20.382s, #calls: 1', 'TOTAL : CPU, 20.370s; elapsed, 20.382s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 20.340s; elapsed, 20.380s, #calls: 1', 'TOTAL : CPU, 20.340s; elapsed, 20.380s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 20.340s; elapsed, 20.385s, #calls: 1', 'TOTAL : CPU, 20.340s; elapsed, 20.385s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 20.360s; elapsed, 20.380s, #calls: 1', 'TOTAL : CPU, 20.360s; elapsed, 20.380s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 20.350s; elapsed, 20.370s, #calls: 1', 'TOTAL : CPU, 20.350s; elapsed, 20.370s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 45.960s; elapsed, 45.975s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.128s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.940s; elapsed, 39.948s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.370s; elapsed, 20.388s, #calls: 1', 'TOTAL : CPU, 106.400s; elapsed, 106.439s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 3166.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 57.900s; elapsed, 29.060s', 'individual call time for EIGEN_LDLT: CPU, 41.590s; elapsed, 41.450s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.107s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', ' = log_2(#threads) + 3', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,053', '# - number of levels = 130', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.302425', '# - matching time = 0.0819871', '# - symmetrization time = 0.0387001', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,053', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0952189', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 243.179 MB', '# - factor time = 19.7828', '# - factor nonzeros = 30,397,416', '# - factor memory = 243.179 MB', '# - total flops = 9.76253e+10, min = 3.33333e+06, max = 5.89486e+10', '# - flop rate = 4.93485 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.01977e+11', '# --------------------------------------------', '# total = 1.01977e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 6.76835e-11\trel.res = 1.63453e-15\tbw.error = 4.41108e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0891449', '# - total flops = 9.18847e+07, min = 46449, max = 4.35803e+07', '# - flop rate = 1.03073 GFlop/s', '# - bytes moved = 243.947 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.73652 GByte/s', '# - solve arithmetic intensity = 0.376659 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 27.120s; elapsed, 20.422s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 22.260s; elapsed, 20.416s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 34.220s; elapsed, 20.415s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 22.930s; elapsed, 20.405s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 28.780s; elapsed, 20.416s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 22.690s; elapsed, 20.418s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 22.490s; elapsed, 20.415s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 40.200s; elapsed, 20.422s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 34.220s; elapsed, 20.415s, #calls: 1', 'TOTAL : CPU, 34.220s; elapsed, 20.415s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 22.260s; elapsed, 20.416s, #calls: 1', 'TOTAL : CPU, 22.260s; elapsed, 20.416s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 27.120s; elapsed, 20.422s, #calls: 1', 'TOTAL : CPU, 27.120s; elapsed, 20.422s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 22.930s; elapsed, 20.405s, #calls: 1', 'TOTAL : CPU, 22.930s; elapsed, 20.405s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 22.690s; elapsed, 20.418s, #calls: 1', 'TOTAL : CPU, 22.690s; elapsed, 20.418s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 28.780s; elapsed, 20.416s, #calls: 1', 'TOTAL : CPU, 28.780s; elapsed, 20.416s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 22.490s; elapsed, 20.415s, #calls: 1', 'TOTAL : CPU, 22.490s; elapsed, 20.415s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 57.900s; elapsed, 29.060s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.107s, #calls: 1', 'time for EIGEN_LDLT: CPU, 41.590s; elapsed, 41.450s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 40.200s; elapsed, 20.422s, #calls: 1', 'TOTAL : CPU, 139.840s; elapsed, 91.039s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 3166.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 78.000s; elapsed, 19.651s', 'individual call time for EIGEN_LDLT: CPU, 46.670s; elapsed, 46.192s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.096s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,053', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.276174', '# - matching time = 0.081959', '# - symmetrization time = 0.0441151', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,053', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.101362', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 243.179 MB', '# - factor time = 18.8785', '# - factor nonzeros = 30,397,416', '# - factor memory = 243.179 MB', '# - total flops = 9.76253e+10, min = 3.33405e+06, max = 5.89486e+10', '# - flop rate = 5.17125 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.01977e+11', '# --------------------------------------------', '# total = 1.01977e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 6.76723e-11\trel.res = 1.63426e-15\tbw.error = 4.40185e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.108999', '# - total flops = 9.18847e+07, min = 46449, max = 4.35803e+07', '# - flop rate = 0.842987 GFlop/s', '# - bytes moved = 243.959 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.23818 GByte/s', '# - solve arithmetic intensity = 0.37664 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 37.060s; elapsed, 19.527s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 45.220s; elapsed, 19.520s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 68.040s; elapsed, 19.522s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 27.170s; elapsed, 19.511s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 26.640s; elapsed, 19.525s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 25.140s; elapsed, 19.523s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 25.850s; elapsed, 19.523s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 76.390s; elapsed, 19.532s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 27.170s; elapsed, 19.511s, #calls: 1', 'TOTAL : CPU, 27.170s; elapsed, 19.511s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 68.040s; elapsed, 19.522s, #calls: 1', 'TOTAL : CPU, 68.040s; elapsed, 19.522s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 25.850s; elapsed, 19.523s, #calls: 1', 'TOTAL : CPU, 25.850s; elapsed, 19.523s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 45.220s; elapsed, 19.520s, #calls: 1', 'TOTAL : CPU, 45.220s; elapsed, 19.520s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 26.640s; elapsed, 19.525s, #calls: 1', 'TOTAL : CPU, 26.640s; elapsed, 19.525s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 25.140s; elapsed, 19.523s, #calls: 1', 'TOTAL : CPU, 25.140s; elapsed, 19.523s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 37.060s; elapsed, 19.527s, #calls: 1', 'TOTAL : CPU, 37.060s; elapsed, 19.527s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 78.000s; elapsed, 19.651s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.096s, #calls: 1', 'time for EIGEN_LDLT: CPU, 46.670s; elapsed, 46.192s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 76.390s; elapsed, 19.532s, #calls: 1', 'TOTAL : CPU, 201.250s; elapsed, 85.470s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 1583.125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 45.730s; elapsed, 45.773s', 'individual call time for EIGEN_LDLT: CPU, 39.370s; elapsed, 39.397s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.125s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 129', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.2644', '# - matching time = 0.0711598', '# - symmetrization time = 0.017853', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.069427', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 17.5398', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74476e+10, min = 3.58592e+06, max = 5.89302e+10', '# - flop rate = 5.55579 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.01444e+11', '# --------------------------------------------', '# total = 1.01444e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.46798e-11\trel.res = 2.28648e-15\tbw.error = 3.3904e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0566521', '# - total flops = 1.34224e+08, min = 40130, max = 4.34304e+07', '# - flop rate = 2.36926 GFlop/s', '# - bytes moved = 235.766 MB, min = 0 MB, max = 0 MB', '# - byte rate = 4.16166 GByte/s', '# - solve arithmetic intensity = 0.569308 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 18.000s; elapsed, 18.035s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 17.980s; elapsed, 18.038s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.980s; elapsed, 18.036s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 18.000s; elapsed, 18.034s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 17.970s; elapsed, 18.037s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 17.990s; elapsed, 18.038s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.030s; elapsed, 18.037s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.010s; elapsed, 18.037s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 18.010s; elapsed, 18.035s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.990s; elapsed, 18.036s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.960s; elapsed, 18.036s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.950s; elapsed, 18.037s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 18.020s; elapsed, 18.036s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.980s; elapsed, 18.030s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 17.960s; elapsed, 18.031s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.030s; elapsed, 18.042s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 18.000s; elapsed, 18.034s, #calls: 1', 'TOTAL : CPU, 18.000s; elapsed, 18.034s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.030s; elapsed, 18.037s, #calls: 1', 'TOTAL : CPU, 18.030s; elapsed, 18.037s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 18.000s; elapsed, 18.035s, #calls: 1', 'TOTAL : CPU, 18.000s; elapsed, 18.035s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.980s; elapsed, 18.036s, #calls: 1', 'TOTAL : CPU, 17.980s; elapsed, 18.036s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.960s; elapsed, 18.036s, #calls: 1', 'TOTAL : CPU, 17.960s; elapsed, 18.036s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 17.980s; elapsed, 18.038s, #calls: 1', 'TOTAL : CPU, 17.980s; elapsed, 18.038s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.990s; elapsed, 18.036s, #calls: 1', 'TOTAL : CPU, 17.990s; elapsed, 18.036s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 17.970s; elapsed, 18.037s, #calls: 1', 'TOTAL : CPU, 17.970s; elapsed, 18.037s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.950s; elapsed, 18.037s, #calls: 1', 'TOTAL : CPU, 17.950s; elapsed, 18.037s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.010s; elapsed, 18.037s, #calls: 1', 'TOTAL : CPU, 18.010s; elapsed, 18.037s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 17.990s; elapsed, 18.038s, #calls: 1', 'TOTAL : CPU, 17.990s; elapsed, 18.038s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 18.020s; elapsed, 18.036s, #calls: 1', 'TOTAL : CPU, 18.020s; elapsed, 18.036s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 17.960s; elapsed, 18.031s, #calls: 1', 'TOTAL : CPU, 17.960s; elapsed, 18.031s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 18.010s; elapsed, 18.035s, #calls: 1', 'TOTAL : CPU, 18.010s; elapsed, 18.035s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.980s; elapsed, 18.030s, #calls: 1', 'TOTAL : CPU, 17.980s; elapsed, 18.030s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 45.730s; elapsed, 45.773s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.125s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.370s; elapsed, 39.397s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.030s; elapsed, 18.042s, #calls: 1', 'TOTAL : CPU, 103.260s; elapsed, 103.337s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25330 1583.125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 56.270s; elapsed, 28.244s', 'individual call time for EIGEN_LDLT: CPU, 40.490s; elapsed, 40.346s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.122s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,330', '# - number of nonzeros = 608,308', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,063', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.306187', '# - matching time = 0.0821331', '# - symmetrization time = 0.0194452', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.087703', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 242.865 MB', '# - factor time = 18.2915', '# - factor nonzeros = 30,358,066', '# - factor memory = 242.865 MB', '# - total flops = 9.74476e+10, min = 3.58592e+06, max = 5.89302e+10', '# - flop rate = 5.32747 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.01444e+11', '# --------------------------------------------', '# total = 1.01444e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.46625e-11\trel.res = 2.28606e-15\tbw.error = 3.3904e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0704439', '# - total flops = 1.34224e+08, min = 40130, max = 4.34304e+07', '# - flop rate = 1.9054 GFlop/s', '# - bytes moved = 235.775 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.34699 GByte/s', '# - solve arithmetic intensity = 0.569288 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 21.260s; elapsed, 18.881s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 20.310s; elapsed, 18.876s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 25.290s; elapsed, 18.880s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 24.320s; elapsed, 18.880s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 20.180s; elapsed, 18.877s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 20.920s; elapsed, 18.873s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 29.130s; elapsed, 18.875s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 21.080s; elapsed, 18.879s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 20.850s; elapsed, 18.879s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 20.640s; elapsed, 18.879s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 20.320s; elapsed, 18.877s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 33.520s; elapsed, 18.877s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 21.390s; elapsed, 18.880s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 21.170s; elapsed, 18.878s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 20.710s; elapsed, 18.874s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 37.210s; elapsed, 18.884s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 20.310s; elapsed, 18.876s, #calls: 1', 'TOTAL : CPU, 20.310s; elapsed, 18.876s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 25.290s; elapsed, 18.880s, #calls: 1', 'TOTAL : CPU, 25.290s; elapsed, 18.880s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 20.920s; elapsed, 18.873s, #calls: 1', 'TOTAL : CPU, 20.920s; elapsed, 18.873s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 20.180s; elapsed, 18.877s, #calls: 1', 'TOTAL : CPU, 20.180s; elapsed, 18.877s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 24.320s; elapsed, 18.880s, #calls: 1', 'TOTAL : CPU, 24.320s; elapsed, 18.880s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 21.260s; elapsed, 18.881s, #calls: 1', 'TOTAL : CPU, 21.260s; elapsed, 18.881s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 21.390s; elapsed, 18.880s, #calls: 1', 'TOTAL : CPU, 21.390s; elapsed, 18.880s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 20.850s; elapsed, 18.879s, #calls: 1', 'TOTAL : CPU, 20.850s; elapsed, 18.879s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 29.130s; elapsed, 18.875s, #calls: 1', 'TOTAL : CPU, 29.130s; elapsed, 18.875s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 20.710s; elapsed, 18.874s, #calls: 1', 'TOTAL : CPU, 20.710s; elapsed, 18.874s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 20.320s; elapsed, 18.877s, #calls: 1', 'TOTAL : CPU, 20.320s; elapsed, 18.877s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 21.170s; elapsed, 18.878s, #calls: 1', 'TOTAL : CPU, 21.170s; elapsed, 18.878s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 21.080s; elapsed, 18.879s, #calls: 1', 'TOTAL : CPU, 21.080s; elapsed, 18.879s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 20.640s; elapsed, 18.879s, #calls: 1', 'TOTAL : CPU, 20.640s; elapsed, 18.879s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 33.520s; elapsed, 18.877s, #calls: 1', 'TOTAL : CPU, 33.520s; elapsed, 18.877s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 56.270s; elapsed, 28.244s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.122s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.490s; elapsed, 40.346s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 37.210s; elapsed, 18.884s, #calls: 1', 'TOTAL : CPU, 134.140s; elapsed, 87.596s']
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Eta-/A_strum_10k_omp1_paramslevmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Eta-/b_strum_10k_omp1_paramslevmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 20333.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 30.810s; elapsed, 30.840s', 'individual call time for EIGEN_LDLT: CPU, 38.180s; elapsed, 38.208s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.116s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 30.810s; elapsed, 30.840s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.116s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.180s; elapsed, 38.208s, #calls: 1', 'TOTAL : CPU, 69.110s; elapsed, 69.165s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 20333.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 41.000s; elapsed, 20.617s', 'individual call time for EIGEN_LDLT: CPU, 38.080s; elapsed, 37.911s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.116s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 41.000s; elapsed, 20.617s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.116s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.080s; elapsed, 37.911s, #calls: 1', 'TOTAL : CPU, 79.250s; elapsed, 58.644s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 20333.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 56.380s; elapsed, 14.216s', 'individual call time for EIGEN_LDLT: CPU, 38.250s; elapsed, 37.729s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.099s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 56.380s; elapsed, 14.216s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.099s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.250s; elapsed, 37.729s, #calls: 1', 'TOTAL : CPU, 94.820s; elapsed, 52.043s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 20333.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 108.810s; elapsed, 13.681s', 'individual call time for EIGEN_LDLT: CPU, 38.950s; elapsed, 37.672s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.092s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 108.810s; elapsed, 13.681s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.092s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.950s; elapsed, 37.672s, #calls: 1', 'TOTAL : CPU, 147.980s; elapsed, 51.445s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 20333.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 256.440s; elapsed, 16.123s', 'individual call time for EIGEN_LDLT: CPU, 40.460s; elapsed, 37.713s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.400s; elapsed, 0.099s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 256.440s; elapsed, 16.123s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.400s; elapsed, 0.099s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.460s; elapsed, 37.713s, #calls: 1', 'TOTAL : CPU, 297.300s; elapsed, 53.936s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 10166.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 30.690s; elapsed, 30.711s', 'individual call time for EIGEN_LDLT: CPU, 38.040s; elapsed, 38.080s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.116s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.185051', '# - matching time = 0.0646098', '# - symmetrization time = 0.039875', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0939381', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 21.516', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70012e+10, min = 2.69076e+10, max = 7.00936e+10', '# - flop rate = 4.50832 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.94363e+10', '# --------------------------------------------', '# total = 9.94363e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.66658e-11\trel.res = 1.85145e-15\tbw.error = 5.42862e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0925951', '# - total flops = 6.83649e+07, min = 1.74802e+07, max = 5.08847e+07', '# - flop rate = 0.738321 GFlop/s', '# - bytes moved = 238.86 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.57962 GByte/s', '# - solve arithmetic intensity = 0.286213 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 21.980s; elapsed, 22.006s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.010s; elapsed, 22.026s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 21.980s; elapsed, 22.006s, #calls: 1', 'TOTAL : CPU, 21.980s; elapsed, 22.006s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 30.690s; elapsed, 30.711s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.116s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.040s; elapsed, 38.080s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.010s; elapsed, 22.026s, #calls: 1', 'TOTAL : CPU, 90.870s; elapsed, 90.934s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 10166.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 35.110s; elapsed, 17.676s', 'individual call time for EIGEN_LDLT: CPU, 39.410s; elapsed, 39.262s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.112s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2CPP -2', ' MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.197675', '# - matching time = 0.0692399', '# - symmetrization time = 0.043412', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.104828', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 15.6384', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70013e+10, min = 2.69076e+10, max = 7.00936e+10', '# - flop rate = 6.20276 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.94363e+10', '# --------------------------------------------', '# total = 9.94363e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.65231e-11\trel.res = 1.848e-15\tbw.error = 5.4308e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.121836', '# - total flops = 6.83712e+07, min = 1.74865e+07, max = 5.08847e+07', '# - flop rate = 0.561174 GFlop/s', '# - bytes moved = 239.112 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.96258 GByte/s', '# - solve arithmetic intensity = 0.285937 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 27.620s; elapsed, 16.194s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 31.950s; elapsed, 16.215s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 27.620s; elapsed, 16.194s, #calls: 1', 'TOTAL : CPU, 27.620s; elapsed, 16.194s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 35.110s; elapsed, 17.676s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.112s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.410s; elapsed, 39.262s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 31.950s; elapsed, 16.215s, #calls: 1', 'TOTAL : CPU, 106.640s; elapsed, 73.265s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 10166.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 55.970s; elapsed, 14.141s', 'individual call time for EIGEN_LDLT: CPU, 38.290s; elapsed, 37.779s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.099s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.186009', '# - matching time = 0.065407', '# - symmetrization time = 0.0403771', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.104122', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 14.6295', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70013e+10, min = 2.69076e+10, max = 7.00936e+10', '# - flop rate = 6.63053 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.94363e+10', '# --------------------------------------------', '# total = 9.94363e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.65479e-11\trel.res = 1.8486e-15\tbw.error = 5.4308e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0887661', '# - total flops = 6.83712e+07, min = 1.74865e+07, max = 5.08847e+07', '# - flop rate = 0.77024 GFlop/s', '# - bytes moved = 239.127 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.6939 GByte/s', '# - solve arithmetic intensity = 0.28592 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 41.930s; elapsed, 15.131s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 59.530s; elapsed, 15.149s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 41.930s; elapsed, 15.131s, #calls: 1', 'TOTAL : CPU, 41.930s; elapsed, 15.131s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 55.970s; elapsed, 14.141s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.099s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.290s; elapsed, 37.779s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 59.530s; elapsed, 15.149s, #calls: 1', 'TOTAL : CPU, 153.980s; elapsed, 67.168s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 10166.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 112.670s; elapsed, 14.158s', 'individual call time for EIGEN_LDLT: CPU, 47.010s; elapsed, 45.733s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.096s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.205432', '# - matching time = 0.0715861', '# - symmetrization time = 0.041888', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.110281', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 15.9251', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70013e+10, min = 2.69076e+10, max = 7.00937e+10', '# - flop rate = 6.09108 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.94363e+10', '# --------------------------------------------', '# total = 9.94363e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.65254e-11\trel.res = 1.84806e-15\tbw.error = 5.4308e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.123', '# - total flops = 6.83712e+07, min = 1.74865e+07, max = 5.08847e+07', '# - flop rate = 0.555863 GFlop/s', '# - bytes moved = 239.156 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.94436 GByte/s', '# - solve arithmetic intensity = 0.285885 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 88.670s; elapsed, 16.494s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 129.540s; elapsed, 16.512s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 88.670s; elapsed, 16.494s, #calls: 1', 'TOTAL : CPU, 88.670s; elapsed, 16.494s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 112.670s; elapsed, 14.158s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.096s, #calls: 1', 'time for EIGEN_LDLT: CPU, 47.010s; elapsed, 45.733s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 129.540s; elapsed, 16.512s, #calls: 1', 'TOTAL : CPU, 289.450s; elapsed, 76.500s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 10166.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 233.540s; elapsed, 14.654s', 'individual call time for EIGEN_LDLT: CPU, 40.550s; elapsed, 37.835s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.400s; elapsed, 0.103s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.21567', '# - matching time = 0.0882251', '# - symmetrization time = 0.04984', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.133991', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 16.6166', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70013e+10, min = 2.69076e+10, max = 7.00937e+10', '# - flop rate = 5.83762 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.94363e+10', '# --------------------------------------------', '# total = 9.94363e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.64901e-11\trel.res = 1.84721e-15\tbw.error = 5.4308e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.105698', '# - total flops = 6.83712e+07, min = 1.74865e+07, max = 5.08847e+07', '# - flop rate = 0.646854 GFlop/s', '# - bytes moved = 239.215 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.26319 GByte/s', '# - solve arithmetic intensity = 0.285815 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 173.960s; elapsed, 17.234s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 270.210s; elapsed, 17.253s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 173.960s; elapsed, 17.234s, #calls: 1', 'TOTAL : CPU, 173.960s; elapsed, 17.234s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 233.540s; elapsed, 14.654s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.400s; elapsed, 0.103s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.550s; elapsed, 37.835s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 270.210s; elapsed, 17.253s, #calls: 1', 'TOTAL : CPU, 544.700s; elapsed, 69.845s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 5083.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.270s; elapsed, 32.295s', 'individual call time for EIGEN_LDLT: CPU, 39.660s; elapsed, 39.698s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.121s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 67', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.203172', '# - matching time = 0.0693901', '# - symmetrization time = 0.0527458', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0828469', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 16.2377', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70012e+10, min = 2.55422e+07, max = 7.00681e+10', '# - flop rate = 5.97383 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.93428e+10', '# --------------------------------------------', '# total = 9.93428e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.94624e-11\trel.res = 2.40198e-15\tbw.error = 4.31906e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0929391', '# - total flops = 7.25822e+07, min = 133141, max = 5.07435e+07', '# - flop rate = 0.780964 GFlop/s', '# - bytes moved = 220.307 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.37045 GByte/s', '# - solve arithmetic intensity = 0.329459 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.720s; elapsed, 16.746s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 16.750s; elapsed, 16.761s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.750s; elapsed, 16.761s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.750s; elapsed, 16.766s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.720s; elapsed, 16.746s, #calls: 1', 'TOTAL : CPU, 16.720s; elapsed, 16.746s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.750s; elapsed, 16.761s, #calls: 1', 'TOTAL : CPU, 16.750s; elapsed, 16.761s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 16.750s; elapsed, 16.761s, #calls: 1', 'TOTAL : CPU, 16.750s; elapsed, 16.761s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.270s; elapsed, 32.295s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.121s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.660s; elapsed, 39.698s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.750s; elapsed, 16.766s, #calls: 1', 'TOTAL : CPU, 88.810s; elapsed, 88.880s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 5083.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 39.730s; elapsed, 19.924s', 'individual call time for EIGEN_LDLT: CPU, 38.510s; elapsed, 38.339s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.111s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.199401', '# - matching time = 0.068573', '# - symmetrization time = 0.059953', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0962582', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 14.2064', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70012e+10, min = 2.55422e+07, max = 7.00681e+10', '# - flop rate = 6.82797 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.93428e+10', '# --------------------------------------------', '# total = 9.93428e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.94247e-11\trel.res = 2.40107e-15\tbw.error = 4.31906e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.115793', '# - total flops = 7.25822e+07, min = 133141, max = 5.07435e+07', '# - flop rate = 0.626827 GFlop/s', '# - bytes moved = 220.321 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.90271 GByte/s', '# - solve arithmetic intensity = 0.329439 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 22.120s; elapsed, 14.756s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 24.990s; elapsed, 14.774s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.240s; elapsed, 14.772s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 28.970s; elapsed, 14.780s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 24.990s; elapsed, 14.774s, #calls: 1', 'TOTAL : CPU, 24.990s; elapsed, 14.774s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 22.120s; elapsed, 14.756s, #calls: 1', 'TOTAL : CPU, 22.120s; elapsed, 14.756s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.240s; elapsed, 14.772s, #calls: 1', 'TOTAL : CPU, 17.240s; elapsed, 14.772s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 39.730s; elapsed, 19.924s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.510s; elapsed, 38.339s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 28.970s; elapsed, 14.780s, #calls: 1', 'TOTAL : CPU, 107.370s; elapsed, 73.153s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 5083.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 55.470s; elapsed, 13.954s', 'individual call time for EIGEN_LDLT: CPU, 39.150s; elapsed, 38.612s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.107s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.200997', '# - matching time = 0.0708141', '# - symmetrization time = 0.052597', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.096231', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 13.8969', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70012e+10, min = 2.55441e+07, max = 7.00681e+10', '# - flop rate = 6.98007 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.93428e+10', '# --------------------------------------------', '# total = 9.93428e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.9437e-11\trel.res = 2.40137e-15\tbw.error = 4.31906e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.126255', '# - total flops = 7.25822e+07, min = 133141, max = 5.07435e+07', '# - flop rate = 0.574885 GFlop/s', '# - bytes moved = 220.338 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.74518 GByte/s', '# - solve arithmetic intensity = 0.329413 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 21.160s; elapsed, 14.467s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 32.420s; elapsed, 14.455s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 41.960s; elapsed, 14.469s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 56.070s; elapsed, 14.475s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 21.160s; elapsed, 14.467s, #calls: 1', 'TOTAL : CPU, 21.160s; elapsed, 14.467s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 32.420s; elapsed, 14.455s, #calls: 1', 'TOTAL : CPU, 32.420s; elapsed, 14.455s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 41.960s; elapsed, 14.469s, #calls: 1', 'TOTAL : CPU, 41.960s; elapsed, 14.469s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 55.470s; elapsed, 13.954s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.107s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.150s; elapsed, 38.612s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 56.070s; elapsed, 14.475s, #calls: 1', 'TOTAL : CPU, 150.870s; elapsed, 67.147s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 5083.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 144.060s; elapsed, 18.234s', 'individual call time for EIGEN_LDLT: CPU, 40.790s; elapsed, 39.540s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.097s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.236389', '# - matching time = 0.080327', '# - symmetrization time = 0.0596118', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.114193', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 14.7271', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70012e+10, min = 2.55442e+07, max = 7.00681e+10', '# - flop rate = 6.58658 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.93428e+10', '# --------------------------------------------', '# total = 9.93428e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.94182e-11\trel.res = 2.40091e-15\tbw.error = 4.31906e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.117395', '# - total flops = 7.25822e+07, min = 133141, max = 5.07435e+07', '# - flop rate = 0.618273 GFlop/s', '# - bytes moved = 220.371 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.87718 GByte/s', '# - solve arithmetic intensity = 0.329363 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 59.480s; elapsed, 15.347s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 28.560s; elapsed, 15.365s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 87.830s; elapsed, 15.364s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 118.690s; elapsed, 15.369s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 59.480s; elapsed, 15.347s, #calls: 1', 'TOTAL : CPU, 59.480s; elapsed, 15.347s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 28.560s; elapsed, 15.365s, #calls: 1', 'TOTAL : CPU, 28.560s; elapsed, 15.365s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 87.830s; elapsed, 15.364s, #calls: 1', 'TOTAL : CPU, 87.830s; elapsed, 15.364s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 144.060s; elapsed, 18.234s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.097s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.790s; elapsed, 39.540s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 118.690s; elapsed, 15.369s, #calls: 1', 'TOTAL : CPU, 303.790s; elapsed, 73.241s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 2541.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 36.880s; elapsed, 36.943s', 'individual call time for EIGEN_LDLT: CPU, 38.530s; elapsed, 38.563s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.113s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 66', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.195898', '# - matching time = 0.0672462', '# - symmetrization time = 0.0305309', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0815339', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 16.6121', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70012e+10, min = 2.55422e+07, max = 7.00086e+10', '# - flop rate = 5.83921 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.915e+10', '# --------------------------------------------', '# total = 9.915e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.02125e-10\trel.res = 2.46627e-15\tbw.error = 4.30241e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.081059', '# - total flops = 8.29426e+07, min = 120436, max = 5.04792e+07', '# - flop rate = 1.02324 GFlop/s', '# - bytes moved = 210.5 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.59687 GByte/s', '# - solve arithmetic intensity = 0.394027 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.060s; elapsed, 17.087s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.030s; elapsed, 17.088s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.060s; elapsed, 17.079s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.040s; elapsed, 17.088s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.060s; elapsed, 17.087s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.070s; elapsed, 17.090s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.060s; elapsed, 17.082s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.060s; elapsed, 17.093s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.060s; elapsed, 17.087s, #calls: 1', 'TOTAL : CPU, 17.060s; elapsed, 17.087s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.030s; elapsed, 17.088s, #calls: 1', 'TOTAL : CPU, 17.030s; elapsed, 17.088s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.060s; elapsed, 17.079s, #calls: 1', 'TOTAL : CPU, 17.060s; elapsed, 17.079s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.040s; elapsed, 17.088s, #calls: 1', 'TOTAL : CPU, 17.040s; elapsed, 17.088s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.060s; elapsed, 17.087s, #calls: 1', 'TOTAL : CPU, 17.060s; elapsed, 17.087s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.070s; elapsed, 17.090s, #calls: 1', 'TOTAL : CPU, 17.070s; elapsed, 17.090s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.060s; elapsed, 17.082s, #calls: 1', 'TOTAL : CPU, 17.060s; elapsed, 17.082s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 36.880s; elapsed, 36.943s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.113s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.530s; elapsed, 38.563s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.060s; elapsed, 17.093s, #calls: 1', 'TOTAL : CPU, 92.580s; elapsed, 92.711s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 2541.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 35.660s; elapsed, 17.872s', 'individual call time for EIGEN_LDLT: CPU, 39.590s; elapsed, 39.418s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.110s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 8 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.217309', '# - matching time = 0.0769148', '# - symmetrization time = 0.0293219', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.096776', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 14.8886', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70012e+10, min = 2.55422e+07, max = 7.00086e+10', '# - flop rate = 6.51514 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.915e+10', '# --------------------------------------------', '# total = 9.915e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.02122e-10\trel.res = 2.46621e-15\tbw.error = 4.30241e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.077673', '# - total flops = 8.29426e+07, min = 120436, max = 5.04792e+07', '# - flop rate = 1.06784 GFlop/s', '# - bytes moved = 210.509 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.71019 GByte/s', '# - solve arithmetic intensity = 0.394011 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.130s; elapsed, 15.404s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 20.580s; elapsed, 15.398s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 25.620s; elapsed, 15.406s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.740s; elapsed, 15.405s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 21.180s; elapsed, 15.397s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.600s; elapsed, 15.404s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.430s; elapsed, 15.404s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 30.190s; elapsed, 15.409s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 25.620s; elapsed, 15.406s, #calls: 1', 'TOTAL : CPU, 25.620s; elapsed, 15.406s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.130s; elapsed, 15.404s, #calls: 1', 'TOTAL : CPU, 17.130s; elapsed, 15.404s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 20.580s; elapsed, 15.398s, #calls: 1', 'TOTAL : CPU, 20.580s; elapsed, 15.398s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.740s; elapsed, 15.405s, #calls: 1', 'TOTAL : CPU, 17.740s; elapsed, 15.405s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 21.180s; elapsed, 15.397s, #calls: 1', 'TOTAL : CPU, 21.180s; elapsed, 15.397s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.600s; elapsed, 15.404s, #calls: 1', 'TOTAL : CPU, 17.600s; elapsed, 15.404s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.430s; elapsed, 15.404s, #calls: 1', 'TOTAL : CPU, 17.430s; elapsed, 15.404s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 35.660s; elapsed, 17.872s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.110s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.590s; elapsed, 39.418s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 30.190s; elapsed, 15.409s, #calls: 1', 'TOTAL : CPU, 105.580s; elapsed, 72.808s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 2541.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 65.430s; elapsed, 16.503s', 'individual call time for EIGEN_LDLT: CPU, 39.240s; elapsed, 38.716s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.104s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 66', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.233801', '# - matching time = 0.0823982', '# - symmetrization time = 0.0446789', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.099885', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 15.9705', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70012e+10, min = 2.55441e+07, max = 7.00086e+10', '# - flop rate = 6.07379 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.915e+10', '# --------------------------------------------', '# total = 9.915e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.02113e-10\trel.res = 2.46598e-15\tbw.error = 4.30241e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0909791', '# - total flops = 8.29426e+07, min = 120436, max = 5.04792e+07', '# - flop rate = 0.911666 GFlop/s', '# - bytes moved = 210.519 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.31393 GByte/s', '# - solve arithmetic intensity = 0.39399 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 23.950s; elapsed, 16.543s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 23.500s; elapsed, 16.543s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 22.780s; elapsed, 16.544s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 22.070s; elapsed, 16.544s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 46.160s; elapsed, 16.543s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 34.300s; elapsed, 16.535s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 30.740s; elapsed, 16.535s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 64.170s; elapsed, 16.550s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 46.160s; elapsed, 16.543s, #calls: 1', 'TOTAL : CPU, 46.160s; elapsed, 16.543s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 34.300s; elapsed, 16.535s, #calls: 1', 'TOTAL : CPU, 34.300s; elapsed, 16.535s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 22.780s; elapsed, 16.544s, #calls: 1', 'TOTAL : CPU, 22.780s; elapsed, 16.544s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 22.070s; elapsed, 16.544s, #calls: 1', 'TOTAL : CPU, 22.070s; elapsed, 16.544s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 23.500s; elapsed, 16.543s, #calls: 1', 'TOTAL : CPU, 23.500s; elapsed, 16.543s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 23.950s; elapsed, 16.543s, #calls: 1', 'TOTAL : CPU, 23.950s; elapsed, 16.543s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 30.740s; elapsed, 16.535s, #calls: 1', 'TOTAL : CPU, 30.740s; elapsed, 16.535s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 65.430s; elapsed, 16.503s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.104s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.240s; elapsed, 38.716s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 64.170s; elapsed, 16.550s, #calls: 1', 'TOTAL : CPU, 169.020s; elapsed, 71.873s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 1270.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 33.910s; elapsed, 33.928s', 'individual call time for EIGEN_LDLT: CPU, 40.000s; elapsed, 40.019s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.131s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 16 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 65', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.225481', '# - matching time = 0.0781739', '# - symmetrization time = 0.0169551', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0908329', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 13.6496', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70012e+10, min = 2.37245e+07, max = 6.98999e+10', '# - flop rate = 7.10653 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.8763e+10', '# --------------------------------------------', '# total = 9.8763e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.17627e-11\trel.res = 2.21604e-15\tbw.error = 4.05167e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.052248', '# - total flops = 1.04874e+08, min = 110416, max = 4.99919e+07', '# - flop rate = 2.00723 GFlop/s', '# - bytes moved = 201.842 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.86315 GByte/s', '# - solve arithmetic intensity = 0.519584 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 14.100s; elapsed, 14.124s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 14.070s; elapsed, 14.128s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 14.100s; elapsed, 14.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 14.050s; elapsed, 14.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 14.050s; elapsed, 14.129s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 14.070s; elapsed, 14.122s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 14.060s; elapsed, 14.123s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 14.090s; elapsed, 14.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 14.110s; elapsed, 14.126s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 14.070s; elapsed, 14.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 14.090s; elapsed, 14.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 14.090s; elapsed, 14.126s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 14.100s; elapsed, 14.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 14.100s; elapsed, 14.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 14.080s; elapsed, 14.124s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 14.120s; elapsed, 14.133s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 14.100s; elapsed, 14.124s, #calls: 1', 'TOTAL : CPU, 14.100s; elapsed, 14.124s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 14.090s; elapsed, 14.127s, #calls: 1', 'TOTAL : CPU, 14.090s; elapsed, 14.127s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 14.070s; elapsed, 14.128s, #calls: 1', 'TOTAL : CPU, 14.070s; elapsed, 14.128s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 14.050s; elapsed, 14.129s, #calls: 1', 'TOTAL : CPU, 14.050s; elapsed, 14.129s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 14.110s; elapsed, 14.126s, #calls: 1', 'TOTAL : CPU, 14.110s; elapsed, 14.126s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 14.090s; elapsed, 14.127s, #calls: 1Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 14.060s; elapsed, 14.123s, #calls: 1', 'TOTAL : CPU, 14.060s; elapsed, 14.123s', '', 'TOTAL : CPU, 14.090s; elapsed, 14.127s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 14.100s; elapsed, 14.127s, #calls: 1', 'TOTAL : CPU, 14.100s; elapsed, 14.127s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 14.070s; elapsed, 14.122s, #calls: 1', 'TOTAL : CPU, 14.070s; elapsed, 14.122s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 14.050s; elapsed, 14.127s, #calls: 1', 'TOTAL : CPU, 14.050s; elapsed, 14.127s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 14.100s; elapsed, 14.127s, #calls: 1', 'TOTAL : CPU, 14.100s; elapsed, 14.127s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 14.090s; elapsed, 14.126s, #calls: 1', 'TOTAL : CPU, 14.090s; elapsed, 14.126s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 14.100s; elapsed, 14.127s, #calls: 1', 'TOTAL : CPU, 14.100s; elapsed, 14.127s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 14.080s; elapsed, 14.124s, #calls: 1', 'TOTAL : CPU, 14.080s; elapsed, 14.124s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 14.070s; elapsed, 14.127s, #calls: 1', 'TOTAL : CPU, 14.070s; elapsed, 14.127s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 33.910s; elapsed, 33.928s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.131s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.000s; elapsed, 40.019s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 14.120s; elapsed, 14.133s, #calls: 1', 'TOTAL : CPU, 88.150s; elapsed, 88.211s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '20333 1270.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 46.200s; elapsed, 23.264s', 'individual call time for EIGEN_LDLT: CPU, 42.770s; elapsed, 42.625s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.129s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 20,333', '# - number of nonzeros = 603,311', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,105', '# - number of levels = 65', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.229821', '# - matching time = 0.0801702', '# - symmetrization time = 0.020319', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,105', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0992281', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 253.199 MB', '# - factor time = 15.4485', '# - factor nonzeros = 31,649,935', '# - factor memory = 253.199 MB', '# - total flops = 9.70012e+10, min = 2.37245e+07, max = 6.98999e+10', '# - flop rate = 6.279 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.8763e+10', '# --------------------------------------------', '# total = 9.8763e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.17589e-11\trel.res = 2.21594e-15\tbw.error = 4.05167e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.064244', '# - total flops = 1.04874e+08, min = 110416, max = 4.99919e+07', '# - flop rate = 1.63243 GFlop/s', '# - bytes moved = 201.847 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.14188 GByte/s', '# - solve arithmetic intensity = 0.519571 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.420s; elapsed, 15.958s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.410s; elapsed, 15.959s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 18.150s; elapsed, 15.959s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 20.330s; elapsed, 15.953s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.660s; elapsed, 15.959s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.360s; elapsed, 15.958s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.960s; elapsed, 15.958s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 21.690s; elapsed, 15.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.730s; elapsed, 15.958s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.760s; elapsed, 15.960s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 25.760s; elapsed, 15.960s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 18.560s; elapsed, 15.958s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 18.220s; elapsed, 15.957s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 18.420s; elapsed, 15.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 20.020s; elapsed, 15.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 31.330s; elapsed, 15.966s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.420s; elapsed, 15.958s, #calls: 1', 'TOTAL : CPU, 17.420s; elapsed, 15.958s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 25.760s; elapsed, 15.960s, #calls: 1', 'TOTAL : CPU, 25.760s; elapsed, 15.960s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 21.690s; elapsed, 15.954s, #calls: 1', 'TOTAL : CPU, 21.690s; elapsed, 15.954s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.410s; elapsed, 15.959s, #calls: 1', 'TOTAL : CPU, 17.410s; elapsed, 15.959s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.660s; elapsed, 15.959s, #calls: 1', 'TOTAL : CPU, 18.660s; elapsed, 15.959s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.760s; elapsed, 15.960s, #calls: 1', 'TOTAL : CPU, 17.760s; elapsed, 15.960s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 18.150s; elapsed, 15.959s, #calls: 1', 'TOTAL : CPU, 18.150s; elapsed, 15.959s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.960s; elapsed, 15.958s, #calls: 1', 'TOTAL : CPU, 17.960s; elapsed, 15.958s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 18.560s; elapsed, 15.958s, #calls: 1', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 18.420s; elapsed, 15.954s, #calls: 1', 'TOTAL : CPU, 18.420s; elapsed, 15.954s', 'TOTAL : CPU, 18.560s; elapsed, 15.958s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 20.020s; elapsed, 15.954s, #calls: 1', 'TOTAL : CPU, 20.020s; elapsed, 15.954s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 20.330s; elapsed, 15.953s, #calls: 1', 'TOTAL : CPU, 20.330s; elapsed, 15.953s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.360s; elapsed, 15.958s, #calls: 1', 'TOTAL : CPU, 18.360s; elapsed, 15.958s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.730s; elapsed, 15.958s, #calls: 1', 'TOTAL : CPU, 17.730s; elapsed, 15.958s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 18.220s; elapsed, 15.957s, #calls: 1', 'TOTAL : CPU, 18.220s; elapsed, 15.957s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 46.200s; elapsed, 23.264s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.129s, #calls: 1', 'time for EIGEN_LDLT: CPU, 42.770s; elapsed, 42.625s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 31.330s; elapsed, 15.966s, #calls: 1', 'TOTAL : CPU, 120.480s; elapsed, 81.984s']
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-/A_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-/b_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 142.860s; elapsed, 142.986s', 'individual call time for EIGEN_LDLT: CPU, 292.710s; elapsed, 292.887s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.282s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 142.860s; elapsed, 142.986s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.282s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.710s; elapsed, 292.887s, #calls: 1', 'TOTAL : CPU, 435.860s; elapsed, 436.155s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 182.690s; elapsed, 92.966s', 'individual call time for EIGEN_LDLT: CPU, 292.910s; elapsed, 292.937s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.390s; elapsed, 0.250s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 182.690s; elapsed, 92.966s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.390s; elapsed, 0.250s, #calls: 1', 'time for EIGEN_LDLT: CPU, 292.910s; elapsed, 292.937s, #calls: 1', 'TOTAL : CPU, 475.990s; elapsed, 386.153s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 326.740s; elapsed, 83.028s', 'individual call time for EIGEN_LDLT: CPU, 293.430s; elapsed, 293.076s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.226s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 326.740s; elapsed, 83.028s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.226s, #calls: 1', 'time for EIGEN_LDLT: CPU, 293.430s; elapsed, 293.076s, #calls: 1', 'TOTAL : CPU, 620.670s; elapsed, 376.330s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 695.260s; elapsed, 88.480s', 'individual call time for EIGEN_LDLT: CPU, 306.330s; elapsed, 305.722s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.710s; elapsed, 0.223s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 695.260s; elapsed, 88.480s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.710s; elapsed, 0.223s, #calls: 1', 'time for EIGEN_LDLT: CPU, 306.330s; elapsed, 305.722s, #calls: 1', 'TOTAL : CPU, 1002.300s; elapsed, 394.425s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1519.880s; elapsed, 96.475s', 'individual call time for EIGEN_LDLT: CPU, 295.600s; elapsed, 293.198s', 'individual call time for EIGEN_BICGSTAB: CPU, 1.040s; elapsed, 0.190s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1519.880s; elapsed, 96.475s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 1.040s; elapsed, 0.190s, #calls: 1', 'time for EIGEN_LDLT: CPU, 295.600s; elapsed, 293.198s, #calls: 1', 'TOTAL : CPU, 1816.520s; elapsed, 389.862s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 150.270s; elapsed, 150.371s', 'individual call time for EIGEN_LDLT: CPU, 370.420s; elapsed, 370.680s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.520s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,197', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.20204', '# - matching time = 0.145009', '# - symmetrization time = 0.100298', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.153986', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.45 MB', '# - factor time = 140.958', '# - factor nonzeros = 113,056,288', '# - factor memory = 904.45 MB', '# - total flops = 7.18415e+11, min = 41770, max = 7.18415e+11', '# - flop rate = 5.09666 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28335e+11', '# --------------------------------------------', '# total = 7.28335e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.20396e-10\trel.res = 2.87732e-15\tbw.error = 7.77224e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.278585', '# - total flops = 2.36199e+08, min = 177542, max = 2.36022e+08', '# - flop rate = 0.847853 GFlop/s', '# - bytes moved = 1221.56 MB, min = 0 MB, max = 0 MB', '# - byte rate = 4.38489 GByte/s', '# - solve arithmetic intensity = 0.193358 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 142.910s; elapsed, 143.026s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 143.010s; elapsed, 143.012s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 142.910s; elapsed, 143.026s, #calls: 1', 'TOTAL : CPU, 142.910s; elapsed, 143.026s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 150.270s; elapsed, 150.371s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.520s, #calls: 1', 'time for EIGEN_LDLT: CPU, 370.420s; elapsed, 370.680s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 143.010s; elapsed, 143.012s, #calls: 1', 'TOTAL : CPU, 664.220s; elapsed, 664.582s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 209.060s; elapsed, 105.904s', 'individual call time for EIGEN_LDLT: CPU, 293.220s; elapsed, 293.217s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.370s; elapsed, 0.248s', 'CPP -2', '# Initializing STRUMPACKCPP -2', '', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,197', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.14095', '# - matching time = 0.125592', '# - symmetrization time = 0.1009', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.157842', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.45 MB', '# - factor time = 76.074', '# - factor nonzeros = 113,056,288', '# - factor memory = 904.45 MB', '# - total flops = 7.18416e+11, min = 41770, max = 7.18416e+11', '# - flop rate = 9.44365 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28335e+11', '# --------------------------------------------', '# total = 7.28335e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.1636e-10\trel.res = 2.78086e-15\tbw.error = 7.76343e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.298569', '# - total flops = 2.36293e+08, min = 177542, max = 2.36116e+08', '# - flop rate = 0.791419 GFlop/s', '# - bytes moved = 1223.99 MB, min = 0 MB, max = 0 MB', '# - byte rate = 4.09953 GByte/s', '# - solve arithmetic intensity = 0.193051 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 154.700s; elapsed, 78.070s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 79.180s; elapsed, 78.062s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 154.700s; elapsed, 78.070s, #calls: 1', 'TOTAL : CPU, 154.700s; elapsed, 78.070s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 209.060s; elapsed, 105.904s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.370s; elapsed, 0.248s, #calls: 1', 'time for EIGEN_LDLT: CPU, 293.220s; elapsed, 293.217s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 79.180s; elapsed, 78.062s, #calls: 1', 'TOTAL : CPU, 581.830s; elapsed, 477.431s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 263.200s; elapsed, 67.225s', 'individual call time for EIGEN_LDLT: CPU, 293.070s; elapsed, 292.847s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.510s; elapsed, 0.239s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,197', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.22281', '# - matching time = 0.145561', '# - symmetrization time = 0.104321', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.144889', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.45 MB', '# - factor time = 66.8679', '# - factor nonzeros = 113,056,288', '# - factor memory = 904.45 MB', '# - total flops = 7.18416e+11, min = 41770, max = 7.18416e+11', '# - flop rate = 10.7438 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28335e+11', '# --------------------------------------------', '# total = 7.28335e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.15759e-10\trel.res = 2.7665e-15\tbw.error = 7.75717e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.463599', '# - total flops = 2.36293e+08, min = 177542, max = 2.36116e+08', '# - flop rate = 0.509693 GFlop/s', '# - bytes moved = 1225.24 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.64289 GByte/s', '# - solve arithmetic intensity = 0.192855 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 268.890s; elapsed, 69.123s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 73.010s; elapsed, 69.120s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 268.890s; elapsed, 69.123s, #calls: 1', 'TOTAL : CPU, 268.890s; elapsed, 69.123s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 263.200s; elapsed, 67.225s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.510s; elapsed, 0.239s, #calls: 1', 'time for EIGEN_LDLT: CPU, 293.070s; elapsed, 292.847s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 73.010s; elapsed, 69.120s, #calls: 1', 'TOTAL : CPU, 629.790s; elapsed, 429.431s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 506.160s; elapsed, 64.834s', 'individual call time for EIGEN_LDLT: CPU, 294.610s; elapsed, 293.457s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.620s; elapsed, 0.198s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,197', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.16933', '# - matching time = 0.156434', '# - symmetrization time = 0.119043', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.143716', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.45 MB', '# - factor time = 59.2749', '# - factor nonzeros = 113,056,288', '# - factor memory = 904.45 MB', '# - total flops = 7.18416e+11, min = 42535, max = 7.18416e+11', '# - flop rate = 12.1201 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28335e+11', '# --------------------------------------------', '# total = 7.28335e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.15744e-10\trel.res = 2.76614e-15\tbw.error = 7.75717e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.365308', '# - total flops = 2.36293e+08, min = 177542, max = 2.36116e+08', '# - flop rate = 0.646833 GFlop/s', '# - bytes moved = 1225.31 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.35418 GByte/s', '# - solve arithmetic intensity = 0.192844 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 478.830s; elapsed, 61.506s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 70.660s; elapsed, 61.499s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 478.830s; elapsed, 61.506s, #calls: 1', 'TOTAL : CPU, 478.830s; elapsed, 61.506s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 506.160s; elapsed, 64.834s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.620s; elapsed, 0.198s, #calls: 1', 'time for EIGEN_LDLT: CPU, 294.610s; elapsed, 293.457s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 70.660s; elapsed, 61.499s, #calls: 1', 'TOTAL : CPU, 872.050s; elapsed, 419.988s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1570.960s; elapsed, 99.793s', 'individual call time for EIGEN_LDLT: CPU, 296.150s; elapsed, 293.632s', 'individual call time for EIGEN_BICGSTAB: CPU, 1.010s; elapsed, 0.195s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,197', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.18803', '# - matching time = 0.203366', '# - symmetrization time = 0.136586', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.168739', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 904.45 MB', '# - factor time = 65.0366', '# - factor nonzeros = 113,056,288', '# - factor memory = 904.45 MB', '# - total flops = 7.1842e+11, min = 43208, max = 7.1842e+11', '# - flop rate = 11.0464 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.28335e+11', '# --------------------------------------------', '# total = 7.28335e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.15728e-10\trel.res = 2.76578e-15\tbw.error = 7.75717e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.389212', '# - total flops = 2.36302e+08, min = 177542, max = 2.36125e+08', '# - flop rate = 0.60713 GFlop/s', '# - bytes moved = 1225.6 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.14893 GByte/s', '# - solve arithmetic intensity = 0.192806 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1053.250s; elapsed, 67.342s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 87.110s; elapsed, 67.334s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1053.250s; elapsed, 67.342s, #calls: 1', 'TOTAL : CPU, 1053.250s; elapsed, 67.342s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1570.960s; elapsed, 99.793s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 1.010s; elapsed, 0.195s, #calls: 1', 'time for EIGEN_LDLT: CPU, 296.150s; elapsed, 293.632s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 87.110s; elapsed, 67.334s, #calls: 1', 'TOTAL : CPU, 1955.230s; elapsed, 460.954s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 148.130s; elapsed, 148.247s', 'individual call time for EIGEN_LDLT: CPU, 297.780s; elapsed, 297.909s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.278s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,205', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.17228', '# - matching time = 0.129232', '# - symmetrization time = 0.0955942', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,205', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.102859', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.151 MB', '# - factor time = 72.9503', '# - factor nonzeros = 113,143,898', '# - factor memory = 905.151 MB', '# - total flops = 7.20027e+11, min = 41770, max = 5.52422e+11', '# - flop rate = 9.8701 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2958e+11', '# --------------------------------------------', '# total = 7.2958e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.10513e-10\trel.res = 2.64113e-15\tbw.error = 7.89018e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.219198', '# - total flops = 2.52082e+08, min = 133387, max = 1.8924e+08', '# - flop rate = 1.15002 GFlop/s', '# - bytes moved = 460.028 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.09869 GByte/s', '# - solve arithmetic intensity = 0.547971 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 74.730s; elapsed, 74.789s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 74.700s; elapsed, 74.762s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 74.700s; elapsed, 74.756s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 74.780s; elapsed, 74.774s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 74.730s; elapsed, 74.789s, #calls: 1', 'TOTAL : CPU, 74.730s; elapsed, 74.789s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 74.700s; elapsed, 74.762s, #calls: 1', 'TOTAL : CPU, 74.700s; elapsed, 74.762s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 74.700s; elapsed, 74.756s, #calls: 1', 'TOTAL : CPU, 74.700s; elapsed, 74.756s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 148.130s; elapsed, 148.247s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.278s, #calls: 1', 'time for EIGEN_LDLT: CPU, 297.780s; elapsed, 297.909s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 74.780s; elapsed, 74.774s, #calls: 1', 'TOTAL : CPU, 520.970s; elapsed, 521.208s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 230.170s; elapsed, 118.565s', 'individual call time for EIGEN_LDLT: CPU, 314.690s; elapsed, 314.766s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.247s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,205', '# - number of levels = 129', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.12244', '# - matching time = 0.132566', '# - symmetrization time = 0.09726', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,205', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.119828', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.151 MB', '# - factor time = 62.0615', '# - factor nonzeros = 113,143,898', '# - factor memory = 905.151 MB', '# - total flops = 7.20027e+11, min = 41770, max = 5.52422e+11', '# - flop rate = 11.6018 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2958e+11', '# --------------------------------------------', '# total = 7.2958e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.10477e-10\trel.res = 2.64028e-15\tbw.error = 7.88507e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.206869', '# - total flops = 2.52099e+08, min = 133387, max = 1.8924e+08', '# - flop rate = 1.21864 GFlop/s', '# - bytes moved = 460.383 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.22548 GByte/s', '# - solve arithmetic intensity = 0.547585 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 92.820s; elapsed, 63.858s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 124.700s; elapsed, 63.834s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 116.590s; elapsed, 63.825s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 64.790s; elapsed, 63.844s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 116.590s; elapsed, 63.825s, #calls: 1', 'TOTAL : CPU, 116.590s; elapsed, 63.825s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 92.820s; elapsed, 63.858s, #calls: 1', 'TOTAL : CPU, 92.820s; elapsed, 63.858s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 124.700s; elapsed, 63.834s, #calls: 1', 'TOTAL : CPU, 124.700s; elapsed, 63.834s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 230.170s; elapsed, 118.565s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.247s, #calls: 1', 'time for EIGEN_LDLT: CPU, 314.690s; elapsed, 314.766s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 64.790s; elapsed, 63.844s, #calls: 1', 'TOTAL : CPU, 610.030s; elapsed, 497.423s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 267.810s; elapsed, 69.161s', 'individual call time for EIGEN_LDLT: CPU, 310.330s; elapsed, 310.086s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.259s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,205', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.15956', '# - matching time = 0.158444', '# - symmetrization time = 0.107268', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,205', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.135009', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.151 MB', '# - factor time = 61.1852', '# - factor nonzeros = 113,143,898', '# - factor memory = 905.151 MB', '# - total flops = 7.20027e+11, min = 41770, max = 5.52422e+11', '# - flop rate = 11.768 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2958e+11', '# --------------------------------------------', '# total = 7.2958e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.1046e-10\trel.res = 2.63986e-15\tbw.error = 7.88507e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.231617', '# - total flops = 2.52099e+08, min = 133387, max = 1.8924e+08', '# - flop rate = 1.08843 GFlop/s', '# - bytes moved = 460.395 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.98774 GByte/s', '# - solve arithmetic intensity = 0.54757 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 142.980s; elapsed, 63.114s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 243.670s; elapsed, 63.089s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 203.220s; elapsed, 63.084s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 66.190s; elapsed, 63.099s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 243.670s; elapsed, 63.089s, #calls: 1', 'TOTAL : CPU, 243.670s; elapsed, 63.089s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 203.220s; elapsed, 63.084s, #calls: 1', 'TOTAL : CPU, 203.220s; elapsed, 63.084s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 142.980s; elapsed, 63.114s, #calls: 1', 'TOTAL : CPU, 142.980s; elapsed, 63.114s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 267.810s; elapsed, 69.161s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.259s, #calls: 1', 'time for EIGEN_LDLT: CPU, 310.330s; elapsed, 310.086s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 66.190s; elapsed, 63.099s, #calls: 1', 'TOTAL : CPU, 644.880s; elapsed, 442.605s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 783.020s; elapsed, 99.964s', 'individual call time for EIGEN_LDLT: CPU, 298.390s; elapsed, 297.408s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.650s; elapsed, 0.210s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,205', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************# - nd time = ', '1.18529', '# - matching time = 0.152289', '# - symmetrization time = 0.109042', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,205', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.134842', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.151 MB', '# - factor time = 69.0459', '# - factor nonzeros = 113,143,898', '# - factor memory = 905.151 MB', '# - total flops = 7.20027e+11, min = 42535, max = 5.52422e+11', '# - flop rate = 10.4282 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2958e+11', '# --------------------------------------------', '# total = 7.2958e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.10436e-10\trel.res = 2.6393e-15\tbw.error = 7.88507e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.330425', '# - total flops = 2.52099e+08, min = 133387, max = 1.8924e+08', '# - flop rate = 0.762953 GFlop/s', '# - bytes moved = 460.421 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.39342 GByte/s', '# - solve arithmetic intensity = 0.54754 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 310.990s; elapsed, 71.110s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 463.060s; elapsed, 71.078s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 540.750s; elapsed, 71.086s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 79.020s; elapsed, 71.097s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 463.060s; elapsed, 71.078s, #calls: 1', 'TOTAL : CPU, 463.060s; elapsed, 71.078s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 540.750s; elapsed, 71.086s, #calls: 1', 'TOTAL : CPU, 540.750s; elapsed, 71.086s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 310.990s; elapsed, 71.110s, #calls: 1', 'TOTAL : CPU, 310.990s; elapsed, 71.110s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 783.020s; elapsed, 99.964s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.650s; elapsed, 0.210s, #calls: 1', 'time for EIGEN_LDLT: CPU, 298.390s; elapsed, 297.408s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 79.020s; elapsed, 71.097s, #calls: 1', 'TOTAL : CPU, 1161.080s; elapsed, 468.679s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 4415.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 150.690s; elapsed, 150.815s', 'individual call time for EIGEN_LDLT: CPU, 302.900s; elapsed, 302.983s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.285s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,181', '# - number of levels = 128', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '1.20141', '# - matching time = 0.13265', '# - symmetrization time = 0.0661378', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,181', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.114449', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.228 MB', '# - factor time = 71.1269', '# - factor nonzeros = 113,153,484', '# - factor memory = 905.228 MB', '# - total flops = 7.19774e+11, min = 41800, max = 5.51872e+11', '# - flop rate = 10.1196 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2855e+11', '# --------------------------------------------', '# total = 7.2855e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.24362e-11\trel.res = 1.97013e-15\tbw.error = 6.11855e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.197882', '# - total flops = 2.88072e+08, min = 111320, max = 1.88757e+08', '# - flop rate = 1.45578 GFlop/s', '# - bytes moved = 434.79 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.19721 GByte/s', '# - solve arithmetic intensity = 0.662556 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 72.850s; elapsed, 72.930s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 72.900s; elapsed, 72.928s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 72.870s; elapsed, 72.926s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 72.900s; elapsed, 72.931s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 72.830s; elapsed, 72.923s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 72.840s; elapsed, 72.945s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 72.900s; elapsed, 72.945s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 72.930s; elapsed, 72.941s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 72.850s; elapsed, 72.930s, #calls: 1', 'TOTAL : CPU, 72.850s; elapsed, 72.930s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 72.870s; elapsed, 72.926s, #calls: 1', 'TOTAL : CPU, 72.870s; elapsed, 72.926s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 72.900s; elapsed, 72.928s, #calls: 1', 'TOTAL : CPU, 72.900s; elapsed, 72.928s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 72.900s; elapsed, 72.945s, #calls: 1', 'TOTAL : CPU, 72.900s; elapsed, 72.945s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 72.830s; elapsed, 72.923s, #calls: 1', 'TOTAL : CPU, 72.830s; elapsed, 72.923s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 72.900s; elapsed, 72.931s, #calls: 1', 'TOTAL : CPU, 72.900s; elapsed, 72.931s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 72.840s; elapsed, 72.945s, #calls: 1', 'TOTAL : CPU, 72.840s; elapsed, 72.945s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 150.690s; elapsed, 150.815s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.285s, #calls: 1', 'time for EIGEN_LDLT: CPU, 302.900s; elapsed, 302.983s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 72.930s; elapsed, 72.941s, #calls: 1', 'TOTAL : CPU, 526.810s; elapsed, 527.025s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 4415.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 214.450s; elapsed, 108.659s', 'individual call time for EIGEN_LDLT: CPU, 314.730s; elapsed, 314.765s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.254s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,181', '# - number of levels = 128', '# - nd time = 1.16882', '# - matching time = 0.135947', '# - symmetrization time = 0.0658722', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,181', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.121035', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.228 MB', '# - factor time = 60.7569', '# - factor nonzeros = 113,153,484', '# - factor memory = 905.228 MB', '# - total flops = 7.19774e+11, min = 41800, max = 5.51872e+11', '# - flop rate = 11.8468 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2855e+11', '# --------------------------------------------', '# total = 7.2855e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.22963e-11\trel.res = 1.96679e-15\tbw.error = 6.11855e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.149498', '# - total flops = 2.88072e+08, min = 111320, max = 1.88757e+08', '# - flop rate = 1.92693 GFlop/s', '# - bytes moved = 434.803 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.90842 GByte/s', '# - solve arithmetic intensity = 0.662536 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 81.040s; elapsed, 62.498s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 120.870s; elapsed, 62.486s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 72.730s; elapsed, 62.487s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 72.370s; elapsed, 62.483s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 74.170s; elapsed, 62.484s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 110.850s; elapsed, 62.481s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 86.950s; elapsed, 62.500s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 63.380s; elapsed, 62.496s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 120.870s; elapsed, 62.486s, #calls: 1', 'TOTAL : CPU, 120.870s; elapsed, 62.486s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 72.730s; elapsed, 62.487s, #calls: 1', 'TOTAL : CPU, 72.730s; elapsed, 62.487s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 110.850s; elapsed, 62.481s, #calls: 1', 'TOTAL : CPU, 110.850s; elapsed, 62.481s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 72.370s; elapsed, 62.483s, #calls: 1', 'TOTAL : CPU, 72.370s; elapsed, 62.483s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 74.170s; elapsed, 62.484s, #calls: 1', 'TOTAL : CPU, 74.170s; elapsed, 62.484s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 86.950s; elapsed, 62.500s, #calls: 1', 'TOTAL : CPU, 86.950s; elapsed, 62.500s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 81.040s; elapsed, 62.498s, #calls: 1', 'TOTAL : CPU, 81.040s; elapsed, 62.498s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 214.450s; elapsed, 108.659s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.254s, #calls: 1', 'time for EIGEN_LDLT: CPU, 314.730s; elapsed, 314.765s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 63.380s; elapsed, 62.496s, #calls: 1', 'TOTAL : CPU, 592.940s; elapsed, 486.174s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 4415.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 258.760s; elapsed, 66.629s', 'individual call time for EIGEN_LDLT: CPU, 308.160s; elapsed, 307.768s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.490s; elapsed, 0.220s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 8 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,181', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.22428', '# - matching time = 0.153419', '# - symmetrization time = 0.0756512', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,181', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.120191', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.228 MB', '# - factor time = 65.8296', '# - factor nonzeros = 113,153,484', '# - factor memory = 905.228 MB', '# - total flops = 7.19774e+11, min = 41800, max = 5.51872e+11', '# - flop rate = 10.9339 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.2855e+11', '# --------------------------------------------', '# total = 7.2855e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.22801e-11\trel.res = 1.9664e-15\tbw.error = 6.11855e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.212694', '# - total flops = 2.88072e+08, min = 111320, max = 1.88757e+08', '# - flop rate = 1.3544 GFlop/s', '# - bytes moved = 434.821 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.04435 GByte/s', '# - solve arithmetic intensity = 0.662508 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 118.050s; elapsed, 67.735s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 156.850s; elapsed, 67.736s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 94.270s; elapsed, 67.722s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 235.640s; elapsed, 67.716s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 99.970s; elapsed, 67.719s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 95.900s; elapsed, 67.721s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 256.220s; elapsed, 67.720s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 70.640s; elapsed, 67.732s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 235.640s; elapsed, 67.716s, #calls: 1', 'TOTAL : CPU, 235.640s; elapsed, 67.716s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 118.050s; elapsed, 67.735s, #calls: 1', 'TOTAL : CPU, 118.050s; elapsed, 67.735s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 256.220s; elapsed, 67.720s, #calls: 1', 'TOTAL : CPU, 256.220s; elapsed, 67.720s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 156.850s; elapsed, 67.736s, #calls: 1', 'TOTAL : CPU, 156.850s; elapsed, 67.736s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 95.900s; elapsed, 67.721s, #calls: 1', 'TOTAL : CPU, 95.900s; elapsed, 67.721s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 99.970s; elapsed, 67.719s, #calls: 1', 'TOTAL : CPU, 99.970s; elapsed, 67.719s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 94.270s; elapsed, 67.722s, #calls: 1', 'TOTAL : CPU, 94.270s; elapsed, 67.722s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 258.760s; elapsed, 66.629s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.490s; elapsed, 0.220s, #calls: 1', 'time for EIGEN_LDLT: CPU, 308.160s; elapsed, 307.768s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 70.640s; elapsed, 67.732s, #calls: 1', 'TOTAL : CPU, 638.050s; elapsed, 442.348s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 2207.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 159.620s; elapsed, 159.742s', 'individual call time for EIGEN_LDLT: CPU, 310.950s; elapsed, 311.139s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.307s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,181', '# - number of levels = 127', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.29305', '# - matching time = 0.145027', '# - symmetrization time = 0.0552728', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,181', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.128587', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.228 MB', '# - factor time = 60.253', '# - factor nonzeros = 113,153,484', '# - factor memory = 905.228 MB', '# - total flops = 7.19774e+11, min = 41800, max = 5.51438e+11', '# - flop rate = 11.9458 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.27001e+11', '# --------------------------------------------', '# total = 7.27001e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.95624e-11\trel.res = 2.14044e-15\tbw.error = 6.17961e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.141461', '# - total flops = 3.64315e+08, min = 100280, max = 1.87793e+08', '# - flop rate = 2.57538 GFlop/s', '# - bytes moved = 412.812 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.91821 GByte/s', '# - solve arithmetic intensity = 0.882521 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 61.980s; elapsed, 62.101s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 61.990s; elapsed, 62.100s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 62.040s; elapsed, 62.097s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 61.910s; elapsed, 62.107s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 62.050s; elapsed, 62.105s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 62.010s; elapsed, 62.109s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 62.000s; elapsed, 62.107s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 61.970s; elapsed, 62.101s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 62.010s; elapsed, 62.101s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 62.060s; elapsed, 62.101s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 62.040s; elapsed, 62.096s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 62.050s; elapsed, 62.101s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 62.020s; elapsed, 62.097s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 62.010s; elapsed, 62.100s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 61.950s; elapsed, 62.097s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 62.080s; elapsed, 62.108s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 62.010s; elapsed, 62.109s, #calls: 1', 'TOTAL : CPU, 62.010s; elapsed, 62.109s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 61.970s; elapsed, 62.101s, #calls: 1', 'TOTAL : CPU, 61.970s; elapsed, 62.101s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 62.010s; elapsed, 62.101s, #calls: 1', 'TOTAL : CPU, 62.010s; elapsed, 62.101s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 62.040s; elapsed, 62.097s, #calls: 1', 'TOTAL : CPU, 62.040s; elapsed, 62.097s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 62.020s; elapsed, 62.097s, #calls: 1', 'TOTAL : CPU, 62.020s; elapsed, 62.097s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 61.980s; elapsed, 62.101s, #calls: 1', 'TOTAL : CPU, 61.980s; elapsed, 62.101s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 61.950s; elapsed, 62.097s, #calls: 1', 'TOTAL : CPU, 61.950s; elapsed, 62.097s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 62.040s; elapsed, 62.096s, #calls: 1', 'TOTAL : CPU, 62.040s; elapsed, 62.096s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 61.910s; elapsed, 62.107s, #calls: 1', 'TOTAL : CPU, 61.910s; elapsed, 62.107s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 62.050s; elapsed, 62.101s, #calls: 1', 'TOTAL : CPU, 62.050s; elapsed, 62.101s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 62.010s; elapsed, 62.100s, #calls: 1', 'TOTAL : CPU, 62.010s; elapsed, 62.100s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 62.060s; elapsed, 62.101s, #calls: 1', 'TOTAL : CPU, 62.060s; elapsed, 62.101s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 62.050s; elapsed, 62.105s, #calls: 1', 'TOTAL : CPU, 62.050s; elapsed, 62.105s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 61.990s; elapsed, 62.100s, #calls: 1', 'TOTAL : CPU, 61.990s; elapsed, 62.100s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 62.000s; elapsed, 62.107s, #calls: 1', 'TOTAL : CPU, 62.000s; elapsed, 62.107s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 159.620s; elapsed, 159.742s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.307s, #calls: 1', 'time for EIGEN_LDLT: CPU, 310.950s; elapsed, 311.139s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 62.080s; elapsed, 62.108s, #calls: 1', 'TOTAL : CPU, 532.950s; elapsed, 533.296s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 2207.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 175.680s; elapsed, 88.581s', 'individual call time for EIGEN_LDLT: CPU, 365.500s; elapsed, 365.574s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.346s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 1,211,274', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,181', '# - number of levels = 127', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 1.27181', '# - matching time = 0.166307', '# - symmetrization time = 0.0745802', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,181', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.138116', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 905.228 MB', '# - factor time = 86.6161', '# - factor nonzeros = 113,153,484', '# - factor memory = 905.228 MB', '# - total flops = 7.19774e+11, min = 41800, max = 5.51438e+11', '# - flop rate = 8.30993 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.27001e+11', '# --------------------------------------------', '# total = 7.27001e+11', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41843\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 8.94628e-11\trel.res = 2.13806e-15\tbw.error = 6.1844e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.135193', '# - total flops = 3.64315e+08, min = 100280, max = 1.87793e+08', '# - flop rate = 2.69478 GFlop/s', '# - bytes moved = 412.822 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.05358 GByte/s', '# - solve arithmetic intensity = 0.882499 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 97.020s; elapsed, 88.505s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 103.890s; elapsed, 88.506s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 103.010s; elapsed, 88.504s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 96.290s; elapsed, 88.497s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 115.400s; elapsed, 88.506s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 96.010s; elapsed, 88.502s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 170.210s; elapsed, 88.500s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 94.980s; elapsed, 88.500s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 94.550s; elapsed, 88.496s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 97.080s; elapsed, 88.496s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 96.670s; elapsed, 88.501s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 95.880s; elapsed, 88.499s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 94.990s; elapsed, 88.502s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 136.260s; elapsed, 88.494s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 96.680s; elapsed, 88.495s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 89.420s; elapsed, 88.511s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 94.550s; elapsed, 88.496s, #calls: 1', 'TOTAL : CPU, 94.550s; elapsed, 88.496s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 96.290s; elapsed, 88.497s, #calls: 1', 'TOTAL : CPU, 96.290s; elapsed, 88.497s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 97.080s; elapsed, 88.496s, #calls: 1', 'TOTAL : CPU, 97.080s; elapsed, 88.496s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 96.010s; elapsed, 88.502s, #calls: 1', 'TOTAL : CPU, 96.010s; elapsed, 88.502s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 95.880s; elapsed, 88.499s, #calls: 1', 'TOTAL : CPU, 95.880s; elapsed, 88.499s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 170.210s; elapsed, 88.500s, #calls: 1', 'TOTAL : CPU, 170.210s; elapsed, 88.500s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 94.990s; elapsed, 88.502s, #calls: 1', 'TOTAL : CPU, 94.990s; elapsed, 88.502s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 97.020s; elapsed, 88.505s, #calls: 1', 'TOTAL : CPU, 97.020s; elapsed, 88.505s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 103.010s; elapsed, 88.504s, #calls: 1', 'TOTAL : CPU, 103.010s; elapsed, 88.504s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 96.680s; elapsed, 88.495s, #calls: 1', 'TOTAL : CPU, 96.680s; elapsed, 88.495s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 103.890s; elapsed, 88.506s, #calls: 1', 'TOTAL : CPU, 103.890s; elapsed, 88.506s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 115.400s; elapsed, 88.506s, #calls: 1', 'TOTAL : CPU, 115.400s; elapsed, 88.506s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 136.260s; elapsed, 88.494s, #calls: 1', 'TOTAL : CPU, 136.260s; elapsed, 88.494s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 94.980s; elapsed, 88.500s, #calls: 1', 'TOTAL : CPU, 94.980s; elapsed, 88.500s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 96.670s; elapsed, 88.501s, #calls: 1', 'TOTAL : CPU, 96.670s; elapsed, 88.501s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 175.680s; elapsed, 88.581s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.346s, #calls: 1', 'time for EIGEN_LDLT: CPU, 365.500s; elapsed, 365.574s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 89.420s; elapsed, 88.511s, #calls: 1', 'TOTAL : CPU, 631.100s; elapsed, 543.012s']
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-/A_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-/b_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 31.350s; elapsed, 31.372s', 'individual call time for EIGEN_LDLT: CPU, 37.620s; elapsed, 37.649s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.120s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 31.350s; elapsed, 31.372s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.120s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.620s; elapsed, 37.649s, #calls: 1', 'TOTAL : CPU, 69.100s; elapsed, 69.141s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 40.630s; elapsed, 20.578s', 'individual call time for EIGEN_LDLT: CPU, 37.840s; elapsed, 37.683s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.112s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 40.630s; elapsed, 20.578s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.112s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.840s; elapsed, 37.683s, #calls: 1', 'TOTAL : CPU, 78.640s; elapsed, 58.372s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 67.890s; elapsed, 17.197s', 'individual call time for EIGEN_LDLT: CPU, 40.040s; elapsed, 39.527s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.102s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 67.890s; elapsed, 17.197s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.102s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.040s; elapsed, 39.527s, #calls: 1', 'TOTAL : CPU, 108.140s; elapsed, 56.825s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 119.830s; elapsed, 15.314s', 'individual call time for EIGEN_LDLT: CPU, 38.900s; elapsed, 37.665s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.119s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 119.830s; elapsed, 15.314s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.119s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.900s; elapsed, 37.665s, #calls: 1', 'TOTAL : CPU, 159.110s; elapsed, 53.098s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 35324.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 293.910s; elapsed, 18.708s', 'individual call time for EIGEN_LDLT: CPU, 40.390s; elapsed, 37.670s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.089s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 293.910s; elapsed, 18.708s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.089s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.390s; elapsed, 37.670s, #calls: 1', 'TOTAL : CPU, 334.770s; elapsed, 56.467s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.770s; elapsed, 32.792s', 'individual call time for EIGEN_LDLT: CPU, 38.960s; elapsed, 38.994s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.124s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,209', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.4046', '# - matching time = 0.0722611', '# - symmetrization time = 0.0358469', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,209', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.085063', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.606 MB', '# - factor time = 31.6957', '# - factor nonzeros = 31,325,750', '# - factor memory = 250.606 MB', '# - total flops = 9.48564e+10, min = 222880, max = 9.48562e+10', '# - flop rate = 2.99272 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73399e+10', '# --------------------------------------------', '# total = 9.73399e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.11914e-11\trel.res = 2.20224e-15\tbw.error = 4.90384e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.120098', '# - total flops = 6.78544e+07, min = 289862, max = 6.75645e+07', '# - flop rate = 0.564992 GFlop/s', '# - bytes moved = 410.995 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.42216 GByte/s', '# - solve arithmetic intensity = 0.165098 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 32.490s; elapsed, 32.511s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 32.510s; elapsed, 32.505s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 32.490s; elapsed, 32.511s, #calls: 1', 'TOTAL : CPU, 32.490s; elapsed, 32.511s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.770s; elapsed, 32.792s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.124s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.960s; elapsed, 38.994s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 32.510s; elapsed, 32.505s, #calls: 1', 'TOTAL : CPU, 104.360s; elapsed, 104.415s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 36.770s; elapsed, 18.630s', 'individual call time for EIGEN_LDLT: CPU, 39.070s; elapsed, 38.922s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.121s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,209', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.380367', '# - matching time = 0.0759459', '# - symmetrization time = 0.0353429', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,209', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0845001', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.606 MB', '# - factor time = 19.3656', '# - factor nonzeros = 31,325,750', '# - factor memory = 250.606 MB', '# - total flops = 9.48567e+10, min = 222880, max = 9.48565e+10', '# - flop rate = 4.8982 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73399e+10', '# --------------------------------------------', '# total = 9.73399e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.04299e-11\trel.res = 2.18385e-15\tbw.error = 4.87162e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0978129', '# - total flops = 6.78737e+07, min = 289862, max = 6.75839e+07', '# - flop rate = 0.693914 GFlop/s', '# - bytes moved = 411.788 MB, min = 0 MB, max = 0 MB', '# - byte rate = 4.20996 GByte/s', '# - solve arithmetic intensity = 0.164827 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 39.960s; elapsed, 20.144s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.860s; elapsed, 20.133s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 39.960s; elapsed, 20.144s, #calls: 1', 'TOTAL : CPU, 39.960s; elapsed, 20.144s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 36.770s; elapsed, 18.630s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.121s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.070s; elapsed, 38.922s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.860s; elapsed, 20.133s, #calls: 1', 'TOTAL : CPU, 96.880s; elapsed, 77.805s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 59.180s; elapsed, 15.084s', 'individual call time for EIGEN_LDLT: CPU, 69.840s; elapsed, 69.335s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.200s; elapsed, 0.105s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,209', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.377711', '# - matching time = 0.0720251', '# - symmetrization time = 0.035423', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,209', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.080492', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.606 MB', '# - factor time = 16.0621', '# - factor nonzeros = 31,325,750', '# - factor memory = 250.606 MB', '# - total flops = 9.48567e+10, min = 222880, max = 9.48565e+10', '# - flop rate = 5.90561 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73399e+10', '# --------------------------------------------', '# total = 9.73399e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.04185e-11\trel.res = 2.18357e-15\tbw.error = 4.87162e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.111008', '# - total flops = 6.78737e+07, min = 289862, max = 6.75839e+07', '# - flop rate = 0.61143 GFlop/s', '# - bytes moved = 411.808 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.70971 GByte/s', '# - solve arithmetic intensity = 0.164819 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 66.460s; elapsed, 16.843s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.070s; elapsed, 16.838s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 66.460s; elapsed, 16.843s, #calls: 1', 'TOTAL : CPU, 66.460s; elapsed, 16.843s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 59.180s; elapsed, 15.084s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.200s; elapsed, 0.105s, #calls: 1', 'time for EIGEN_LDLT: CPU, 69.840s; elapsed, 69.335s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.070s; elapsed, 16.838s, #calls: 1', 'TOTAL : CPU, 148.290s; elapsed, 101.362s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 116.070s; elapsed, 14.850s', 'individual call time for EIGEN_LDLT: CPU, 40.230s; elapsed, 39.012s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.088s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,209', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.408596', '# - matching time = 0.0735049', '# - symmetrization time = 0.0369918', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,209', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0991609', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.606 MB', '# - factor time = 15.4322', '# - factor nonzeros = 31,325,750', '# - factor memory = 250.606 MB', '# - total flops = 9.48567e+10, min = 224830, max = 9.48565e+10', '# - flop rate = 6.14666 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73399e+10', '# --------------------------------------------', '# total = 9.73399e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.04226e-11\trel.res = 2.18367e-15\tbw.error = 4.86687e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.161948', '# - total flops = 6.78737e+07, min = 289862, max = 6.75839e+07', '# - flop rate = 0.419108 GFlop/s', '# - bytes moved = 411.848 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.54309 GByte/s', '# - solve arithmetic intensity = 0.164803 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 128.390s; elapsed, 16.319s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.000s; elapsed, 16.312s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 128.390s; elapsed, 16.319s, #calls: 1', 'TOTAL : CPU, 128.390s; elapsed, 16.319s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 116.070s; elapsed, 14.850s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.088s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.230s; elapsed, 39.012s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.000s; elapsed, 16.312s, #calls: 1', 'TOTAL : CPU, 178.570s; elapsed, 70.262s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 17662.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 253.890s; elapsed, 16.229s', 'individual call time for EIGEN_LDLT: CPU, 40.450s; elapsed, 37.825s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.095s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,209', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.423476', '# - matching time = 0.109122', '# - symmetrization time = 0.0572441', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,209', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.104183', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.606 MB', '# - factor time = 14.5253', '# - factor nonzeros = 31,325,750', '# - factor memory = 250.606 MB', '# - total flops = 9.48577e+10, min = 226855, max = 9.48575e+10', '# - flop rate = 6.53052 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73399e+10', '# --------------------------------------------', '# total = 9.73399e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.04391e-11\trel.res = 2.18407e-15\tbw.error = 4.87578e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.126633', '# - total flops = 6.78784e+07, min = 289862, max = 6.75885e+07', '# - flop rate = 0.536025 GFlop/s', '# - bytes moved = 412.003 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.25352 GByte/s', '# - solve arithmetic intensity = 0.164752 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 241.990s; elapsed, 15.480s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 28.120s; elapsed, 15.471s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 241.990s; elapsed, 15.480s, #calls: 1', 'TOTAL : CPU, 241.990s; elapsed, 15.480s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 253.890s; elapsed, 16.229s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.095s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.450s; elapsed, 37.825s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 28.120s; elapsed, 15.471s, #calls: 1', 'TOTAL : CPU, 322.980s; elapsed, 69.620s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 34.780s; elapsed, 34.813s', 'individual call time for EIGEN_LDLT: CPU, 39.810s; elapsed, 39.832s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.136s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,221', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.54868', '# - matching time = 0.0796881', '# - symmetrization time = 0.0392759', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,221', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0620191', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 249.884 MB', '# - factor time = 15.6672', '# - factor nonzeros = 31,235,540', '# - factor memory = 249.884 MB', '# - total flops = 9.44698e+10, min = 223014, max = 7.03507e+10', '# - flop rate = 6.02978 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.68638e+10', '# --------------------------------------------', '# total = 9.68638e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.00117e-10\trel.res = 2.4178e-15\tbw.error = 5.7017e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.093749', '# - total flops = 7.15328e+07, min = 245763, max = 5.0953e+07', '# - flop rate = 0.763024 GFlop/s', '# - bytes moved = 219.539 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.34177 GByte/s', '# - solve arithmetic intensity = 0.325832 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.530s; elapsed, 16.547s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.510s; elapsed, 16.534s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 16.500s; elapsed, 16.531s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.540s; elapsed, 16.543s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.530s; elapsed, 16.547s, #calls: 1', 'TOTAL : CPU, 16.530s; elapsed, 16.547s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 16.500s; elapsed, 16.531s, #calls: 1', 'TOTAL : CPU, 16.500s; elapsed, 16.531s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.510s; elapsed, 16.534s, #calls: 1', 'TOTAL : CPU, 16.510s; elapsed, 16.534s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 34.780s; elapsed, 34.813s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.136s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.810s; elapsed, 39.832s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.540s; elapsed, 16.543s, #calls: 1', 'TOTAL : CPU, 91.270s; elapsed, 91.324s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 41.140s; elapsed, 20.889s', 'individual call time for EIGEN_LDLT: CPU, 37.950s; elapsed, 37.798s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.116s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,221', '# - number of levels = 131', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.459112', '# - matching time = 0.0695028', '# - symmetrization time = 0.0358191', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,221', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.064703', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 249.884 MB', '# - factor time = 14.3552', '# - factor nonzeros = 31,235,540', '# - factor memory = 249.884 MB', '# - total flops = 9.44698e+10, min = 223014, max = 7.03507e+10', '# - flop rate = 6.58089 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.68638e+10', '# --------------------------------------------', '# total = 9.68638e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.99927e-11\trel.res = 2.41479e-15\tbw.error = 5.7017e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0859909', '# - total flops = 7.15328e+07, min = 245763, max = 5.0953e+07', '# - flop rate = 0.831865 GFlop/s', '# - bytes moved = 219.551 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.55319 GByte/s', '# - solve arithmetic intensity = 0.325814 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 23.320s; elapsed, 15.134s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 25.830s; elapsed, 15.124s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 29.510s; elapsed, 15.118s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 15.790s; elapsed, 15.131s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 23.320s; elapsed, 15.134s, #calls: 1', 'TOTAL : CPU, 23.320s; elapsed, 15.134s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 25.830s; elapsed, 15.124s, #calls: 1', 'TOTAL : CPU, 25.830s; elapsed, 15.124s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 29.510s; elapsed, 15.118s, #calls: 1', 'TOTAL : CPU, 29.510s; elapsed, 15.118s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 41.140s; elapsed, 20.889s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.116s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.950s; elapsed, 37.798s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 15.790s; elapsed, 15.131s, #calls: 1', 'TOTAL : CPU, 95.050s; elapsed, 73.933s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 68.560s; elapsed, 17.442s', 'individual call time for EIGEN_LDLT: CPU, 40.820s; elapsed, 40.320s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.116s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,221', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.505306', '# - matching time = 0.0820031', '# - symmetrization time = 0.0443912', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,221', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.067661', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 249.884 MB', '# - factor time = 15.0046', '# - factor nonzeros = 31,235,540', '# - factor memory = 249.884 MB', '# - total flops = 9.44698e+10, min = 223014, max = 7.03507e+10', '# - flop rate = 6.29607 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.68638e+10', '# --------------------------------------------', '# total = 9.68638e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.99807e-11\trel.res = 2.4145e-15\tbw.error = 5.7017e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0940771', '# - total flops = 7.15328e+07, min = 245763, max = 5.0953e+07', '# - flop rate = 0.760363 GFlop/s', '# - bytes moved = 219.566 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.33389 GByte/s', '# - solve arithmetic intensity = 0.325792 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 45.130s; elapsed, 15.858s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 33.900s; elapsed, 15.869s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 61.460s; elapsed, 15.852s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.920s; elapsed, 15.864s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 61.460s; elapsed, 15.852s, #calls: 1', 'TOTAL : CPU, 61.460s; elapsed, 15.852s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 45.130s; elapsed, 15.858s, #calls: 1', 'TOTAL : CPU, 45.130s; elapsed, 15.858s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 33.900s; elapsed, 15.869s, #calls: 1', 'TOTAL : CPU, 33.900s; elapsed, 15.869s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 68.560s; elapsed, 17.442s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.116s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.820s; elapsed, 40.320s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.920s; elapsed, 15.864s, #calls: 1', 'TOTAL : CPU, 127.510s; elapsed, 73.742s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 8831.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 113.430s; elapsed, 14.489s', 'individual call time for EIGEN_LDLT: CPU, 40.390s; elapsed, 39.134s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.104s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,221', '# - number of levels = 131', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.503232', '# - matching time = 0.0831001', '# - symmetrization time = 0.0405271', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,221', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0808351', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 249.884 MB', '# - factor time = 15.9388', '# - factor nonzeros = 31,235,540', '# - factor memory = 249.884 MB', '# - total flops = 9.44698e+10, min = 224964, max = 7.03507e+10', '# - flop rate = 5.92703 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.68638e+10', '# --------------------------------------------', '# total = 9.68638e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.9961e-11\trel.res = 2.41402e-15\tbw.error = 5.7017e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0939651', '# - total flops = 7.15328e+07, min = 245763, max = 5.0953e+07', '# - flop rate = 0.76127 GFlop/s', '# - bytes moved = 219.596 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.33699 GByte/s', '# - solve arithmetic intensity = 0.325748 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 80.390s; elapsed, 16.805s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 60.690s; elapsed, 16.815s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 129.540s; elapsed, 16.799s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 21.830s; elapsed, 16.812s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 80.390s; elapsed, 16.805s, #calls: 1', 'TOTAL : CPU, 80.390s; elapsed, 16.805s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 129.540s; elapsed, 16.799s, #calls: 1', 'TOTAL : CPU, 129.540s; elapsed, 16.799s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 60.690s; elapsed, 16.815s, #calls: 1', 'TOTAL : CPU, 60.690s; elapsed, 16.815s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 113.430s; elapsed, 14.489s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.104s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.390s; elapsed, 39.134s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 21.830s; elapsed, 16.812s, #calls: 1', 'TOTAL : CPU, 175.940s; elapsed, 70.538s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 4415.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 33.240s; elapsed, 33.242s', 'individual call time for EIGEN_LDLT: CPU, 39.450s; elapsed, 39.459s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.127s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,201', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.499541', '# - matching time = 0.0728931', '# - symmetrization time = 0.026294', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,201', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0576689', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 251.27 MB', '# - factor time = 15.3904', '# - factor nonzeros = 31,408,706', '# - factor memory = 251.27 MB', '# - total flops = 9.51783e+10, min = 223014, max = 7.03714e+10', '# - flop rate = 6.18427 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73851e+10', '# --------------------------------------------', '# total = 9.73851e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.01808e-10\trel.res = 2.45864e-15\tbw.error = 4.93262e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.078059', '# - total flops = 8.13911e+07, min = 131786, max = 5.07409e+07', '# - flop rate = 1.04269 GFlop/s', '# - bytes moved = 212.161 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.71795 GByte/s', '# - solve arithmetic intensity = 0.38363 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 16.160s; elapsed, 16.180s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 16.140s; elapsed, 16.178s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 16.160s; elapsed, 16.179s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 16.130s; elapsed, 16.168s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 16.130s; elapsed, 16.173s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.160s; elapsed, 16.167s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.160s; elapsed, 16.173s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.180s; elapsed, 16.180s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 16.160s; elapsed, 16.180s, #calls: 1', 'TOTAL : CPU, 16.160s; elapsed, 16.180s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 16.160s; elapsed, 16.179s, #calls: 1', 'TOTAL : CPU, 16.160s; elapsed, 16.179s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 16.130s; elapsed, 16.173s, #calls: 1', 'TOTAL : CPU, 16.130s; elapsed, 16.173s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 16.130s; elapsed, 16.168s, #calls: 1', 'TOTAL : CPU, 16.130s; elapsed, 16.168s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.160s; elapsed, 16.167s, #calls: 1', 'TOTAL : CPU, 16.160s; elapsed, 16.167s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 16.140s; elapsed, 16.178s, #calls: 1', 'TOTAL : CPU, 16.140s; elapsed, 16.178s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.160s; elapsed, 16.173s, #calls: 1', 'TOTAL : CPU, 16.160s; elapsed, 16.173s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 33.240s; elapsed, 33.242s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.127s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.450s; elapsed, 39.459s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.180s; elapsed, 16.180s, #calls: 1', 'TOTAL : CPU, 89.000s; elapsed, 89.007s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 4415.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 36.510s; elapsed, 18.470s', 'individual call time for EIGEN_LDLT: CPU, 39.030s; elapsed, 38.883s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.107s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,201', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.486414', '# - matching time = 0.0734921', '# - symmetrization time = 0.0299671', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,201', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0651522', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 251.27 MB', '# - factor time = 13.9594', '# - factor nonzeros = 31,408,706', '# - factor memory = 251.27 MB', '# - total flops = 9.51783e+10, min = 223014, max = 7.03714e+10', '# - flop rate = 6.81821 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73851e+10', '# --------------------------------------------', '# total = 9.73851e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.01772e-10\trel.res = 2.45777e-15\tbw.error = 4.93262e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.077909', '# - total flops = 8.13911e+07, min = 131786, max = 5.07409e+07', '# - flop rate = 1.04469 GFlop/s', '# - bytes moved = 212.169 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.72329 GByte/s', '# - solve arithmetic intensity = 0.383615 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.130s; elapsed, 14.748s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 20.570s; elapsed, 14.749s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 28.550s; elapsed, 14.742s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 18.840s; elapsed, 14.749s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.680s; elapsed, 14.737s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 24.750s; elapsed, 14.737s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.060s; elapsed, 14.743s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 15.400s; elapsed, 14.750s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 28.550s; elapsed, 14.742s, #calls: 1', 'TOTAL : CPU, 28.550s; elapsed, 14.742s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.130s; elapsed, 14.748s, #calls: 1', 'TOTAL : CPU, 17.130s; elapsed, 14.748s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 20.570s; elapsed, 14.749s, #calls: 1', 'TOTAL : CPU, 20.570s; elapsed, 14.749s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.680s; elapsed, 14.737s, #calls: 1', 'TOTAL : CPU, 16.680s; elapsed, 14.737s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 18.840s; elapsed, 14.749s, #calls: 1', 'TOTAL : CPU, 18.840s; elapsed, 14.749s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 24.750s; elapsed, 14.737s, #calls: 1', 'TOTAL : CPU, 24.750s; elapsed, 14.737s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.060s; elapsed, 14.743s, #calls: 1', 'TOTAL : CPU, 17.060s; elapsed, 14.743s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 36.510s; elapsed, 18.470s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.107s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.030s; elapsed, 38.883s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 15.400s; elapsed, 14.750s, #calls: 1', 'TOTAL : CPU, 91.090s; elapsed, 72.211s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 4415.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 63.810s; elapsed, 16.253s', 'individual call time for EIGEN_LDLT: CPU, 40.720s; elapsed, 40.197s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.110s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,201', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.56894', '# - matching time = 0.0841079', '# - symmetrization time = 0.0454419', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,201', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.072114', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 251.27 MB', '# - factor time = 15.1333', '# - factor nonzeros = 31,408,706', '# - factor memory = 251.27 MB', '# - total flops = 9.51783e+10, min = 223014, max = 7.03714e+10', '# - flop rate = 6.28933 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.73851e+10', '# --------------------------------------------', '# total = 9.73851e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.01768e-10\trel.res = 2.45766e-15\tbw.error = 4.92775e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0940831', '# - total flops = 8.13911e+07, min = 131786, max = 5.07409e+07', '# - flop rate = 0.865098 GFlop/s', '# - bytes moved = 212.179 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.25523 GByte/s', '# - solve arithmetic intensity = 0.383597 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 23.620s; elapsed, 16.061s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 35.660s; elapsed, 16.062s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 27.900s; elapsed, 16.060s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 43.880s; elapsed, 16.048s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 61.290s; elapsed, 16.054s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 22.700s; elapsed, 16.057s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 21.890s; elapsed, 16.047s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.150s; elapsed, 16.061s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 22.700s; elapsed, 16.057s, #calls: 1', 'TOTAL : CPU, 22.700s; elapsed, 16.057s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 23.620s; elapsed, 16.061s, #calls: 1', 'TOTAL : CPU, 23.620s; elapsed, 16.061s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 43.880s; elapsed, 16.048s, #calls: 1', 'TOTAL : CPU, 43.880s; elapsed, 16.048s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 61.290s; elapsed, 16.054s, #calls: 1', 'TOTAL : CPU, 61.290s; elapsed, 16.054s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 35.660s; elapsed, 16.062s, #calls: 1', 'TOTAL : CPU, 35.660s; elapsed, 16.062s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 21.890s; elapsed, 16.047s, #calls: 1', 'TOTAL : CPU, 21.890s; elapsed, 16.047s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 27.900s; elapsed, 16.060s, #calls: 1', 'TOTAL : CPU, 27.900s; elapsed, 16.060s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 63.810s; elapsed, 16.253s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.110s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.720s; elapsed, 40.197s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.150s; elapsed, 16.061s, #calls: 1', 'TOTAL : CPU, 122.900s; elapsed, 72.621s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 2207.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 37.820s; elapsed, 37.893s', 'individual call time for EIGEN_LDLT: CPU, 38.850s; elapsed, 38.878s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.126s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,191', '# - number of levels = 129', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.503834', '# - matching time = 0.0723329', '# - symmetrization time = 0.024868', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,191', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.069247', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.38 MB', '# - factor time = 16.3407', '# - factor nonzeros = 31,297,498', '# - factor memory = 250.38 MB', '# - total flops = 9.44704e+10, min = 223014, max = 7.0108e+10', '# - flop rate = 5.78128 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.62837e+10', '# --------------------------------------------', '# total = 9.62837e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 6.88967e-11\trel.res = 1.66383e-15\tbw.error = 3.71324e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.079927', '# - total flops = 1.00397e+08, min = 114336, max = 5.02255e+07', '# - flop rate = 1.25612 GFlop/s', '# - bytes moved = 204.061 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.55309 GByte/s', '# - solve arithmetic intensity = 0.491997 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 17.100s; elapsed, 17.139s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 17.110s; elapsed, 17.141s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.090s; elapsed, 17.141s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 17.080s; elapsed, 17.141s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 17.120s; elapsed, 17.141s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.120s; elapsed, 17.138s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.100s; elapsed, 17.138s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 17.110s; elapsed, 17.132s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 17.080s; elapsed, 17.141s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 17.090s; elapsed, 17.140s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.120s; elapsed, 17.137s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.080s; elapsed, 17.137s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.110s; elapsed, 17.138s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.070s; elapsed, 17.140s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.070s; elapsed, 17.131s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.150s; elapsed, 17.145s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.070s; elapsed, 17.140s, #calls: 1', 'TOTAL : CPU, 17.070s; elapsed, 17.140s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 17.080s; elapsed, 17.141s, #calls: 1', 'TOTAL : CPU, 17.080s; elapsed, 17.141s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 17.120s; elapsed, 17.141s, #calls: 1', 'TOTAL : CPU, 17.120s; elapsed, 17.141s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.120s; elapsed, 17.138s, #calls: 1', 'TOTAL : CPU, 17.120s; elapsed, 17.138s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.110s; elapsed, 17.138s, #calls: 1', 'TOTAL : CPU, 17.110s; elapsed, 17.138s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 17.110s; elapsed, 17.132s, #calls: 1', 'TOTAL : CPU, 17.110s; elapsed, 17.132s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 17.080s; elapsed, 17.141s, #calls: 1', 'TOTAL : CPU, 17.080s; elapsed, 17.141s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.100s; elapsed, 17.138s, #calls: 1', 'TOTAL : CPU, 17.100s; elapsed, 17.138s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 17.090s; elapsed, 17.140s, #calls: 1', 'TOTAL : CPU, 17.090s; elapsed, 17.140s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 17.100s; elapsed, 17.139s, #calls: 1', 'TOTAL : CPU, 17.100s; elapsed, 17.139s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.090s; elapsed, 17.141s, #calls: 1', 'TOTAL : CPU, 17.090s; elapsed, 17.141s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 17.110s; elapsed, 17.141s, #calls: 1', 'TOTAL : CPU, 17.110s; elapsed, 17.141s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.070s; elapsed, 17.131s, #calls: 1', 'TOTAL : CPU, 17.070s; elapsed, 17.131s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.080s; elapsed, 17.137s, #calls: 1', 'TOTAL : CPU, 17.080s; elapsed, 17.137s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.120s; elapsed, 17.137s, #calls: 1', 'TOTAL : CPU, 17.120s; elapsed, 17.137s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 37.820s; elapsed, 37.893s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.126s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.850s; elapsed, 38.878s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.150s; elapsed, 17.145s, #calls: 1', 'TOTAL : CPU, 93.950s; elapsed, 94.041s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '35324 2207.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 39.080s; elapsed, 19.805s', 'individual call time for EIGEN_LDLT: CPU, 40.030s; elapsed, 39.881s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.116s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 35,324', '# - number of nonzeros = 618,302', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,191', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.567631', '# - matching time = 0.0831571', '# - symmetrization time = 0.027292', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,191', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0821381', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.38 MB', '# - factor time = 14.1487', '# - factor nonzeros = 31,297,498', '# - factor memory = 250.38 MB', '# - total flops = 9.44704e+10, min = 223014, max = 7.0108e+10', '# - flop rate = 6.67699 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.62837e+10', '# --------------------------------------------', '# total = 9.62837e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 6.88907e-11\trel.res = 1.66368e-15\tbw.error = 3.71324e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.082845', '# - total flops = 1.00397e+08, min = 114336, max = 5.02255e+07', '# - flop rate = 1.21187 GFlop/s', '# - bytes moved = 204.065 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.46321 GByte/s', '# - solve arithmetic intensity = 0.491988 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 17.710s; elapsed, 15.051s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.120s; elapsed, 15.052s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 17.580s; elapsed, 15.052s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 19.200s; elapsed, 15.052s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 17.350s; elapsed, 15.052s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 19.170s; elapsed, 15.052s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.700s; elapsed, 15.049s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.420s; elapsed, 15.049s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 21.220s; elapsed, 15.053s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 29.260s; elapsed, 15.050s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.150s; elapsed, 15.049s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 16.770s; elapsed, 15.050s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 16.790s; elapsed, 15.050s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 16.640s; elapsed, 15.042s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 23.950s; elapsed, 15.044s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 15.740s; elapsed, 15.057s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 17.710s; elapsed, 15.051s, #calls: 1', 'TOTAL : CPU, 17.710s; elapsed, 15.051s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.700s; elapsed, 15.049s, #calls: 1', 'TOTAL : CPU, 17.700s; elapsed, 15.049s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 17.580s; elapsed, 15.052s, #calls: 1', 'TOTAL : CPU, 17.580s; elapsed, 15.052s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 17.120s; elapsed, 15.052s, #calls: 1', 'TOTAL : CPU, 17.120s; elapsed, 15.052s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.420s; elapsed, 15.049s, #calls: 1', 'TOTAL : CPU, 17.420s; elapsed, 15.049s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.150s; elapsed, 15.049s, #calls: 1', 'TOTAL : CPU, 17.150s; elapsed, 15.049s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 19.200s; elapsed, 15.052s, #calls: 1', 'TOTAL : CPU, 19.200s; elapsed, 15.052s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 23.950s; elapsed, 15.044s, #calls: 1', 'TOTAL : CPU, 23.950s; elapsed, 15.044s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 17.350s; elapsed, 15.052s, #calls: 1', 'TOTAL : CPU, 17.350s; elapsed, 15.052s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 29.260s; elapsed, 15.050s, #calls: 1', 'TOTAL : CPU, 29.260s; elapsed, 15.050s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 16.640s; elapsed, 15.042s, #calls: 1', 'TOTAL : CPU, 16.640s; elapsed, 15.042s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 16.790s; elapsed, 15.050s, #calls: 1', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 16.770s; elapsed, 15.050s, #calls: 1', 'TOTAL : CPU, 16.790s; elapsed, 15.050s', 'TOTAL : CPU, 16.770s; elapsed, 15.050s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 19.170s; elapsed, 15.052s, #calls: 1', 'TOTAL : CPU, 19.170s; elapsed, 15.052s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 21.220s; elapsed, 15.053s, #calls: 1', 'TOTAL : CPU, 21.220s; elapsed, 15.053s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 39.080s; elapsed, 19.805s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.116s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.030s; elapsed, 39.881s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 15.740s; elapsed, 15.057s, #calls: 1', 'TOTAL : CPU, 95.010s; elapsed, 74.859s']
Data Set Name:=strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-/A_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_10k/out_strumpack_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-/b_strum_10k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 31.060s; elapsed, 31.085s', 'individual call time for EIGEN_LDLT: CPU, 37.640s; elapsed, 37.659s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.115s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 31.060s; elapsed, 31.085s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.115s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.640s; elapsed, 37.659s, #calls: 1', 'TOTAL : CPU, 68.810s; elapsed, 68.860s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 34.780s; elapsed, 17.597s', 'individual call time for EIGEN_LDLT: CPU, 37.780s; elapsed, 37.596s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.121s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 34.780s; elapsed, 17.597s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.121s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.780s; elapsed, 37.596s, #calls: 1', 'TOTAL : CPU, 72.730s; elapsed, 55.314s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 57.660s; elapsed, 14.674s', 'individual call time for EIGEN_LDLT: CPU, 38.240s; elapsed, 37.709s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.106s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 57.660s; elapsed, 14.674s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.106s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.240s; elapsed, 37.709s, #calls: 1', 'TOTAL : CPU, 96.110s; elapsed, 52.489s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 121.690s; elapsed, 15.460s', 'individual call time for EIGEN_LDLT: CPU, 38.970s; elapsed, 37.714s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.093s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 121.690s; elapsed, 15.460s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.093s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.970s; elapsed, 37.714s, #calls: 1', 'TOTAL : CPU, 160.920s; elapsed, 53.267s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 30327.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 275.770s; elapsed, 17.569s', 'individual call time for EIGEN_LDLT: CPU, 40.330s; elapsed, 37.708s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.100s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 275.770s; elapsed, 17.569s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.100s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.330s; elapsed, 37.708s, #calls: 1', 'TOTAL : CPU, 316.600s; elapsed, 55.377s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.200s; elapsed, 32.237s', 'individual call time for EIGEN_LDLT: CPU, 37.670s; elapsed, 37.690s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.117s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 130', '# - nd time = 0.422451', '# - matching time = 0.0672159', '# - symmetrization time = 0.0524902', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0864451', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 31.6537', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35496e+10, min = 41770, max = 9.35496e+10', '# - flop rate = 2.95541 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.60414e+10', '# --------------------------------------------', '# total = 9.60414e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.73537e-11\trel.res = 1.86806e-15\tbw.error = 5.7415e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.142907', '# - total flops = 6.72147e+07, min = 165052, max = 6.70497e+07', '# - flop rate = 0.470338 GFlop/s', '# - bytes moved = 408.126 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.85588 GByte/s', '# - solve arithmetic intensity = 0.164691 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 32.480s; elapsed, 32.509s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 32.530s; elapsed, 32.521s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 32.480s; elapsed, 32.509s, #calls: 1', 'TOTAL : CPU, 32.480s; elapsed, 32.509s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.200s; elapsed, 32.237s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.117s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.670s; elapsed, 37.690s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 32.530s; elapsed, 32.521s, #calls: 1', 'TOTAL : CPU, 102.510s; elapsed, 102.565s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 34.840s; elapsed, 17.632s', 'individual call time for EIGEN_LDLT: CPU, 37.920s; elapsed, 37.773s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.111s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.416875', '# - matching time = 0.0671329', '# - symmetrization time = 0.0533619', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0789139', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 17.2193', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35498e+10, min = 41770, max = 9.35498e+10', '# - flop rate = 5.43285 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.60414e+10', '# --------------------------------------------', '# total = 9.60414e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.80099e-11\trel.res = 1.88391e-15\tbw.error = 5.74116e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.125427', '# - total flops = 6.72328e+07, min = 165052, max = 6.70678e+07', '# - flop rate = 0.536031 GFlop/s', '# - bytes moved = 408.883 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.25993 GByte/s', '# - solve arithmetic intensity = 0.164431 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 35.740s; elapsed, 18.049s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.820s; elapsed, 18.061s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 35.740s; elapsed, 18.049s, #calls: 1', 'TOTAL : CPU, 35.740s; elapsed, 18.049s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 34.840s; elapsed, 17.632s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.920s; elapsed, 37.773s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.820s; elapsed, 18.061s, #calls: 1', 'TOTAL : CPU, 91.750s; elapsed, 73.577s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 60.630s; elapsed, 15.418s', 'individual call time for EIGEN_LDLT: CPU, 39.180s; elapsed, 38.663s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.111s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.431975', '# - matching time = 0.068692', '# - symmetrization time = 0.052408', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0735681', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 14.4334', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35498e+10, min = 41770, max = 9.35498e+10', '# - flop rate = 6.48149 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.60414e+10', '# --------------------------------------------', '# total = 9.60414e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.79581e-11\trel.res = 1.88266e-15\tbw.error = 5.74116e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.148777', '# - total flops = 6.72328e+07, min = 165052, max = 6.70678e+07', '# - flop rate = 0.451903 GFlop/s', '# - bytes moved = 408.902 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.74842 GByte/s', '# - solve arithmetic intensity = 0.164423 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 60.020s; elapsed, 15.299s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.640s; elapsed, 15.308s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 60.020s; elapsed, 15.299s, #calls: 1', 'TOTAL : CPU, 60.020s; elapsed, 15.299s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 60.630s; elapsed, 15.418s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.180s; elapsed, 38.663s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.640s; elapsed, 15.308s, #calls: 1', 'TOTAL : CPU, 117.670s; elapsed, 69.499s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 139.320s; elapsed, 17.783s', 'individual call time for EIGEN_LDLT: CPU, 40.220s; elapsed, 39.019s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.089s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.465776', '# - matching time = 0.073633', '# - symmetrization time = 0.0551162', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0766551', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 13.9159', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35498e+10, min = 42535, max = 9.35498e+10', '# - flop rate = 6.72251 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.60414e+10', '# --------------------------------------------', '# total = 9.60414e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.79316e-11\trel.res = 1.88202e-15\tbw.error = 5.74116e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.155214', '# - total flops = 6.72328e+07, min = 165052, max = 6.70678e+07', '# - flop rate = 0.433163 GFlop/s', '# - bytes moved = 408.94 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.63469 GByte/s', '# - solve arithmetic intensity = 0.164408 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 116.070s; elapsed, 14.839s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.480s; elapsed, 14.851s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 116.070s; elapsed, 14.839s, #calls: 1', 'TOTAL : CPU, 116.070s; elapsed, 14.839s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 139.320s; elapsed, 17.783s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.089s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.220s; elapsed, 39.019s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.480s; elapsed, 14.851s, #calls: 1', 'TOTAL : CPU, 200.290s; elapsed, 71.742s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 15163.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 296.940s; elapsed, 18.914s', 'individual call time for EIGEN_LDLT: CPU, 67.250s; elapsed, 64.608s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.102s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.51771', '# - matching time = 0.0950909', '# - symmetrization time = 0.0673761', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.096982', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 16.609', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35508e+10, min = 43208, max = 9.35507e+10', '# - flop rate = 5.63255 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.60414e+10', '# --------------------------------------------', '# total = 9.60414e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.68634e-11\trel.res = 1.85622e-15\tbw.error = 5.74638e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.16012', '# - total flops = 6.72375e+07, min = 165052, max = 6.70724e+07', '# - flop rate = 0.41992 GFlop/s', '# - bytes moved = 409.091 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.55491 GByte/s', '# - solve arithmetic intensity = 0.164358 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 275.130s; elapsed, 17.656s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 30.410s; elapsed, 17.664s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 275.130s; elapsed, 17.656s, #calls: 1', 'TOTAL : CPU, 275.130s; elapsed, 17.656s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 296.940s; elapsed, 18.914s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.102s, #calls: 1', 'time for EIGEN_LDLT: CPU, 67.250s; elapsed, 64.608s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 30.410s; elapsed, 17.664s, #calls: 1', 'TOTAL : CPU, 395.070s; elapsed, 101.288s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 34.290s; elapsed, 34.319s', 'individual call time for EIGEN_LDLT: CPU, 39.840s; elapsed, 39.874s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.134s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.503283', '# - matching time = 0.077836', '# - symmetrization time = 0.0592399', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0661089', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 15.2315', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35496e+10, min = 41770, max = 7.04191e+10', '# - flop rate = 6.14186 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.59508e+10', '# --------------------------------------------', '# total = 9.59508e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.73296e-11\trel.res = 1.86748e-15\tbw.error = 5.73834e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.102796', '# - total flops = 7.09444e+07, min = 127142, max = 5.09433e+07', '# - flop rate = 0.690147 GFlop/s', '# - bytes moved = 217.757 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.11834 GByte/s', '# - solve arithmetic intensity = 0.325796 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.080s; elapsed, 16.105s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 16.080s; elapsed, 16.095s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.060s; elapsed, 16.081s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.090s; elapsed, 16.101s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.080s; elapsed, 16.105s, #calls: 1', 'TOTAL : CPU, 16.080s; elapsed, 16.105s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 16.080s; elapsed, 16.095s, #calls: 1', 'TOTAL : CPU, 16.080s; elapsed, 16.095s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.060s; elapsed, 16.081s, #calls: 1', 'TOTAL : CPU, 16.060s; elapsed, 16.081s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 34.290s; elapsed, 34.319s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.134s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.840s; elapsed, 39.874s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.090s; elapsed, 16.101s, #calls: 1', 'TOTAL : CPU, 90.350s; elapsed, 90.429s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 37.980s; elapsed, 19.221s', 'individual call time for EIGEN_LDLT: CPU, 40.370s; elapsed, 40.217s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.113s', 'CPP -2', '# Initializing STRUMPACKCPP -2', 'CPP -2', 'CPP -2', '', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.502221', '# - matching time = 0.079648', '# - symmetrization time = 0.0596969', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0704589', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 15.0184', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35496e+10, min = 41770, max = 7.04191e+10', '# - flop rate = 6.22899 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.59508e+10', '# --------------------------------------------', '# total = 9.59508e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.71926e-11\trel.res = 1.86417e-15\tbw.error = 5.74381e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.108056', '# - total flops = 7.09444e+07, min = 127142, max = 5.09433e+07', '# - flop rate = 0.656551 GFlop/s', '# - bytes moved = 217.769 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.01533 GByte/s', '# - solve arithmetic intensity = 0.325778 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 23.830s; elapsed, 15.910s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 26.820s; elapsed, 15.885s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 31.080s; elapsed, 15.901s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.600s; elapsed, 15.906s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 23.830s; elapsed, 15.910s, #calls: 1', 'TOTAL : CPU, 23.830s; elapsed, 15.910s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 26.820s; elapsed, 15.885s, #calls: 1', 'TOTAL : CPU, 26.820s; elapsed, 15.885s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 31.080s; elapsed, 15.901s, #calls: 1', 'TOTAL : CPU, 31.080s; elapsed, 15.901s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 37.980s; elapsed, 19.221s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.113s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.370s; elapsed, 40.217s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.600s; elapsed, 15.906s, #calls: 1', 'TOTAL : CPU, 95.110s; elapsed, 75.458s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 70.200s; elapsed, 17.826s', 'individual call time for EIGEN_LDLT: CPU, 75.090s; elapsed, 74.598s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.111s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.433724', '# - matching time = 0.071959', '# - symmetrization time = 0.0539112', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0734382', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 14.206', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35496e+10, min = 41770, max = 7.04191e+10', '# - flop rate = 6.5852 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.59508e+10', '# --------------------------------------------', '# total = 9.59508e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.7193e-11\trel.res = 1.86418e-15\tbw.error = 5.74381e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.127258', '# - total flops = 7.09444e+07, min = 127142, max = 5.09433e+07', '# - flop rate = 0.557484 GFlop/s', '# - bytes moved = 217.784 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.71136 GByte/s', '# - solve arithmetic intensity = 0.325755 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 36.170s; elapsed, 15.038s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 44.810s; elapsed, 15.016s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 58.350s; elapsed, 15.029s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.200s; elapsed, 15.034s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 44.810s; elapsed, 15.016s, #calls: 1', 'TOTAL : CPU, 44.810s; elapsed, 15.016s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 36.170s; elapsed, 15.038s, #calls: 1', 'TOTAL : CPU, 36.170s; elapsed, 15.038s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 58.350s; elapsed, 15.029s, #calls: 1', 'TOTAL : CPU, 58.350s; elapsed, 15.029s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 70.200s; elapsed, 17.826s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_LDLT: CPU, 75.090s; elapsed, 74.598s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.200s; elapsed, 15.034s, #calls: 1', 'TOTAL : CPU, 162.710s; elapsed, 107.569s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 7581.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 137.720s; elapsed, 17.551s', 'individual call time for EIGEN_LDLT: CPU, 40.400s; elapsed, 39.166s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.095s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,187', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.461751', '# - matching time = 0.083421', '# - symmetrization time = 0.0664868', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,187', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0851312', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 248.219 MB', '# - factor time = 15.6386', '# - factor nonzeros = 31,027,329', '# - factor memory = 248.219 MB', '# - total flops = 9.35496e+10, min = 42535, max = 7.04191e+10', '# - flop rate = 5.98197 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.59508e+10', '# --------------------------------------------', '# total = 9.59508e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 7.71813e-11\trel.res = 1.8639e-15\tbw.error = 5.74928e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.120055', '# - total flops = 7.09444e+07, min = 127142, max = 5.09433e+07', '# - flop rate = 0.590932 GFlop/s', '# - bytes moved = 217.815 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.81429 GByte/s', '# - solve arithmetic intensity = 0.325709 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 59.030s; elapsed, 16.532s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 90.580s; elapsed, 16.510s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 127.640s; elapsed, 16.524s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 21.730s; elapsed, 16.530s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 127.640s; elapsed, 16.524s, #calls: 1', 'TOTAL : CPU, 127.640s; elapsed, 16.524s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 90.580s; elapsed, 16.510s, #calls: 1', 'TOTAL : CPU, 90.580s; elapsed, 16.510s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 59.030s; elapsed, 16.532s, #calls: 1', 'TOTAL : CPU, 59.030s; elapsed, 16.532s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 137.720s; elapsed, 17.551s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.095s, #calls: 1', 'time for EIGEN_LDLT: CPU, 40.400s; elapsed, 39.166s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 21.730s; elapsed, 16.530s, #calls: 1', 'TOTAL : CPU, 200.120s; elapsed, 73.341s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 3790.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.810s; elapsed, 32.833s', 'individual call time for EIGEN_LDLT: CPU, 39.430s; elapsed, 39.429s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.125s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,171', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# - nd time = # ******************************************************************', '0.46956', '# - matching time = 0.0716159', '# - symmetrization time = 0.0388072', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.061811', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.518 MB', '# - factor time = 16.4571', '# - factor nonzeros = 31,314,729', '# - factor memory = 250.518 MB', '# - total flops = 9.5061e+10, min = 41770, max = 7.01469e+10', '# - flop rate = 5.77629 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.72559e+10', '# --------------------------------------------', '# total = 9.72559e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.01663e-10\trel.res = 2.45512e-15\tbw.error = 4.56971e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.100274', '# - total flops = 8.12073e+07, min = 108187, max = 5.0687e+07', '# - flop rate = 0.809855 GFlop/s', '# - bytes moved = 211.022 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.10446 GByte/s', '# - solve arithmetic intensity = 0.384828 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.220s; elapsed, 17.258s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.240s; elapsed, 17.257s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.230s; elapsed, 17.253s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.230s; elapsed, 17.254s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.210s; elapsed, 17.251s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.230s; elapsed, 17.252s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.230s; elapsed, 17.241s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.270s; elapsed, 17.260s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 17.230s; elapsed, 17.253s, #calls: 1', 'TOTAL : CPU, 17.230s; elapsed, 17.253s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.230s; elapsed, 17.254s, #calls: 1', 'TOTAL : CPU, 17.230s; elapsed, 17.254s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 17.220s; elapsed, 17.258s, #calls: 1', 'TOTAL : CPU, 17.220s; elapsed, 17.258s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.240s; elapsed, 17.257s, #calls: 1', 'TOTAL : CPU, 17.240s; elapsed, 17.257s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 17.210s; elapsed, 17.251s, #calls: 1', 'TOTAL : CPU, 17.210s; elapsed, 17.251s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 17.230s; elapsed, 17.241s, #calls: 1', 'TOTAL : CPU, 17.230s; elapsed, 17.241s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.230s; elapsed, 17.252s, #calls: 1', 'TOTAL : CPU, 17.230s; elapsed, 17.252s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.810s; elapsed, 32.833s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.125s, #calls: 1', 'time for EIGEN_LDLT: CPU, 39.430s; elapsed, 39.429s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.270s; elapsed, 17.260s, #calls: 1', 'TOTAL : CPU, 89.630s; elapsed, 89.646s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 3790.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 43.950s; elapsed, 22.275s', 'individual call time for EIGEN_LDLT: CPU, 38.280s; elapsed, 38.121s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.125s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 8 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,171', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.480459', '# - matching time = 0.081213', '# - symmetrization time = 0.0428212', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0897632', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.518 MB', '# - factor time = 15.2662', '# - factor nonzeros = 31,314,729', '# - factor memory = 250.518 MB', '# - total flops = 9.5061e+10, min = 41770, max = 7.01469e+10', '# - flop rate = 6.22687 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.72559e+10', '# --------------------------------------------', '# total = 9.72559e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.01627e-10\trel.res = 2.45425e-15\tbw.error = 4.56971e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.089186', '# - total flops = 8.12073e+07, min = 108187, max = 5.0687e+07', '# - flop rate = 0.910539 GFlop/s', '# - bytes moved = 211.031 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.36619 GByte/s', '# - solve arithmetic intensity = 0.384812 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 18.650s; elapsed, 16.114s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 20.300s; elapsed, 16.118s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 31.350s; elapsed, 16.113s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.310s; elapsed, 16.114s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 24.590s; elapsed, 16.102s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 22.850s; elapsed, 16.119s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 18.090s; elapsed, 16.114s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.830s; elapsed, 16.120s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 18.650s; elapsed, 16.114s, #calls: 1', 'TOTAL : CPU, 18.650s; elapsed, 16.114s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 31.350s; elapsed, 16.113s, #calls: 1', 'TOTAL : CPU, 31.350s; elapsed, 16.113s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.310s; elapsed, 16.114s, #calls: 1', 'TOTAL : CPU, 18.310s; elapsed, 16.114s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 20.300s; elapsed, 16.118s, #calls: 1', 'TOTAL : CPU, 20.300s; elapsed, 16.118s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 24.590s; elapsed, 16.102s, #calls: 1', 'TOTAL : CPU, 24.590s; elapsed, 16.102s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 18.090s; elapsed, 16.114s, #calls: 1', 'TOTAL : CPU, 18.090s; elapsed, 16.114s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 22.850s; elapsed, 16.119s, #calls: 1', 'TOTAL : CPU, 22.850s; elapsed, 16.119s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 43.950s; elapsed, 22.275s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.125s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.280s; elapsed, 38.121s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.830s; elapsed, 16.120s, #calls: 1', 'TOTAL : CPU, 99.230s; elapsed, 76.641s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 3790.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 72.880s; elapsed, 18.513s', 'individual call time for EIGEN_LDLT: CPU, 38.520s; elapsed, 38.003s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.100s', 'CPP -2', 'CPP -2', 'CPP -2CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,171', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.533917', '# - matching time = 0.0805659', '# - symmetrization time = 0.0470691', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0791359', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.518 MB', '# - factor time = 15.7734', '# - factor nonzeros = 31,314,729', '# - factor memory = 250.518 MB', '# - total flops = 9.5061e+10, min = 41770, max = 7.01469e+10', '# - flop rate = 6.02667 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.72559e+10', '# --------------------------------------------', '# total = 9.72559e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.0162e-10\trel.res = 2.45408e-15\tbw.error = 4.56971e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.118165', '# - total flops = 8.12073e+07, min = 108187, max = 5.0687e+07', '# - flop rate = 0.687236 GFlop/s', '# - bytes moved = 211.042 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.78599 GByte/s', '# - solve arithmetic intensity = 0.384792 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 24.350s; elapsed, 16.703s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 28.820s; elapsed, 16.710s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 35.590s; elapsed, 16.709s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 22.690s; elapsed, 16.705s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 23.520s; elapsed, 16.697sindividual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 47.520s; elapsed, 16.693s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 63.550s; elapsed, 16.705s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.930s; elapsed, 16.713s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 24.350s; elapsed, 16.703s, #calls: 1', 'TOTAL : CPU, 24.350s; elapsed, 16.703s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 22.690s; elapsed, 16.705s, #calls: 1', 'TOTAL : CPU, 22.690s; elapsed, 16.705s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 23.520s; elapsed, 16.697s, #calls: 1', 'TOTAL : CPU, 23.520s; elapsed, 16.697s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 47.520s; elapsed, 16.693s, #calls: 1', 'TOTAL : CPU, 47.520s; elapsed, 16.693s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 35.590s; elapsed, 16.709s, #calls: 1', 'TOTAL : CPU, 35.590s; elapsed, 16.709s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 28.820s; elapsed, 16.710s, #calls: 1', 'TOTAL : CPU, 28.820s; elapsed, 16.710s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 63.550s; elapsed, 16.705s, #calls: 1', 'TOTAL : CPU, 63.550s; elapsed, 16.705s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 72.880s; elapsed, 18.513s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.100s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.520s; elapsed, 38.003s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.930s; elapsed, 16.713s, #calls: 1', 'TOTAL : CPU, 130.510s; elapsed, 73.328s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '30327 1895.4375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 36.560s; elapsed, 36.612s', 'individual call time for EIGEN_LDLT: CPU, 38.770s; elapsed, 38.805s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.123s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 30,327', '# - number of nonzeros = 613,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,171', '# - number of levels = 127', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.464769', '# - matching time = 0.071492', '# - symmetrization time = 0.020879', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0638721', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 250.518 MB', '# - factor time = 13.6226', '# - factor nonzeros = 31,314,729', '# - factor memory = 250.518 MB', '# - total flops = 9.5061e+10, min = 41770, max = 7.00432e+10', '# - flop rate = 6.97817 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 9.68706e+10', '# --------------------------------------------', '# total = 9.68706e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 41408.5\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.56982e-11\trel.res = 2.31108e-15\tbw.error = 3.94337e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.075362', '# - total flops = 1.01113e+08, min = 98712, max = 5.0209e+07', '# - flop rate = 1.3417 GFlop/s', '# - bytes moved = 203.539 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.70082 GByte/s', '# - solve arithmetic intensity = 0.496775 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 14.360s; elapsed, 14.368s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 14.340s; elapsed, 14.366s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 14.300s; elapsed, 14.367s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 14.290s; elapsed, 14.366s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 14.340s; elapsed, 14.368s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 14.340s; elapsed, 14.367s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 14.310s; elapsed, 14.368s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 14.320s; elapsed, 14.368s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 14.340s; elapsed, 14.367s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 14.310s; elapsed, 14.366s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 14.290s; elapsed, 14.365s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 14.340s; elapsed, 14.358s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 14.320s; elapsed, 14.362s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 14.360s; elapsed, 14.366s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 14.320s; elapsed, 14.359s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 14.360s; elapsed, 14.372s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 14.320s; elapsed, 14.368s, #calls: 1', 'TOTAL : CPU, 14.320s; elapsed, 14.368s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 14.340s; elapsed, 14.367s, #calls: 1', 'TOTAL : CPU, 14.340s; elapsed, 14.367s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 14.340s; elapsed, 14.366s, #calls: 1', 'TOTAL : CPU, 14.340s; elapsed, 14.366s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 14.340s; elapsed, 14.368s, #calls: 1', 'TOTAL : CPU, 14.340s; elapsed, 14.368s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 14.320s; elapsed, 14.362s, #calls: 1', 'TOTAL : CPU, 14.320s; elapsed, 14.362s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 14.360s; elapsed, 14.366s, #calls: 1', 'TOTAL : CPU, 14.360s; elapsed, 14.366s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 14.310s; elapsed, 14.366s, #calls: 1', 'TOTAL : CPU, 14.310s; elapsed, 14.366s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 14.360s; elapsed, 14.368s, #calls: 1', 'TOTAL : CPU, 14.360s; elapsed, 14.368s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 14.300s; elapsed, 14.367s, #calls: 1', 'TOTAL : CPU, 14.300s; elapsed, 14.367s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 14.310s; elapsed, 14.368s, #calls: 1', 'TOTAL : CPU, 14.310s; elapsed, 14.368s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 14.320s; elapsed, 14.359s, #calls: 1', 'TOTAL : CPU, 14.320s; elapsed, 14.359s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 14.340s; elapsed, 14.358s, #calls: 1', 'TOTAL : CPU, 14.340s; elapsed, 14.358s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 14.290s; elapsed, 14.365s, #calls: 1', 'TOTAL : CPU, 14.290s; elapsed, 14.365s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 14.340s; elapsed, 14.367s, #calls: 1', 'TOTAL : CPU, 14.340s; elapsed, 14.367s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 14.290s; elapsed, 14.366s, #calls: 1', 'TOTAL : CPU, 14.290s; elapsed, 14.366s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 36.560s; elapsed, 36.612s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.123s, #calls: 1', 'time for EIGEN_LDLT: CPU, 38.770s; elapsed, 38.805s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 14.360s; elapsed, 14.372s, #calls: 1', 'TOTAL : CPU, 89.810s; elapsed, 89.912s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/ra