STRUMPACK vs EIGEN performance
The goal of this notebook is to allow the documentation of STRUMPACK vs EIGEN performance to be maintained in a single accessible location. The environment within which this notebook is run follows the standard cctbx conda build instructions available here. For this instance, we are using the STRUMPACK-enabled build of cctbx located at ExaFEL:cctbx_project(str_merge). STRUMPACK is currently built using the installation script STRUMPACK_installer_shared.sh, and if the installation takes place within the same directory as moddules and build, the cctbx build process can make use of it as a backend. After the STRUMPACK solver finishes, enter the build directory, run libtbx.refresh and make. The STRUMPACK-supported modules should now build and link with the new backend.
The solution of the below systems assume that we have some A and b data already, and we can provide these in CSV file format to the solver (A is in “row col value” format). We can begin by importing all of the required modules. We make use of Numpy’s ability to parse CSV files into numeric format, and SciPy’s sparse storage format to format the matrices into the required CSR sparse storage format for solving.
%matplotlib inline
%config InlineBackend.figure_format = 'retina'
import matplotlib
import matplotlib.pyplot as plt
font = {'family' : 'serif',
#'weight' : 'bold',
'size' : 12}
matplotlib.rc('font', **font)
matplotlib.rcParams['figure.figsize'] = (16,10)
matplotlib.rcParams['figure.dpi']= 150
import matplotlib.gridspec as gridspec
from __future__ import division
from cctbx.array_family import flex
from libtbx.test_utils import approx_equal
from libtbx.development.timers import Profiler
import sys
import numpy as np
import scipy.sparse as sps
%env BOOST_ADAPTBX_FPE_DEFAULT=1
%env BOOST_ADAPTBX_SIGNALS_DEFAULT=1
env: BOOST_ADAPTBX_FPE_DEFAULT=1
env: BOOST_ADAPTBX_SIGNALS_DEFAULT=1
We load the A and b data set for solving from the locations specified below. Ideally, we can loop over and load all different data sets and process each individually. For a working example, we begin by processing a single data set, comparing Eigen and Strumpack for varying levels of shared parallelism.
A_path="/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv"
A_mat = np.loadtxt(A_path,dtype={'names':('rows','cols','vals'),'formats':('i8','i8','f8')})
b_path="/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv"
b_vec = np.loadtxt(b_path)
We are solving system of normal equations, and so we can safely assume that we have a square matrix. This ensures the number of rows and colums will be identical, which can be read from the length of the numpy array b_vec.
n_rows = len(b_vec)
n_cols = n_rows
We next wish to create a SciPy CSR sparse matrix type using the data from A_mat.
A_sp = sps.csr_matrix((A_mat['vals'],(A_mat['rows'],A_mat['cols'])))
With this, we can now create the appropriate flex data types for solving the linear system. While the operations performed upto and including this point involve many copy steps, it can be stated that the time to solve the system is much greater than the time to perform internal copies, and so we can ignore the times required for these steps for any realistic datasets.
For the solvers we require the row index pointers, the column indices, and sparse matrix values. However, it is import to first ensure that our A matrix is symmetric, and not the upper or lower triangle only. While Eigen’s Cholesky deocmposition can use the upper or lower triangle, other algorithms and solvers tend to expect a full matrix. We can create this as follows.
tu=sps.triu(A_sp)
tl=sps.tril(A_sp)
sd=sps.diags(A_sp.diagonal())
if tu.nnz == sd.getnnz() or tl.nnz == sd.getnnz():
A_sp = A_sp + A_sp.transpose() - sd
plt.spy(A_sp, marker='.',markersize=1.0)
plt.show()
 |
The resulting data sets can be read directly and converted to flex types as follows.
A_indptr = flex.int(A_sp.indptr)
A_indices = flex.int(A_sp.indices)
A_values = flex.double(A_sp.data)
b = flex.double(b_vec)
With the data in the required format, we can now create a solver object and solve the system. We also include a built-in profiler object to time this solver step for both the STRUMPACK and EIGEN backends.
Next, we load the OpenMP-enabled STRUMPACK solver module and the Eigen based solver. These are contained within “modules/cctbx_project/scitbx/examples/bevington/strumpack_solver_ext.cpp” and where a Boost.Python wrapper for both an EIGEN and STRUMPACK Ax=b solver object is provided.
A non-notebook script exists to test this functionality at “modules/cctbx_project/scitbx/examples/bevington/strumpack_eigen_solver.py”, which can be called as:
OMP_NUM_THREADS=x libtbx.python strumpack_eigen_solver.py A_mat.csv b_vec.csv
where A_mat.csv and b_vec.csv are the CSV files with the linear system to be solved, and x specifies the number of OpenMP threads to spawn. Additionally, the MPI-enabled STRUMPACK backend is provided by “modules/cctbx_project/scitbx/examples/bevington/strumpack_solver_ext_mpi_dist.cpp”, with the Python script to test the result located at “modules/cctbx_project/scitbx/examples/bevington/strumpack_eigen_solver_mpi_dist.py”. This is called as
OMP_NUM_THREADS=x mpirun -n y libtbx.python strumpack_eigen_solver.py A_mat.csv b_vec.csv
As an example, the previously loaded can be provided to the solvers as follows:
import boost.python
ext_omp = boost.python.import_ext("scitbx_examples_strumpack_solver_ext")
ext_mpi = boost.python.import_ext("scitbx_examples_strumpack_mpi_dist_solver_ext")
es = ext_omp.eigen_solver
ss = ext_omp.strumpack_solver
ssmd = ext_mpi.strumpack_mpi_dist_solver
def run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b):
P = Profiler("STRUMPACK_SCOTCH")
res_strum_sc = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.scotch, ext_omp.auto)
del P
P = Profiler("STRUMPACK_METIS")
res_strum_me = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.metis, ext_omp.auto)
del P
P = Profiler("EIGEN_LDLT")
res_eig_ldlt = es(1, n_rows, n_cols, A_indptr, A_indices, A_values, b)
del P
P = Profiler("EIGEN_BICGSTAB")
res_eig_bicgstab = es(2, n_rows, n_cols, A_indptr, A_indices, A_values, b)
del P
return res_strum_sc, res_strum_me, res_eig_ldlt, res_eig_bicgstab
r_s_sc, r_s_me, r_e_ldlt, r_e_bicgstab = run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b)
for i in xrange(len(r_s_sc.x)):
assert( approx_equal(r_s_sc.x[i], r_s_me.x[i]) )
assert( approx_equal(r_s_sc.x[i], r_e_ldlt.x[i]) )
assert( approx_equal(r_s_sc.x[i], r_e_bicgstab.x[i]) )
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.
try: mod = __import__(name)
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.
try: mod = __import__(name)
individual call time for STRUMPACK_SCOTCH: CPU, 19.210s; elapsed, 0.424s
individual call time for STRUMPACK_METIS: CPU, 176.190s; elapsed, 2.805s
individual call time for EIGEN_LDLT: CPU, 3.940s; elapsed, 0.063s
individual call time for EIGEN_BICGSTAB: CPU, 1.400s; elapsed, 0.023s
While we can use the above method to process the data, we must respecify the OMP_NUM_THREADS variable to observe different levels of scalability. %env OMP_NUM_THREADS is sufficient to test this for the first run, however for an undetermined reason the notebook must be started to allow the value to be changed. There, we can run the above tasks from a !<command> cell, specifying the thread count here instead.
OMP_SOLVER='''
from __future__ import division
from cctbx.array_family import flex
from libtbx.test_utils import approx_equal
from libtbx.development.timers import Profiler
import sys
import numpy as np
import scipy.sparse as sps
import matplotlib
import matplotlib.pyplot as plt
import matplotlib
import matplotlib.pyplot as plt
matplotlib.rcParams['figure.figsize'] = (16,10)
A_path=sys.argv[1]
A_mat = np.loadtxt(A_path,dtype={'names':('rows','cols','vals'),'formats':('i8','i8','f8')})
b_path=sys.argv[2]
b_vec = np.loadtxt(b_path)
n_rows = len(b_vec)
n_cols = n_rows
A_sp = sps.csr_matrix((A_mat['vals'],(A_mat['rows'],A_mat['cols'])))
#Check for triangular matrix. If so, A_sp := A+A^T - diag(A)
tu=sps.triu(A_sp)
tl=sps.tril(A_sp)
sd=sps.diags(A_sp.diagonal())
A_spS = A_sp
if tu.nnz == sd.getnnz() or tl.nnz == sd.getnnz():
A_spS = A_sp + A_sp.transpose() - sd
A_indptr = flex.int(A_sp.indptr)
A_indices = flex.int(A_sp.indices)
A_values = flex.double(A_sp.data)
b = flex.double(b_vec)
#import time
#timing_dict = {"strum":0, "eigen":0}
import boost.python
ext_omp = boost.python.import_ext("scitbx_examples_strumpack_solver_ext")
ext_mpi = boost.python.import_ext("scitbx_examples_strumpack_mpi_dist_solver_ext")
es = ext_omp.eigen_solver
ss = ext_omp.strumpack_solver
def run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS):
P = Profiler("EIGEN_LLT_CHOL")
res_eig_llt_chol = es(0, n_rows, n_cols, A_indptr, A_indices, A_values, b)
del P
P = Profiler("EIGEN_LDLT_CHOL")
res_eig_ldlt_chol = es(1, n_rows, n_cols, A_indptr, A_indices, A_values, b)
del P
A_indptr = flex.int(A_spS.indptr)
A_indices = flex.int(A_spS.indices)
A_values = flex.double(A_spS.data)
P = Profiler("EIGEN_BICGSTAB")
res_eig_bicgstab = es(2, n_rows, n_cols, A_indptr, A_indices, A_values, b)
del P
P = Profiler("STRUMPACK_SCOTCH_AUTO")
res_strum_sc_a = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.scotch, ext_omp.auto)
del P
P = Profiler("STRUMPACK_METIS_AUTO")
res_strum_mt_a = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.metis, ext_omp.auto)
del P
P = Profiler("STRUMPACK_SCOTCH_BICGSTAB")
res_strum_sc_bi = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.scotch, ext_omp.bicgstab)
del P
P = Profiler("STRUMPACK_METIS_BICGSTAB")
res_strum_mt_bi = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.metis, ext_omp.bicgstab)
del P
P = Profiler("STRUMPACK_SCOTCH_PRECBICGSTAB")
res_strum_sc_pr = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.scotch, ext_omp.prec_bicgstab)
del P
P = Profiler("STRUMPACK_METIS_PRECBICGSTAB")
res_strum_mt_pr = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.metis, ext_omp.prec_bicgstab)
del P
for i in xrange(len(res_strum_sc.x)):
assert( approx_equal(res_eig_llt_chol.x[i], res_eig_ldlt_chol.x[i]) )
assert( approx_equal(res_strum_sc_a.x[i], res_eig_bicgstab.x[i]) )
assert( approx_equal(res_strum_sc_a.x[i], res_eig_ldlt_chol.x[i]) )
assert( approx_equal(res_strum_mt_a.x[i], res_strum_sc_a.x[i]) )
assert( approx_equal(res_strum_mt_a.x[i], res_strum_sc_bi.x[i]) )
assert( approx_equal(res_strum_mt_a.x[i], res_strum_mt_bi.x[i]) )
assert( approx_equal(res_strum_mt_a.x[i], res_strum_sc_pr.x[i]) )
assert( approx_equal(res_strum_mt_a.x[i], res_strum_mt_pr.x[i]) )
run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)
'''
OMP_SOLVER = OMP_SOLVER
OMP_SOLVER_FILE = open("OMP_SOLVER.py", "w")
OMP_SOLVER_FILE.write(OMP_SOLVER)
OMP_SOLVER_FILE.close()
DATAPATH="/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/"
A_LIST = !find {DATAPATH} -iname "A*.csv"
B_LIST = [ii.replace('/A_','/b_') for ii in A_LIST]
We now record the indices of the data with a specific number of images. We can use these indices to later submit jobs with a given number of frames, and hence resulting matrix size.
list_idx={}
for imgs in ['1k','5k','10k','32k']:
list_idx.update({imgs:[i for i, j in enumerate(A_LIST) if imgs in j]})
list_idx
{'10k': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14],
'1k': [15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29],
'32k': [30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44],
'5k': [45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59]}
With the file writtent to disk, we now create a loop in mixed Python/Bash to call the solver, then extract the standard output from the profiler to examine the OpenMP scalability of STRUMPACK. The profiler code wraps each solver, and does not take into account the load-times for the data.
str_out={}
import os
threads_list = [16]
for imgs_size in list_idx:
print "Data Set Size:=%s"%imgs_size
#Subselect the smallest data size for now
if imgs_size != "1k":
print "Skipping %s"%imgs_size
continue
for imgs_idx in list_idx[imgs_size]:
A_path = A_LIST[imgs_idx]; b_path = B_LIST[imgs_idx]
dat_name = A_path.split('/')[-1][2:-4]
print "Data Set Name:=%s"%(dat_name)
#Ensure the A and b data are matched correctly
assert(os.path.dirname(A_path) == os.path.dirname(b_path))
print {A_path}, {b_path}
for threads in threads_list:
print "OMP_NUM_THREADS:=%d"%threads
val = !OMP_NUM_THREADS={threads} libtbx.python ./OMP_SOLVER.py {A_path} {b_path}
print val
for s in val:
if "assert" in s:
raise Exception("Solutions not equal! Halting")
key = 'omp' + str(threads) + '_' + dat_name
str_out.update({key:val})
—CLICK FOR OUTPUT—
Data Set Size:=10k
Skipping 10k
Data Set Size:=1k
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.810s; elapsed, 0.815s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.870s; elapsed, 0.863s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.025s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 4.280s; elapsed, 0.271s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 31.080s; elapsed, 1.960s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.930s; elapsed, 0.121s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.880s; elapsed, 0.117s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.920s; elapsed, 0.182s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 33.550s; elapsed, 2.101s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 33.550s; elapsed, 2.101s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.810s; elapsed, 0.815s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.930s; elapsed, 0.121s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.025s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 4.280s; elapsed, 0.271s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.920s; elapsed, 0.182s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.880s; elapsed, 0.117s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.870s; elapsed, 0.863s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 31.080s; elapsed, 1.960s, #calls: 1', 'TOTAL : CPU, 77.570s; elapsed, 6.455s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.220s; elapsed, 0.232s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.238s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.570s; elapsed, 0.049s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 7.060s; elapsed, 0.447s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 28.970s; elapsed, 1.831s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.720s; elapsed, 0.169s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.830s; elapsed, 0.177s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.750s; elapsed, 0.422s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 31.770s; elapsed, 2.001s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 31.770s; elapsed, 2.001s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.220s; elapsed, 0.232s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.720s; elapsed, 0.169s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.570s; elapsed, 0.049s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 7.060s; elapsed, 0.447s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.750s; elapsed, 0.422s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.830s; elapsed, 0.177s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.238s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 28.970s; elapsed, 1.831s, #calls: 1', 'TOTAL : CPU, 81.130s; elapsed, 5.566s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Deff-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.240s; elapsed, 0.237s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.230s; elapsed, 0.232s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.024s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.400s; elapsed, 0.215s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 28.710s; elapsed, 1.812s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.750s; elapsed, 0.109s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.740s; elapsed, 0.109s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.070s; elapsed, 0.192s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.190s; elapsed, 1.893s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.190s; elapsed, 1.893s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.240s; elapsed, 0.237s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.750s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.024s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.400s; elapsed, 0.215s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.070s; elapsed, 0.192s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.740s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.230s; elapsed, 0.232s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 28.710s; elapsed, 1.812s, #calls: 1', 'TOTAL : CPU, 69.570s; elapsed, 4.823s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.070s; elapsed, 0.064s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.050s; elapsed, 0.052s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.025s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.350s; elapsed, 0.212s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 28.510s; elapsed, 1.799s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.670s; elapsed, 0.105s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.760s; elapsed, 0.110s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.310s; elapsed, 0.207s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 26.660s; elapsed, 1.673s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 26.660s; elapsed, 1.673s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.070s; elapsed, 0.064s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.670s; elapsed, 0.105s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.025s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.350s; elapsed, 0.212s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.310s; elapsed, 0.207s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.760s; elapsed, 0.110s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.050s; elapsed, 0.052s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 28.510s; elapsed, 1.799s, #calls: 1', 'TOTAL : CPU, 65.610s; elapsed, 4.247s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 1.990s; elapsed, 1.995s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 2.110s; elapsed, 2.120s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.048s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 6.590s; elapsed, 0.417s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 34.870s; elapsed, 2.191s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.440s; elapsed, 0.153s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.620s; elapsed, 0.163s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.920s; elapsed, 0.433s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 38.160s; elapsed, 2.391s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 38.160s; elapsed, 2.391s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 1.990s; elapsed, 1.995s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.440s; elapsed, 0.153s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.048s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 6.590s; elapsed, 0.417s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.920s; elapsed, 0.433s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.620s; elapsed, 0.163s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 2.110s; elapsed, 2.120s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 34.870s; elapsed, 2.191s, #calls: 1', 'TOTAL : CPU, 96.220s; elapsed, 9.909s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 2.040s; elapsed, 2.044s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 2.090s; elapsed, 2.091s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.024s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.470s; elapsed, 0.221s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 35.250s; elapsed, 2.219s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.910s; elapsed, 0.118s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.840s; elapsed, 0.115s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.990s; elapsed, 0.188s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 38.560s; elapsed, 2.412s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 38.560s; elapsed, 2.412s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 2.040s; elapsed, 2.044s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.910s; elapsed, 0.118s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.024s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.470s; elapsed, 0.221s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.990s; elapsed, 0.188s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.840s; elapsed, 0.115s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 2.090s; elapsed, 2.091s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 35.250s; elapsed, 2.219s, #calls: 1', 'TOTAL : CPU, 88.400s; elapsed, 9.434s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.830s; elapsed, 0.819s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.860s; elapsed, 0.858s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.030s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.520s; elapsed, 0.223s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 30.270s; elapsed, 1.907s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.000s; elapsed, 0.125s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.070s; elapsed, 0.130s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.260s; elapsed, 0.204s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.920s; elapsed, 1.944s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.920s; elapsed, 1.944s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.830s; elapsed, 0.819s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.000s; elapsed, 0.125s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.030s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.520s; elapsed, 0.223s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.260s; elapsed, 0.204s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.070s; elapsed, 0.130s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.860s; elapsed, 0.858s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 30.270s; elapsed, 1.907s, #calls: 1', 'TOTAL : CPU, 74.030s; elapsed, 6.238s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.850s; elapsed, 0.843s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.860s; elapsed, 0.864s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.050s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 7.220s; elapsed, 0.459s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 31.850s; elapsed, 2.009s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.370s; elapsed, 0.148s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.580s; elapsed, 0.162s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.480s; elapsed, 0.406s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 34.160s; elapsed, 2.146s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 34.160s; elapsed, 2.146s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.850s; elapsed, 0.843s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.370s; elapsed, 0.148s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.050s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 7.220s; elapsed, 0.459s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.480s; elapsed, 0.406s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.580s; elapsed, 0.162s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.860s; elapsed, 0.864s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 31.850s; elapsed, 2.009s, #calls: 1', 'TOTAL : CPU, 86.900s; elapsed, 7.087s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.220s; elapsed, 0.231s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.236s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.046s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 7.200s; elapsed, 0.455s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 28.890s; elapsed, 1.823s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.610s; elapsed, 0.163s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.930s; elapsed, 0.183s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 7.140s; elapsed, 0.447s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.950s; elapsed, 1.941s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.950s; elapsed, 1.941s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.220s; elapsed, 0.231s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.610s; elapsed, 0.163s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.046s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 7.200s; elapsed, 0.455s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 7.140s; elapsed, 0.447s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.930s; elapsed, 0.183s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.236s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 28.890s; elapsed, 1.823s, #calls: 1', 'TOTAL : CPU, 80.710s; elapsed, 5.525s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.240s; elapsed, 0.238s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.232s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.023s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.330s; elapsed, 0.212s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 29.220s; elapsed, 1.835s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.750s; elapsed, 0.109s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.750s; elapsed, 0.109s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.090s; elapsed, 0.194s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 28.990s; elapsed, 1.830s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 28.990s; elapsed, 1.830s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.240s; elapsed, 0.238s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.750s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.023s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.330s; elapsed, 0.212s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.090s; elapsed, 0.194s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.750s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.232s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 29.220s; elapsed, 1.835s, #calls: 1', 'TOTAL : CPU, 68.820s; elapsed, 4.782s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 3.880s; elapsed, 3.883s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 4.090s; elapsed, 4.098s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.050s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 6.870s; elapsed, 0.435s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 37.710s; elapsed, 2.365s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.460s; elapsed, 0.154s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.680s; elapsed, 0.167s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 7.180s; elapsed, 0.450s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 40.580s; elapsed, 2.539s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 40.580s; elapsed, 2.539s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 3.880s; elapsed, 3.883s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.460s; elapsed, 0.154s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.050s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 6.870s; elapsed, 0.435s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 7.180s; elapsed, 0.450s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.680s; elapsed, 0.167s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 4.090s; elapsed, 4.098s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 37.710s; elapsed, 2.365s, #calls: 1', 'TOTAL : CPU, 106.000s; elapsed, 14.140s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 2.020s; elapsed, 2.023s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 2.110s; elapsed, 2.119s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.048s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 6.330s; elapsed, 0.401s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 38.360s; elapsed, 2.411s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.420s; elapsed, 0.151s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.580s; elapsed, 0.161s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.920s; elapsed, 0.434s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 37.560s; elapsed, 2.354s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 37.560s; elapsed, 2.354s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 2.020s; elapsed, 2.023s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.420s; elapsed, 0.151s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.048s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 6.330s; elapsed, 0.401s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.920s; elapsed, 0.434s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.580s; elapsed, 0.161s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 2.110s; elapsed, 2.119s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 38.360s; elapsed, 2.411s, #calls: 1', 'TOTAL : CPU, 98.830s; elapsed, 10.102s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 1.990s; elapsed, 1.994s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 2.120s; elapsed, 2.119s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.024s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.440s; elapsed, 0.220s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 34.260s; elapsed, 2.157s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.770s; elapsed, 0.111s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.670s; elapsed, 0.104s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.980s; elapsed, 0.187s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 37.390s; elapsed, 2.350s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 37.390s; elapsed, 2.350s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 1.990s; elapsed, 1.994s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.770s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.024s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.440s; elapsed, 0.220s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.980s; elapsed, 0.187s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.670s; elapsed, 0.104s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 2.120s; elapsed, 2.119s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 34.260s; elapsed, 2.157s, #calls: 1', 'TOTAL : CPU, 85.860s; elapsed, 9.266s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.830s; elapsed, 0.835s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.870s; elapsed, 0.863s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.540s; elapsed, 0.051s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 7.370s; elapsed, 0.466s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 32.030s; elapsed, 2.025s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.370s; elapsed, 0.148s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.660s; elapsed, 0.166s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.810s; elapsed, 0.426s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 33.250s; elapsed, 2.094s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 33.250s; elapsed, 2.094s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.830s; elapsed, 0.835s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.370s; elapsed, 0.148s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.540s; elapsed, 0.051s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 7.370s; elapsed, 0.466s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.810s; elapsed, 0.426s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.660s; elapsed, 0.166s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.870s; elapsed, 0.863s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 32.030s; elapsed, 2.025s, #calls: 1', 'TOTAL : CPU, 86.730s; elapsed, 7.075s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 3.890s; elapsed, 3.903s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 4.070s; elapsed, 4.073s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.047s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 7.190s; elapsed, 0.454s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 38.310s; elapsed, 2.408s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.510s; elapsed, 0.157s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.690s; elapsed, 0.168s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.390s; elapsed, 0.401s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 41.380s; elapsed, 2.597s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 41.380s; elapsed, 2.597s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 3.890s; elapsed, 3.903s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.510s; elapsed, 0.157s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.047s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 7.190s; elapsed, 0.454s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.390s; elapsed, 0.401s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.690s; elapsed, 0.168s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 4.070s; elapsed, 4.073s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 38.310s; elapsed, 2.408s, #calls: 1', 'TOTAL : CPU, 106.950s; elapsed, 14.208s']
Data Set Size:=5k
Skipping 5k
Data Set Size:=32k
Skipping 32k
In the event of a crash, it is wise to save the resulting data (JSON and Pickle) to disk…
import json, cPickle
with open("OMP_SOLVER_TIME_1k_all.json", "w") as OMP_SOLVER_TIME_FILE:
OMP_SOLVER_TIME_FILE.write(json.dumps(str_out))
with open("OMP_SOLVER_TIME_1k_all.pickle", "w") as OMP_SOLVER_TIME_FILE:
OMP_SOLVER_TIME_FILE.write(cPickle.dumps(str_out))
…which can then be easily reloaded later. Now, we parse the string, extract the timing data, and add it to the python object at the specified thread number index. This allows for easily sorting and plot the data later.
import cPickle
str_out=cPickle.load(open('OMP_SOLVER_TIME_1k_all.pickle', 'rb'))
To simplify indexing later for plots, we create shortened keys. Additionally, we ca also separate the data based upon unique index set and thread numbers.
str_out_sm={}
for k in str_out.keys():
str_out_sm.update({k.replace('levmar.parameter_flags=','').replace('_strum_1k_omp1_params','_'):str_out[k]})
import pandas as pd
u_dat={}
threads_list=[16]
uniq_ref = set([k.split('_')[1] for k in str_out_sm.keys()])
df_list=[]
for u in uniq_ref:
same_t = lambda: None #Functions are objects, so legit
same_t.strum_scotch_a = []
same_t.strum_metis_a = []
same_t.strum_scotch_bi = []
same_t.strum_metis_bi = []
same_t.strum_scotch_pr = []
same_t.strum_metis_pr = []
same_t.eig_llt = []
same_t.eig_ldlt = []
same_t.eig_bicgstab = []
for t in threads_list:
same_t.strum_metis_a.append(str_out_sm["omp%d_%s"%(t,u)][-2].rsplit(',')[-2].strip()[0:-1])
same_t.eig_ldlt.append(str_out_sm["omp%d_%s"%(t,u)][-3].rsplit(',')[-2].strip()[0:-1])
same_t.strum_metis_bi.append(str_out_sm["omp%d_%s"%(t,u)][-4].rsplit(',')[-2].strip()[0:-1])
same_t.strum_scotch_pr.append(str_out_sm["omp%d_%s"%(t,u)][-5].rsplit(',')[-2].strip()[0:-1])
same_t.strum_scotch_a.append(str_out_sm["omp%d_%s"%(t,u)][-6].rsplit(',')[-2].strip()[0:-1])
same_t.eig_bicgstab.append(str_out_sm["omp%d_%s"%(t,u)][-7].rsplit(',')[-2].strip()[0:-1])
same_t.strum_scotch_bi.append(str_out_sm["omp%d_%s"%(t,u)][-8].rsplit(',')[-2].strip()[0:-1])
same_t.eig_llt.append(str_out_sm["omp%d_%s"%(t,u)][-9].rsplit(',')[-2].strip()[0:-1])
same_t.strum_metis_pr.append(str_out_sm["omp%d_%s"%(t,u)][-10].rsplit(',')[-2].strip()[0:-1])
u_dat.update({u:same_t})
str_E_LLT = "EIG_LLT_%s"%u
str_E_LDLT = "EIG_LDLT_%s"%u
str_E_BICGSTAB = "EIG_BICGSTAB_%s"%u
str_S_SCOTCH_A = "STRUM_SCOTCHA_%s"%u
str_S_METIS_A = "STRUM_METISA_%s"%u
str_S_SCOTCH_BI = "STRUM_SCOTCHBI_%s"%u
str_S_METIS_BI = "STRUM_METISBI_%s"%u
str_S_SCOTCH_PR = "STRUM_SCOTCHPR_%s"%u
str_S_METIS_PR = "STRUM_METISPR_%s"%u
df_list.append( pd.DataFrame({str_E_LLT:same_t.eig_llt, str_E_LDLT:same_t.eig_ldlt, str_E_BICGSTAB:same_t.eig_bicgstab, str_S_SCOTCH_A:same_t.strum_scotch_a, str_S_METIS_A:same_t.strum_metis_a, str_S_SCOTCH_BI:same_t.strum_scotch_bi, str_S_METIS_BI:same_t.strum_metis_bi, str_S_SCOTCH_PR:same_t.strum_scotch_pr, str_S_METIS_PR:same_t.strum_metis_pr }, index=threads_list).transpose() )
We now have a list of Pandas Dataframes, with which we can combine to a single entity. We can then proceed to plot the resulting columns, comparing both Eigen and STRUMPACK for each specific set of refined parameters.
perf_s = pd.concat(df_list).transpose()
perf_s.transpose()
—CLICK FOR OUTPUT—
| 16 | |
|---|---|
| EIG_BICGSTAB_Rxy-Deff-Eta- | 0.024 |
| EIG_LDLT_Rxy-Deff-Eta- | 2.119 |
| EIG_LLT_Rxy-Deff-Eta- | 1.994 |
| STRUM_METISA_Rxy-Deff-Eta- | 2.157 |
| STRUM_METISBI_Rxy-Deff-Eta- | 0.104 |
| STRUM_METISPR_Rxy-Deff-Eta- | 2.350 |
| STRUM_SCOTCHA_Rxy-Deff-Eta- | 0.220 |
| STRUM_SCOTCHBI_Rxy-Deff-Eta- | 0.111 |
| STRUM_SCOTCHPR_Rxy-Deff-Eta- | 0.187 |
| EIG_BICGSTAB_Bfactor- | 0.049 |
| EIG_LDLT_Bfactor- | 0.238 |
| EIG_LLT_Bfactor- | 0.232 |
| STRUM_METISA_Bfactor- | 1.831 |
| STRUM_METISBI_Bfactor- | 0.177 |
| STRUM_METISPR_Bfactor- | 2.001 |
| STRUM_SCOTCHA_Bfactor- | 0.447 |
| STRUM_SCOTCHBI_Bfactor- | 0.169 |
| STRUM_SCOTCHPR_Bfactor- | 0.422 |
| EIG_BICGSTAB_Bfactor-Deff-Eta- | 0.051 |
| EIG_LDLT_Bfactor-Deff-Eta- | 0.863 |
| EIG_LLT_Bfactor-Deff-Eta- | 0.835 |
| STRUM_METISA_Bfactor-Deff-Eta- | 2.025 |
| STRUM_METISBI_Bfactor-Deff-Eta- | 0.166 |
| STRUM_METISPR_Bfactor-Deff-Eta- | 2.094 |
| STRUM_SCOTCHA_Bfactor-Deff-Eta- | 0.466 |
| STRUM_SCOTCHBI_Bfactor-Deff-Eta- | 0.148 |
| STRUM_SCOTCHPR_Bfactor-Deff-Eta- | 0.426 |
| EIG_BICGSTAB_Rxy-Bfactor-Deff-Eta- | 0.047 |
| EIG_LDLT_Rxy-Bfactor-Deff-Eta- | 4.073 |
| EIG_LLT_Rxy-Bfactor-Deff-Eta- | 3.903 |
| ... | ... |
| STRUM_SCOTCHA_Deff- | 0.215 |
| STRUM_SCOTCHBI_Deff- | 0.109 |
| STRUM_SCOTCHPR_Deff- | 0.192 |
| EIG_BICGSTAB_Rxy-Bfactor- | 0.048 |
| EIG_LDLT_Rxy-Bfactor- | 2.120 |
| EIG_LLT_Rxy-Bfactor- | 1.995 |
| STRUM_METISA_Rxy-Bfactor- | 2.191 |
| STRUM_METISBI_Rxy-Bfactor- | 0.163 |
| STRUM_METISPR_Rxy-Bfactor- | 2.391 |
| STRUM_SCOTCHA_Rxy-Bfactor- | 0.417 |
| STRUM_SCOTCHBI_Rxy-Bfactor- | 0.153 |
| STRUM_SCOTCHPR_Rxy-Bfactor- | 0.433 |
| EIG_BICGSTAB_Rxy-Bfactor-Eta- | 0.048 |
| EIG_LDLT_Rxy-Bfactor-Eta- | 2.119 |
| EIG_LLT_Rxy-Bfactor-Eta- | 2.023 |
| STRUM_METISA_Rxy-Bfactor-Eta- | 2.411 |
| STRUM_METISBI_Rxy-Bfactor-Eta- | 0.161 |
| STRUM_METISPR_Rxy-Bfactor-Eta- | 2.354 |
| STRUM_SCOTCHA_Rxy-Bfactor-Eta- | 0.401 |
| STRUM_SCOTCHBI_Rxy-Bfactor-Eta- | 0.151 |
| STRUM_SCOTCHPR_Rxy-Bfactor-Eta- | 0.434 |
| EIG_BICGSTAB_Rxy-Bfactor-Deff- | 0.050 |
| EIG_LDLT_Rxy-Bfactor-Deff- | 4.098 |
| EIG_LLT_Rxy-Bfactor-Deff- | 3.883 |
| STRUM_METISA_Rxy-Bfactor-Deff- | 2.365 |
| STRUM_METISBI_Rxy-Bfactor-Deff- | 0.167 |
| STRUM_METISPR_Rxy-Bfactor-Deff- | 2.539 |
| STRUM_SCOTCHA_Rxy-Bfactor-Deff- | 0.435 |
| STRUM_SCOTCHBI_Rxy-Bfactor-Deff- | 0.154 |
| STRUM_SCOTCHPR_Rxy-Bfactor-Deff- | 0.450 |
135 rows × 1 columns
With the performance data in an easily accessible format, we can plot each respective data set for all solvers. Each bar corresponds to the parameters being refined, where the colour indicates the respective solver used, and the y-axis is the time to solve the system in seconds.
# define the figure size and grid layout properties
matplotlib.rcParams.update({'font.size': 22})
figsize = (20, 16)
cols = 4
fig3 = plt.figure(num=3, figsize=figsize)
ax3 = plt.axes()
ii = []
uu=[]
keys=['EIG_LLT','EIG_LDLT','STRUM_METISA','STRUM_SCOTCHA','STRUM_METISBI','STRUM_SCOTCHBI','STRUM_METISPR','STRUM_SCOTCHPR','EIG_BICGSTAB']
colours = ['#a6cee3','#1f78b4','#b2df8a','#33a02c','#fb9a99','#e31a1c','#fdbf6f','#ff7f00','#cab2d6']
key_col = {k:v for (k,v) in zip(keys,colours)}
hatch_pattern = [ '/' , '\\' , '|' , '-' , '+' , 'x' ,'o' ,'O' , '.', '*']
for i, u in enumerate(uniq_ref):
p_list=[]
ax3.set_ylabel('time [s]')
keys_df = [k+'_'+u for k in keys]
dfl=perf_s[keys_df]
m = zip(dfl.as_matrix().astype(np.float)[0], dfl.keys().tolist() )
m = sorted(m,key=lambda x: x[0])
for p in reversed(xrange(len(m))):
c_label = "_".join(m[p][1].split('_')[0:2])
p_list.append( plt.bar(i, m[p][0], label=m[p][1], color=key_col[c_label], edgecolor='k') )
ii.append(i)
uu.append(u)
ax3.set_yscale('log',basey=10)
plt.title('1k frames OMP_NUM_THREADS=16', fontsize=24)
plt.xticks(ii, uu)
plt.setp(ax3.get_xticklabels(), rotation=90, ha='left')
ax3.legend(p_list,keys,bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.,)
plt.savefig('omp_1kframes_allsolvers_omp16.pdf', pad_inches=10)
 |
So far we have examined the single threaded and OpenMP enabled solver performance. Strumpack also allows us to use the distributed MPI backend. We can set this up as follows using the ipyparallel backend, or invoke an mpirun call using the cell bash bash environment. For completeness, the ipyparallel environment can be set up as follows:
conda install ipyparallel
jupyter serverextension enable --py ipyparallel --user
jupyter nbextension install --py ipyparallel --user
jupyter nbextension enable --py ipyparallel --user
ipcluster start --profile=mpi -n 4
Take note of the controller client json file <HOME>/.ipython/profile_<PROFILE_NAME>/security/ipcontroller-client.json. This will be used to allow the notebook to connect to the ipyparallel cluster profile mpi. We can then return the handle of the cluster environment with:
import ipyparallel as ipp
#c = ipp.Client('/net/cci-filer2/raid1/home/mlxd/.ipython/profile_mpi/security/ipcontroller-client.json')
c = ipp.Client()
rc=c[:]
c.ids #Instance IDs
[0]
For our purposes it can be easier to call the solver using the ! bash environment, as was used for the OpenMP studies earlier. A simple test script can be performed as follows, which tests the listed solvers against the MPI backend.
!OMP_NUM_THREADS=1 mpirun -np 2 libtbx.python \
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/\
modules/cctbx_project/scitbx/examples/bevington/strumpack_eigen_solver_mpi_dist.py \
/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/\
out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv \
/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/\
out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.
try: mod = __import__(name)
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.
try: mod = __import__(name)
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.
try: mod = __import__(name)
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.
try: mod = __import__(name)
14151 7075.5
# ***** WARNING ****************************************************
# Detected a large number of levels in the frontal/elimination tree.
# STRUMPACK currently does not handle this safely, which
# could lead to segmentation faults due to stack overflows.
# As a remedy, you can try to increase the stack size,
# or try a different ordering (metis, scotch, ..).
# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,
# (enabled by default) iso --sp_enable_METIS_NodeND.
# ******************************************************************
individual call time for STRUMPACK_OMP: CPU, 0.350s; elapsed, 0.344s
individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.050s
individual call time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.027s
CPP -2
# Initializing STRUMPACK
# using 1 OpenMP thread
CPP -2
# using 2 MPI processes
# matching job: maximum matching with row and column scaling
# initial matrix:
# - number of unknowns = 14,151
# - number of nonzeros = 72,305
# nested dissection reordering:
# - Scotch reordering
# - strategy parameter = 8
# - number of separators = 4,075
# - number of levels = 68
# ***** WARNING ****************************************************
# Detected a large number of levels in the frontal/elimination tree.
# STRUMPACK currently does not handle this safely, which
# could lead to segmentation faults due to stack overflows.
# As a remedy, you can try to increase the stack size,
# or try a different ordering (metis, scotch, ..).
# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,
# (enabled by default) iso --sp_enable_METIS_NodeND.
# ******************************************************************
# - nd time = 0.0810261
# - matching time = 0.0116782
# - symmetrization time = 0.00337219
# symbolic factorization:
# - nr of dense Frontal matrices = 4,075
# - nr of HSS Frontal matrices = 0
# - symb-factor time = 0.021524
# multifrontal factorization:
# - estimated memory usage (exact solver) = 7.70359 MB
# - factor time = 0.223322
# - factor nonzeros = 962,949
# - factor memory = 7.70359 MB
# - total flops = 2.49112e+08, min = 3.46562e+07, max = 2.14456e+08
# - flop rate = 1.11548 GFlop/s
# - factor memory/nonzeros = 100 % of multifrontal
# - maximum HSS rank = 0
# - HSS compression = false
# - relative compression tolerance = 0.01
# - absolute compression tolerance = 1e-08
# - normal(0,1) distribution with minstd_rand engine
# ----- FLOP BREAKDOWN ---------------------
# compression = 0
# random = 0
# ID = 0
# QR = 0
# ortho = 0
# reduce_samples = 0
# update_samples = 0
# extraction = 0
# sampling = 0
# CB_sample = 0
# sparse_sampling = 0
# ULV_factor = 0
# Schur = 0
# full_rank = 2.63271e+08
# --------------------------------------------
# total = 2.63271e+08
# --------------------------------------------
REFINEMENT it. 0 res = 15322 rel.res = 1 bw.error = 1
REFINEMENT it. 1 res = 1.1079e-11 rel.res = 7.23077e-16 bw.error = 1.25632e-15
# DIRECT/GMRES solve:
# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000
# - number of Krylov iterations = 1
# - solve time = 0.011821
# - total flops = 2.47602e+06, min = 838849, max = 1.63717e+06
# - flop rate = 0.209459 GFlop/s
# - bytes moved = 21.8361 MB, min = 0 MB, max = 0 MB
# - byte rate = 1.84722 GByte/s
# - solve arithmetic intensity = 0.113391 flop/byte
individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.350s; elapsed, 0.359s
individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.370s; elapsed, 0.365s
Exiting profiler
time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.350s; elapsed, 0.359s, #calls: 1
TOTAL : CPU, 0.350s; elapsed, 0.359s
Strumpack solutions agree with Eigen
Exiting profiler
time for STRUMPACK_OMP: CPU, 0.350s; elapsed, 0.344s, #calls: 1
time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.027s, #calls: 1
time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.050s, #calls: 1
time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.370s; elapsed, 0.365s, #calls: 1
TOTAL : CPU, 0.790s; elapsed, 0.786s
We now use the same procedure of creating a solver file, write to disk, and call with the respective matrix arguments with OMP_NUM_THREADS and mpirun -n. For simplicity, we examine the OpenMP STRUM_SCOTCHA, EIGEN_LDLT and EIGEN_BICGSTAB, compared with the MPI-enabled STRUM_SCOTCHA (we may also examine the full range of algorithms as used in the previous example).
MPI_SOLVER='''
from __future__ import division
import mpi4py
mpi4py.rc.threads = True
mpi4py.rc.thread_level = "funneled"
from mpi4py import MPI
assert MPI.Is_initialized()
assert MPI.Query_thread() == MPI.THREAD_FUNNELED
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size()
from scitbx.matrix import sqr,col
from cctbx.array_family import flex
from libtbx.test_utils import approx_equal
from libtbx.development.timers import Profiler
import boost.python
ext_omp = boost.python.import_ext("scitbx_examples_strumpack_solver_ext")
ext_mpi = boost.python.import_ext("scitbx_examples_strumpack_mpi_dist_solver_ext")
import sys
import numpy as np
import scipy.sparse as sps
if rank==0:
A_mat = np.loadtxt(sys.argv[1],dtype={'names':('rows','cols','vals'),'formats':('i8','i8','f8')})
b_vec = np.loadtxt(sys.argv[2])
n_rows = len(b_vec)
n_cols = n_rows
nnz = len(A_mat['vals'])
print n_rows, n_rows/size
#Convert the sparse CSR to flex doubles, then use them to solve using the implemented framework
A_sp = sps.csr_matrix((A_mat['vals'],(A_mat['rows'],A_mat['cols'])))
#Check if upper/lower triangular only, and generate full if so
tu=sps.triu(A_sp)
tl=sps.tril(A_sp)
sd=sps.diags(A_sp.diagonal())
A_spS = A_sp
if tu.nnz == sd.getnnz() or tl.nnz == sd.getnnz():
A_spS = A_sp + A_sp.transpose() - sd
import numpy as np
row_idx_split = np.array_split(np.arange(n_rows),size)
len_row_idx_split = flex.int( np.cumsum( np.append([0], [len(i) for i in row_idx_split]) ).tolist() )
P = Profiler("STRUMPACK_OMP")
A_indptr = flex.int( A_spS.indptr )
A_indices = flex.int( A_spS.indices )
A_data = flex.double( A_spS.data )
b = flex.double( b_vec )
res_strum_omp = ext_omp.strumpack_solver(
n_rows, n_cols,
A_indptr, A_indices,
A_data, b,
ext_omp.scotch,
ext_omp.auto
)
del P
P = Profiler("EIGEN_LDLT")
res_eig_ldlt = ext_omp.eigen_solver(1, n_rows, n_cols, A_indptr, A_indices, A_data, b)
del P
P = Profiler("EIGEN_BICGSTAB")
res_eig_bicgstab = ext_omp.eigen_solver(2, n_rows, n_cols, A_indptr, A_indices, A_data, b)
del P
else:
A_spS=None
row_idx_split=None
len_row_idx_split=None
b_vec=None
n_cols=None
if size>1:
#Broadcast data to each rank
A_spS = comm.bcast(A_spS, root=0)
row_idx_split = comm.bcast(row_idx_split, root=0)
len_row_idx_split = comm.bcast(len_row_idx_split, root=0)
b_vec = comm.bcast(b_vec, root=0)
n_cols = comm.bcast(n_cols, root=0)
#Take subset of data for each rank
A_row_offset = flex.int(A_spS[row_idx_split[rank],:].indptr)
A_col_offset = flex.int(A_spS[row_idx_split[rank],:].indices)
A_values = flex.double(A_spS[row_idx_split[rank],:].data)
b = flex.double(b_vec[row_idx_split[rank]])
P = Profiler("STRUMPACK_MPI_DIST_RANK=%d"%rank)
res_strum_mpi_local = ext_mpi.strumpack_mpi_dist_solver(len(row_idx_split[rank]), n_cols, comm, A_row_offset, A_col_offset, A_values, b, len_row_idx_split, ext_mpi.scotch, ext_mpi.auto)
strum_result_mpi_list = comm.gather(res_strum_mpi_local.x, root=0)
del P
if rank==0:
strum_result_mpi = flex.double()
for l in strum_result_mpi_list:
strum_result_mpi.extend(l)
#MPI.Finalize()
if rank==0:
strum_result_omp = res_strum_omp.x
eig_result_ldlt = res_eig_ldlt.x
eig_result_bicgstab = res_eig_bicgstab.x
for ii in xrange(len(strum_result_omp)):
assert approx_equal( strum_result_omp[ii], eig_result_ldlt[ii] )
assert approx_equal( strum_result_omp[ii], eig_result_bicgstab[ii] )
if size>1:
assert approx_equal( strum_result_omp[ii], strum_result_mpi[ii] )
print "Strumpack solutions agree with Eigen"
'''
MPI_SOLVER_FILE = open("MPI_SOLVER.py", "w")
MPI_SOLVER_FILE.write(MPI_SOLVER)
MPI_SOLVER_FILE.close()
DATAPATH="/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/"
A_LIST = !find {DATAPATH} -iname "A*.csv"
B_LIST = [ii.replace('/A_','/b_') for ii in A_LIST]
We intend to examine performance while varying both OMP_NUM_THREADS and the MPI process number to determine an optimal set of parameters for the distributed backend. In the following example we test between 1 to 16 OpenMP threads per process, and from 1 to 16 MPI processes, assuming a total of threads*processes <= 32 cores. The resulting data set will be a scalar field of time as a function of both variables, which we can plot as a heatmap. We can begin by running the analysis on a 1000 image dataset.
str_out={}
import os
threads_list = [1,2,4,8,16]
mpi_proc_list = [1,2,4,8,16]
for imgs_size in list_idx:
print "Data Set Size:=%s"%imgs_size
#Subselect the smallest data size for now
if imgs_size != "5k":
print "Skipping %s"%imgs_size
continue
for imgs_idx in list_idx[imgs_size]:
A_path = A_LIST[imgs_idx]; b_path = B_LIST[imgs_idx]
dat_name = A_path.split('/')[-1][2:-4]
print "Data Set Name:=%s"%(dat_name)
#Ensure the A and b data are matched correctly
assert(os.path.dirname(A_path) == os.path.dirname(b_path))
print {A_path}, {b_path}
for procs in mpi_proc_list:
for threads in threads_list:
if threads*procs <= 32:
print "mpirun -n %d"%procs
print "OMP_NUM_THREADS:=%d"%threads
val = !OMP_NUM_THREADS={threads} mpirun -n {procs} libtbx.python ./MPI_SOLVER.py {A_path} {b_path}
print val
for s in val:
if "assert" in s:
raise Exception("Solutions not equal! Halting with log: %s"%val)
key = 'mpi' + str(procs) + 'omp' + str(threads) + '_' + dat_name
str_out.update({key:val})
—CLICK FOR OUTPUT—
Data Set Size:=10k
Skipping 10k
Data Set Size:=1k
Skipping 1k
Data Set Size:=5k
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Rxy-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-/A_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-/b_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.630s; elapsed, 5.624s', 'individual call time for EIGEN_LDLT: CPU, 2.490s; elapsed, 2.496s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.056s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.630s; elapsed, 5.624s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.056s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.490s; elapsed, 2.496s, #calls: 1', 'TOTAL : CPU, 8.170s; elapsed, 8.175s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.920s; elapsed, 3.464s', 'individual call time for EIGEN_LDLT: CPU, 2.660s; elapsed, 2.483s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.044s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.920s; elapsed, 3.464s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.044s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.660s; elapsed, 2.483s, #calls: 1', 'TOTAL : CPU, 9.640s; elapsed, 5.992s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 9.110s; elapsed, 2.283s', 'individual call time for EIGEN_LDLT: CPU, 3.630s; elapsed, 3.089s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.041s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 9.110s; elapsed, 2.283s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.041s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.630s; elapsed, 3.089s, #calls: 1', 'TOTAL : CPU, 12.840s; elapsed, 5.413s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 22.270s; elapsed, 2.789s', 'individual call time for EIGEN_LDLT: CPU, 5.570s; elapsed, 4.334s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.042s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 22.270s; elapsed, 2.789s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.042s, #calls: 1', 'time for EIGEN_LDLT: CPU, 5.570s; elapsed, 4.334s, #calls: 1', 'TOTAL : CPU, 28.020s; elapsed, 7.164s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 38.440s; elapsed, 2.409s', 'individual call time for EIGEN_LDLT: CPU, 5.150s; elapsed, 2.538s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.041s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 38.440s; elapsed, 2.409s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.041s, #calls: 1', 'time for EIGEN_LDLT: CPU, 5.150s; elapsed, 2.538s, #calls: 1', 'TOTAL : CPU, 43.870s; elapsed, 4.989s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.910s; elapsed, 5.904s', 'individual call time for EIGEN_LDLT: CPU, 2.560s; elapsed, 2.559s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.058s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,927', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.173102', '# - matching time = 0.031208', '# - symmetrization time = 0.015681', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,927', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0421081', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.1376 MB', '# - factor time = 3.87764', '# - factor nonzeros = 6,142,201', '# - factor memory = 49.1376 MB', '# - total flops = 7.46369e+09, min = 2.94438e+09, max = 4.51931e+09', '# - flop rate = 1.92481 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.06944e+09', '# --------------------------------------------', '# total = 8.06944e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.6413e-11\trel.res = 1.55862e-15\tbw.error = 3.62098e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.042994', '# - total flops = 1.47533e+07, min = 6.98046e+06, max = 7.77288e+06', '# - flop rate = 0.343149 GFlop/s', '# - bytes moved = 105.81 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.46104 GByte/s', '# - solve arithmetic intensity = 0.139433 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.200s; elapsed, 4.201s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.200s; elapsed, 4.211s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.200s; elapsed, 4.201s, #calls: 1', 'TOTAL : CPU, 4.200s; elapsed, 4.201s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.910s; elapsed, 5.904s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.058s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.560s; elapsed, 2.559s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.200s; elapsed, 4.211s, #calls: 1', 'TOTAL : CPU, 12.720s; elapsed, 12.732s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.910s; elapsed, 3.958s', 'individual call time for EIGEN_LDLT: CPU, 2.650s; elapsed, 2.465s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.048s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,927', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.165474', '# - matching time = 0.032366', '# - symmetrization time = 0.0156758', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,927', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0455151', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.1376 MB', '# - factor time = 2.88028', '# - factor nonzeros = 6,142,201', '# - factor memory = 49.1376 MB', '# - total flops = 7.46376e+09, min = 2.94438e+09, max = 4.51938e+09', '# - flop rate = 2.59133 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.06944e+09', '# --------------------------------------------', '# total = 8.06944e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.67124e-11\trel.res = 1.57144e-15\tbw.error = 3.63192e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0395958', '# - total flops = 1.47589e+07, min = 6.98046e+06, max = 7.77839e+06', '# - flop rate = 0.372737 GFlop/s', '# - bytes moved = 106.027 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.67773 GByte/s', '# - solve arithmetic intensity = 0.139199 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 5.760s; elapsed, 3.205s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 6.430s; elapsed, 3.215s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 5.760s; elapsed, 3.205s, #calls: 1', 'TOTAL : CPU, 5.760s; elapsed, 3.205s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.910s; elapsed, 3.958s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.048s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.650s; elapsed, 2.465s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 6.430s; elapsed, 3.215s, #calls: 1', 'TOTAL : CPU, 17.060s; elapsed, 9.685s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 10.940s; elapsed, 2.741s', 'individual call time for EIGEN_LDLT: CPU, 6.260s; elapsed, 5.728s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.057s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,927', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.165105', '# - matching time = 0.0321209', '# - symmetrization time = 0.01565', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,927', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0445859', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.1376 MB', '# - factor time = 2.10367', '# - factor nonzeros = 6,142,201', '# - factor memory = 49.1376 MB', '# - total flops = 7.46376e+09, min = 2.94438e+09, max = 4.51938e+09', '# - flop rate = 3.54796 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.06944e+09', '# --------------------------------------------', '# total = 8.06944e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.6699e-11\trel.res = 1.57086e-15\tbw.error = 3.63192e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0426569', '# - total flops = 1.47589e+07, min = 6.98046e+06, max = 7.77839e+06', '# - flop rate = 0.34599 GFlop/s', '# - bytes moved = 106.038 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.48584 GByte/s', '# - solve arithmetic intensity = 0.139184 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.690s; elapsed, 2.430s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.750s; elapsed, 2.439s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.690s; elapsed, 2.430s, #calls: 1', 'TOTAL : CPU, 9.690s; elapsed, 2.430s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 10.940s; elapsed, 2.741s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.057s, #calls: 1', 'time for EIGEN_LDLT: CPU, 6.260s; elapsed, 5.728s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.750s; elapsed, 2.439s, #calls: 1', 'TOTAL : CPU, 27.070s; elapsed, 10.966s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 22.810s; elapsed, 2.860s', 'individual call time for EIGEN_LDLT: CPU, 3.760s; elapsed, 2.491s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.038s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,927', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.194211', '# - matching time = 0.0363538', '# - symmetrization time = 0.0187259', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,927', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0537779', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.1376 MB', '# - factor time = 2.60438', '# - factor nonzeros = 6,142,201', '# - factor memory = 49.1376 MB', '# - total flops = 7.46376e+09, min = 2.94438e+09, max = 4.51938e+09', '# - flop rate = 2.86585 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.06944e+09', '# --------------------------------------------', '# total = 8.06944e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.66517e-11\trel.res = 1.56884e-15\tbw.error = 3.63192e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0529561', '# - total flops = 1.47589e+07, min = 6.98046e+06, max = 7.77839e+06', '# - flop rate = 0.2787 GFlop/s', '# - bytes moved = 106.061 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.0028 GByte/s', '# - solve arithmetic intensity = 0.139155 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 21.600s; elapsed, 2.991s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 23.830s; elapsed, 3.000s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 21.600s; elapsed, 2.991s, #calls: 1', 'TOTAL : CPU, 21.600s; elapsed, 2.991s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 22.810s; elapsed, 2.860s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.038s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.760s; elapsed, 2.491s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 23.830s; elapsed, 3.000s, #calls: 1', 'TOTAL : CPU, 50.550s; elapsed, 8.389s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 40.870s; elapsed, 2.576s', 'individual call time for EIGEN_LDLT: CPU, 5.590s; elapsed, 2.941s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.039s', 'CPP -2', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,927', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.208532', '# - matching time = 0.05004', '# - symmetrization time = 0.02074', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,927', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.061316', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.1376 MB', '# - factor time = 2.22156', '# - factor nonzeros = 6,142,201', '# - factor memory = 49.1376 MB', '# - total flops = 7.46395e+09, min = 2.94438e+09, max = 4.51957e+09', '# - flop rate = 3.35978 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.06944e+09', '# --------------------------------------------', '# total = 8.06944e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.65568e-11\trel.res = 1.56478e-15\tbw.error = 3.57131e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0466371', '# - total flops = 1.47589e+07, min = 6.98046e+06, max = 7.77839e+06', '# - flop rate = 0.316462 GFlop/s', '# - bytes moved = 106.109 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.2752 GByte/s', '# - solve arithmetic intensity = 0.139092 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 41.450s; elapsed, 2.640s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 41.650s; elapsed, 2.644s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 41.450s; elapsed, 2.640s, #calls: 1', 'TOTAL : CPU, 41.450s; elapsed, 2.640s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 40.870s; elapsed, 2.576s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.039s, #calls: 1', 'time for EIGEN_LDLT: CPU, 5.590s; elapsed, 2.941s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 41.650s; elapsed, 2.644s, #calls: 1', 'TOTAL : CPU, 88.370s; elapsed, 8.199s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.280s; elapsed, 6.282s', 'individual call time for EIGEN_LDLT: CPU, 2.610s; elapsed, 2.614s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.060s', 'CPP -2', '# Initializing STRUMPACK', 'CPP -2', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,931', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.189291', '# - matching time = 0.036289', '# - symmetrization time = 0.012743', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,931', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.036417', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.3058 MB', '# - factor time = 2.50507', '# - factor nonzeros = 6,163,223', '# - factor memory = 49.3058 MB', '# - total flops = 7.53221e+09, min = 865688, max = 2.95202e+09', '# - flop rate = 3.00679 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.13207e+09', '# --------------------------------------------', '# total = 8.13207e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.02608e-11\trel.res = 1.29528e-15\tbw.error = 2.43099e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0349491', '# - total flops = 1.6493e+07, min = 43055, max = 6.92318e+06', '# - flop rate = 0.471915 GFlop/s', '# - bytes moved = 98.2063 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.80998 GByte/s', '# - solve arithmetic intensity = 0.167942 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.800s; elapsed, 2.823s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.810s; elapsed, 2.824s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.810s; elapsed, 2.821s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.830s; elapsed, 2.829s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.800s; elapsed, 2.823s, #calls: 1', 'TOTAL : CPU, 2.800s; elapsed, 2.823s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.810s; elapsed, 2.824s, #calls: 1', 'TOTAL : CPU, 2.810s; elapsed, 2.824s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.810s; elapsed, 2.821s, #calls: 1', 'TOTAL : CPU, 2.810s; elapsed, 2.821s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.280s; elapsed, 6.282s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.060s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.610s; elapsed, 2.614s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.830s; elapsed, 2.829s, #calls: 1', 'TOTAL : CPU, 11.780s; elapsed, 11.785s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 8.020s; elapsed, 4.018s', 'individual call time for EIGEN_LDLT: CPU, 2.700s; elapsed, 2.522s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.060s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,931', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.174702', '# - matching time = 0.032347', '# - symmetrization time = 0.014987', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,931', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.039263', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.3058 MB', '# - factor time = 2.04756', '# - factor nonzeros = 6,163,223', '# - factor memory = 49.3058 MB', '# - total flops = 7.53223e+09, min = 865688, max = 2.95202e+09', '# - flop rate = 3.67864 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.13207e+09', '# --------------------------------------------', '# total = 8.13207e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.01024e-11\trel.res = 1.2885e-15\tbw.error = 2.41537e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0343449', '# - total flops = 1.64951e+07, min = 43055, max = 6.92318e+06', '# - flop rate = 0.480278 GFlop/s', '# - bytes moved = 98.2807 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.86158 GByte/s', '# - solve arithmetic intensity = 0.167837 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.070s; elapsed, 2.357s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.710s; elapsed, 2.358s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.990s; elapsed, 2.362s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.700s; elapsed, 2.366s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.710s; elapsed, 2.358s, #calls: 1', 'TOTAL : CPU, 4.710s; elapsed, 2.358s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.990s; elapsed, 2.362s, #calls: 1', 'TOTAL : CPU, 2.990s; elapsed, 2.362s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.070s; elapsed, 2.357s, #calls: 1', 'TOTAL : CPU, 4.070s; elapsed, 2.357s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 8.020s; elapsed, 4.018s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.060s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.700s; elapsed, 2.522s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.700s; elapsed, 2.366s, #calls: 1', 'TOTAL : CPU, 15.520s; elapsed, 8.966s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 10.990s; elapsed, 2.758s', 'individual call time for EIGEN_LDLT: CPU, 4.350s; elapsed, 3.797s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.045s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,931', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.206648', '# - matching time = 0.039171', '# - symmetrization time = 0.0150321', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,931', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0403199', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.3058 MB', '# - factor time = 2.20638', '# - factor nonzeros = 6,163,223', '# - factor memory = 49.3058 MB', '# - total flops = 7.53223e+09, min = 866384, max = 2.95202e+09', '# - flop rate = 3.41384 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.13207e+09', '# --------------------------------------------', '# total = 8.13207e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.01136e-11\trel.res = 1.28898e-15\tbw.error = 2.43099e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.038558', '# - total flops = 1.64951e+07, min = 43055, max = 6.92318e+06', '# - flop rate = 0.4278 GFlop/s', '# - bytes moved = 98.2891 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.54912 GByte/s', '# - solve arithmetic intensity = 0.167822 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 7.960s; elapsed, 2.562s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.530s; elapsed, 2.565sindividual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.930s; elapsed, 2.563s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.060s; elapsed, 2.570s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.930s; elapsed, 2.563s, #calls: 1', 'TOTAL : CPU, 9.930s; elapsed, 2.563s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 7.960s; elapsed, 2.562s, #calls: 1', 'TOTAL : CPU, 7.960s; elapsed, 2.562s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.530s; elapsed, 2.565s, #calls: 1', 'TOTAL : CPU, 4.530s; elapsed, 2.565s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 10.990s; elapsed, 2.758s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.045s, #calls: 1', 'time for EIGEN_LDLT: CPU, 4.350s; elapsed, 3.797s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.060s; elapsed, 2.570s, #calls: 1', 'TOTAL : CPU, 25.510s; elapsed, 9.170s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 19.800s; elapsed, 2.481s', 'individual call time for EIGEN_LDLT: CPU, 5.070s; elapsed, 3.836s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.038s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,931', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.202113', '# - matching time = 0.0367491', '# - symmetrization time = 0.015816', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,931', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0441282', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.3058 MB', '# - factor time = 2.54222', '# - factor nonzeros = 6,163,223', '# - factor memory = 49.3058 MB', '# - total flops = 7.53224e+09, min = 866384, max = 2.95202e+09', '# - flop rate = 2.96286 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.13207e+09', '# --------------------------------------------', '# total = 8.13207e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.01093e-11\trel.res = 1.2888e-15\tbw.error = 2.43099e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0427811', '# - total flops = 1.64951e+07, min = 43055, max = 6.92318e+06', '# - flop rate = 0.38557 GFlop/s', '# - bytes moved = 98.3065 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.2979 GByte/s', '# - solve arithmetic intensity = 0.167793 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 7.520s; elapsed, 2.905s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 20.270s; elapsed, 2.903s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.040s; elapsed, 2.904s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.860s; elapsed, 2.911s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.040s; elapsed, 2.904s, #calls: 1', 'TOTAL : CPU, 16.040s; elapsed, 2.904s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 20.270s; elapsed, 2.903s, #calls: 1', 'TOTAL : CPU, 20.270s; elapsed, 2.903s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 7.520s; elapsed, 2.905s, #calls: 1', 'TOTAL : CPU, 7.520s; elapsed, 2.905s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 19.800s; elapsed, 2.481s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.038s, #calls: 1', 'time for EIGEN_LDLT: CPU, 5.070s; elapsed, 3.836s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.860s; elapsed, 2.911s, #calls: 1', 'TOTAL : CPU, 47.870s; elapsed, 9.266s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 2663.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.430s; elapsed, 6.432s', 'individual call time for EIGEN_LDLT: CPU, 2.470s; elapsed, 2.473s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.063s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,945', '# - number of levels = 126', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.163474', '# - matching time = 0.0298009', '# - symmetrization time = 0.0128169', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,945', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0309041', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.269 MB', '# - factor time = 2.14071', '# - factor nonzeros = 6,158,621', '# - factor memory = 49.269 MB', '# - total flops = 7.49292e+09, min = 698726, max = 2.95246e+09', '# - flop rate = 3.50021 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.05389e+09', '# --------------------------------------------', '# total = 8.05389e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.57236e-11\trel.res = 1.10107e-15\tbw.error = 2.56229e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0334761', '# - total flops = 2.07556e+07, min = 27135, max = 6.90876e+06', '# - flop rate = 0.620012 GFlop/s', '# - bytes moved = 93.7947 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.80184 GByte/s', '# - solve arithmetic intensity = 0.221288 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.400s; elapsed, 2.421s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.400s; elapsed, 2.417s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.420s; elapsed, 2.423s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.380s; elapsed, 2.421s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.410s; elapsed, 2.422s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.390s; elapsed, 2.419s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.410s; elapsed, 2.422s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.430s; elapsed, 2.427s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.400s; elapsed, 2.417s, #calls: 1', 'TOTAL : CPU, 2.400s; elapsed, 2.417s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.380s; elapsed, 2.421s, #calls: 1', 'TOTAL : CPU, 2.380s; elapsed, 2.421s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.390s; elapsed, 2.419s, #calls: 1', 'TOTAL : CPU, 2.390s; elapsed, 2.419s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.410s; elapsed, 2.422s, #calls: 1', 'TOTAL : CPU, 2.410s; elapsed, 2.422s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.400s; elapsed, 2.421s, #calls: 1', 'TOTAL : CPU, 2.400s; elapsed, 2.421s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.410s; elapsed, 2.422s, #calls: 1', 'TOTAL : CPU, 2.410s; elapsed, 2.422s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.420s; elapsed, 2.423s, #calls: 1', 'TOTAL : CPU, 2.420s; elapsed, 2.423s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.430s; elapsed, 6.432s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.063s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.470s; elapsed, 2.473s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.430s; elapsed, 2.427s, #calls: 1', 'TOTAL : CPU, 11.390s; elapsed, 11.394s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 2663.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.380s; elapsed, 3.692s', 'individual call time for EIGEN_LDLT: CPU, 2.740s; elapsed, 2.572s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.053s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,945', '# - number of levels = 126', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************# - nd time = ', '0.176074', '# - matching time = 0.033097', '# - symmetrization time = 0.013762', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,945', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0355759', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.269 MB', '# - factor time = 2.18519', '# - factor nonzeros = 6,158,621', '# - factor memory = 49.269 MB', '# - total flops = 7.49293e+09, min = 698726, max = 2.95246e+09', '# - flop rate = 3.42897 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.05389e+09', '# --------------------------------------------', '# total = 8.05389e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.55859e-11\trel.res = 1.09518e-15\tbw.error = 2.50926e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.041822', '# - total flops = 2.07556e+07, min = 27135, max = 6.90876e+06', '# - flop rate = 0.496285 GFlop/s', '# - bytes moved = 93.8014 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.24287 GByte/s', '# - solve arithmetic intensity = 0.221272 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 4.330s; elapsed, 2.499s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 4.050s; elapsed, 2.504s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.190s; elapsed, 2.504s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.140s; elapsed, 2.503s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.910s; elapsed, 2.503s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 3.430s; elapsed, 2.503s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 3.350s; elapsed, 2.501s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 5.010s; elapsed, 2.508s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 4.330s; elapsed, 2.499s, #calls: 1', 'TOTAL : CPU, 4.330s; elapsed, 2.499s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 4.050s; elapsed, 2.504s, #calls: 1', 'TOTAL : CPU, 4.050s; elapsed, 2.504s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 3.430s; elapsed, 2.503s, #calls: 1', 'TOTAL : CPU, 3.430s; elapsed, 2.503s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 3.350s; elapsed, 2.501s, #calls: 1', 'TOTAL : CPU, 3.350s; elapsed, 2.501s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.140s; elapsed, 2.503s, #calls: 1', 'TOTAL : CPU, 3.140s; elapsed, 2.503s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.910s; elapsed, 2.503s, #calls: 1', 'TOTAL : CPU, 4.910s; elapsed, 2.503s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.190s; elapsed, 2.504s, #calls: 1', 'TOTAL : CPU, 3.190s; elapsed, 2.504s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.380s; elapsed, 3.692s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.053s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.740s; elapsed, 2.572s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 5.010s; elapsed, 2.508s, #calls: 1', 'TOTAL : CPU, 15.210s; elapsed, 8.825s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 2663.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 11.610s; elapsed, 2.911s', 'individual call time for EIGEN_LDLT: CPU, 3.150s; elapsed, 2.638s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.053s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,945', '# - number of levels = 126', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.202794', '# - matching time = 0.0369658', '# - symmetrization time = 0.0166161', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,945', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0442638', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.269 MB', '# - factor time = 2.17352', '# - factor nonzeros = 6,158,621', '# - factor memory = 49.269 MB', '# - total flops = 7.49293e+09, min = 699330, max = 2.95246e+09', '# - flop rate = 3.44737 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.05389e+09', '# --------------------------------------------', '# total = 8.05389e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.55917e-11\trel.res = 1.09543e-15\tbw.error = 2.50926e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0405071', '# - total flops = 2.07556e+07, min = 27135, max = 6.90876e+06', '# - flop rate = 0.512394 GFlop/s', '# - bytes moved = 93.8099 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.31589 GByte/s', '# - solve arithmetic intensity = 0.221252 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.570s; elapsed, 2.535s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 5.270s; elapsed, 2.532s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 5.450s; elapsed, 2.537sindividual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 8.080s; elapsed, 2.530s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 7.050s; elapsed, 2.537s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.480s; elapsed, 2.536s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.790s; elapsed, 2.535s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.950s; elapsed, 2.541s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 5.270s; elapsed, 2.532s, #calls: 1', 'TOTAL : CPU, 5.270s; elapsed, 2.532s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.570s; elapsed, 2.535s, #calls: 1', 'TOTAL : CPU, 4.570s; elapsed, 2.535s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 5.450s; elapsed, 2.537s, #calls: 1', 'TOTAL : CPU, 5.450s; elapsed, 2.537s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 8.080s; elapsed, 2.530s, #calls: 1', 'TOTAL : CPU, 8.080s; elapsed, 2.530s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.480s; elapsed, 2.536s, #calls: 1', 'TOTAL : CPU, 4.480s; elapsed, 2.536s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.790s; elapsed, 2.535s, #calls: 1', 'TOTAL : CPU, 9.790s; elapsed, 2.535s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 7.050s; elapsed, 2.537s, #calls: 1', 'TOTAL : CPU, 7.050s; elapsed, 2.537s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 11.610s; elapsed, 2.911s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.053s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.150s; elapsed, 2.638s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.950s; elapsed, 2.541s, #calls: 1', 'TOTAL : CPU, 24.820s; elapsed, 8.143s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 1331.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.950s; elapsed, 5.950s', 'individual call time for EIGEN_LDLT: CPU, 2.530s; elapsed, 2.529s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.054s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,953', '# - number of levels = 125', '# - nd time = 0.176164', '# - matching time = 0.0321488', '# - symmetrization time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.016413', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,953', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0375569', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.1334 MB', '# - factor time = 2.02451', '# - factor nonzeros = 6,141,669', '# - factor memory = 49.1334 MB', '# - total flops = 7.46606e+09, min = 582404, max = 2.95054e+09', '# - flop rate = 3.68784 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.98609e+09', '# --------------------------------------------', '# total = 7.98609e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.31462e-11\trel.res = 9.90751e-16\tbw.error = 2.15469e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0347619', '# - total flops = 3.13629e+07, min = 19616, max = 6.85515e+06', '# - flop rate = 0.902221 GFlop/s', '# - bytes moved = 90.1412 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.5931 GByte/s', '# - solve arithmetic intensity = 0.347931 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 2.290s; elapsed, 2.334s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.330s; elapsed, 2.335s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.280s; elapsed, 2.334s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 2.320s; elapsed, 2.333s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.310s; elapsed, 2.333s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 2.280s; elapsed, 2.335s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 2.310s; elapsed, 2.333s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.280s; elapsed, 2.334s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 2.250s; elapsed, 2.334s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 2.250s; elapsed, 2.330s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.320s; elapsed, 2.333s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.300s; elapsed, 2.334s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.300s; elapsed, 2.334s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 2.280s; elapsed, 2.333s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 2.290s; elapsed, 2.328s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.330s; elapsed, 2.340s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 2.280s; elapsed, 2.335s, #calls: 1', 'TOTAL : CPU, 2.280s; elapsed, 2.335s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.330s; elapsed, 2.335s, #calls: 1', 'TOTAL : CPU, 2.330s; elapsed, 2.335s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.310s; elapsed, 2.333s, #calls: 1', 'TOTAL : CPU, 2.310s; elapsed, 2.333s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 2.320s; elapsed, 2.333s, #calls: 1', 'TOTAL : CPU, 2.320s; elapsed, 2.333s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 2.290s; elapsed, 2.334s, #calls: 1', 'TOTAL : CPU, 2.290s; elapsed, 2.334s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.280s; elapsed, 2.334s, #calls: 1', 'TOTAL : CPU, 2.280s; elapsed, 2.334s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 2.310s; elapsed, 2.333s, #calls: 1', 'TOTAL : CPU, 2.310s; elapsed, 2.333s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.320s; elapsed, 2.333s, #calls: 1', 'TOTAL : CPU, 2.320s; elapsed, 2.333s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 2.280s; elapsed, 2.333s, #calls: 1', 'TOTAL : CPU, 2.280s; elapsed, 2.333s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.280s; elapsed, 2.334s, #calls: 1', 'TOTAL : CPU, 2.280s; elapsed, 2.334s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.300s; elapsed, 2.334s, #calls: 1', 'TOTAL : CPU, 2.300s; elapsed, 2.334s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 2.250s; elapsed, 2.334s, #calls: 1', 'TOTAL : CPU, 2.250s; elapsed, 2.334s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.300s; elapsed, 2.334s, #calls: 1', 'TOTAL : CPU, 2.300s; elapsed, 2.334s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 2.250s; elapsed, 2.330s, #calls: 1', 'TOTAL : CPU, 2.250s; elapsed, 2.330s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 2.290s; elapsed, 2.328s, #calls: 1', 'TOTAL : CPU, 2.290s; elapsed, 2.328s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.950s; elapsed, 5.950s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.054s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.530s; elapsed, 2.529s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.330s; elapsed, 2.340s, #calls: 1', 'TOTAL : CPU, 10.870s; elapsed, 10.873s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 1331.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.840s; elapsed, 3.934s', 'individual call time for EIGEN_LDLT: CPU, 5.270s; elapsed, 5.094s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.056s', 'CPP -2', '# Initializing STRUMPACK', '# using 2CPP -2', ' OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2CPP -2', 'CPP -2CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '', '', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,953', '# - number of levels = 125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.20229', '# - matching time = 0.0385211', '# - symmetrization time = 0.0163429', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,953', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.044646', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.1334 MB', '# - factor time = 3.59134', '# - factor nonzeros = 6,141,669', '# - factor memory = 49.1334 MB', '# - total flops = 7.46607e+09, min = 582404, max = 2.95054e+09', '# - flop rate = 2.07891 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.98609e+09', '# --------------------------------------------', '# total = 7.98609e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.31206e-11\trel.res = 9.89653e-16\tbw.error = 2.15469e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0408621', '# - total flops = 3.13629e+07, min = 19616, max = 6.85515e+06', '# - flop rate = 0.767531 GFlop/s', '# - bytes moved = 90.1488 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.20617 GByte/s', '# - solve arithmetic intensity = 0.347902 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.680s; elapsed, 3.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 4.860s; elapsed, 3.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 5.100s; elapsed, 3.957s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 6.690s; elapsed, 3.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 4.950s; elapsed, 3.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 4.840s; elapsed, 3.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.640s; elapsed, 3.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.610s; elapsed, 3.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 4.580s; elapsed, 3.955s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 5.750s; elapsed, 3.951s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 4.500s; elapsed, 3.955s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 5.550s; elapsed, 3.956s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 4.920s; elapsed, 3.955s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 5.150s; elapsed, 3.951s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 5.170s; elapsed, 3.956s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 7.840s; elapsed, 3.961s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.640s; elapsed, 3.954s, #calls: 1', 'TOTAL : CPU, 4.640s; elapsed, 3.954s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 6.690s; elapsed, 3.954s, #calls: 1', 'TOTAL : CPU, 6.690s; elapsed, 3.954s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 5.550s; elapsed, 3.956s, #calls: 1', 'TOTAL : CPU, 5.550s; elapsed, 3.956s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 5.100s; elapsed, 3.957s, #calls: 1', 'TOTAL : CPU, 5.100s; elapsed, 3.957s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 4.860s; elapsed, 3.954s, #calls: 1', 'TOTAL : CPU, 4.860s; elapsed, 3.954s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 4.580s; elapsed, 3.955s, #calls: 1', 'TOTAL : CPU, 4.580s; elapsed, 3.955s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 4.950s; elapsed, 3.954s, #calls: 1', 'TOTAL : CPU, 4.950s; elapsed, 3.954s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 5.750s; elapsed, 3.951s, #calls: 1', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.610s; elapsed, 3.954s, #calls: 1', 'TOTAL : CPU, 4.610s; elapsed, 3.954s', 'TOTAL : CPU, 5.750s; elapsed, 3.951s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.680s; elapsed, 3.954s, #calls: 1', 'TOTAL : CPU, 4.680s; elapsed, 3.954s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 4.500s; elapsed, 3.955s, #calls: 1', 'TOTAL : CPU, 4.500s; elapsed, 3.955s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 4.840s; elapsed, 3.954s, #calls: 1', 'TOTAL : CPU, 4.840s; elapsed, 3.954s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 5.150s; elapsed, 3.951s, #calls: 1', 'TOTAL : CPU, 5.150s; elapsed, 3.951s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 5.170s; elapsed, 3.956s, #calls: 1', 'TOTAL : CPU, 5.170s; elapsed, 3.956s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 4.920s; elapsed, 3.955s, #calls: 1', 'TOTAL : CPU, 4.920s; elapsed, 3.955s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.840s; elapsed, 3.934s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.056s, #calls: 1', 'time for EIGEN_LDLT: CPU, 5.270s; elapsed, 5.094s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 7.840s; elapsed, 3.961s, #calls: 1', 'TOTAL : CPU, 21.030s; elapsed, 13.045s']
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Bfactor-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Bfactor-/A_strum_5k_omp1_paramslevmar.parameter_flags=Bfactor-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Bfactor-/b_strum_5k_omp1_paramslevmar.parameter_flags=Bfactor-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 18.170s; elapsed, 18.179s', 'individual call time for EIGEN_LDLT: CPU, 18.950s; elapsed, 18.960s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.167s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 18.170s; elapsed, 18.179s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.167s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.950s; elapsed, 18.960s, #calls: 1', 'TOTAL : CPU, 37.280s; elapsed, 37.307s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 21.090s; elapsed, 10.559s', 'individual call time for EIGEN_LDLT: CPU, 18.850s; elapsed, 18.686s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.151s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 21.090s; elapsed, 10.559s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.151s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.850s; elapsed, 18.686s, #calls: 1', 'TOTAL : CPU, 40.190s; elapsed, 29.396s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 43.330s; elapsed, 10.849s', 'individual call time for EIGEN_LDLT: CPU, 20.710s; elapsed, 20.170s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.105s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 43.330s; elapsed, 10.849s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.105s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.710s; elapsed, 20.170s, #calls: 1', 'TOTAL : CPU, 64.310s; elapsed, 31.124s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 65.200s; elapsed, 8.179s', 'individual call time for EIGEN_LDLT: CPU, 20.040s; elapsed, 18.752s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.430s; elapsed, 0.094s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 65.200s; elapsed, 8.179s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.430s; elapsed, 0.094s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.040s; elapsed, 18.752s, #calls: 1', 'TOTAL : CPU, 85.670s; elapsed, 27.025s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 140.140s; elapsed, 8.804s', 'individual call time for EIGEN_LDLT: CPU, 21.870s; elapsed, 19.155s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.790s; elapsed, 0.095s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 140.140s; elapsed, 8.804s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.790s; elapsed, 0.095s, #calls: 1', 'time for EIGEN_LDLT: CPU, 21.870s; elapsed, 19.155s, #calls: 1', 'TOTAL : CPU, 162.800s; elapsed, 28.054s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 19.130s; elapsed, 19.160s', 'individual call time for EIGEN_LDLT: CPU, 18.960s; elapsed, 18.984s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.160s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,121', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.163049', '# - matching time = 0.052469', '# - symmetrization time = 0.0295382', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,121', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.069936', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 169.261 MB', '# - factor time = 10.8386', '# - factor nonzeros = 21,157,571', '# - factor memory = 169.261 MB', '# - total flops = 5.06936e+10, min = 2.21744e+10, max = 2.85192e+10', '# - flop rate = 4.67714 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.21012e+10', '# --------------------------------------------', '# total = 5.21012e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.85836e-11\trel.res = 1.98671e-15\tbw.error = 5.28691e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0650389', '# - total flops = 4.62998e+07, min = 1.70328e+07, max = 2.9267e+07', '# - flop rate = 0.711879 GFlop/s', '# - bytes moved = 199.391 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.06573 GByte/s', '# - solve arithmetic intensity = 0.232206 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.230s; elapsed, 11.233s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 11.240s; elapsed, 11.251s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.230s; elapsed, 11.233s, #calls: 1', 'TOTAL : CPU, 11.230s; elapsed, 11.233s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 19.130s; elapsed, 19.160s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.160s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.960s; elapsed, 18.984s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 11.240s; elapsed, 11.251s, #calls: 1', 'TOTAL : CPU, 49.480s; elapsed, 49.556s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 21.130s; elapsed, 10.588s', 'individual call time for EIGEN_LDLT: CPU, 19.020s; elapsed, 18.857s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.131s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,121', '# - number of levels = 68', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.161264', '# - matching time = 0.0531778', '# - symmetrization time = 0.0313871', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,121', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0867958', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 169.261 MB', '# - factor time = 8.11767', '# - factor nonzeros = 21,157,571', '# - factor memory = 169.261 MB', '# - total flops = 5.06937e+10, min = 2.21745e+10, max = 2.85192e+10', '# - flop rate = 6.24486 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.21012e+10', '# --------------------------------------------', '# total = 5.21012e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.89278e-11\trel.res = 2.00078e-15\tbw.error = 5.28761e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0683291', '# - total flops = 4.631e+07, min = 1.7043e+07, max = 2.9267e+07', '# - flop rate = 0.677749 GFlop/s', '# - bytes moved = 199.824 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.92444 GByte/s', '# - solve arithmetic intensity = 0.231754 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 15.050s; elapsed, 8.537s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.020s; elapsed, 8.556s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 15.050s; elapsed, 8.537s, #calls: 1', 'TOTAL : CPU, 15.050s; elapsed, 8.537s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 21.130s; elapsed, 10.588s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.131s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.020s; elapsed, 18.857s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.020s; elapsed, 8.556s, #calls: 1', 'TOTAL : CPU, 57.400s; elapsed, 38.132s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 41.850s; elapsed, 10.474s', 'individual call time for EIGEN_LDLT: CPU, 51.090s; elapsed, 50.552s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.330s; elapsed, 0.120s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,121', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.170684', '# - matching time = 0.054136', '# - symmetrization time = 0.0304191', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,121', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.080976', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 169.261 MB', '# - factor time = 7.24615', '# - factor nonzeros = 21,157,571', '# - factor memory = 169.261 MB', '# - total flops = 5.06937e+10, min = 2.21745e+10, max = 2.85192e+10', '# - flop rate = 6.99595 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.21012e+10', '# --------------------------------------------', '# total = 5.21012e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.88728e-11\trel.res = 1.99853e-15\tbw.error = 5.28761e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0733261', '# - total flops = 4.631e+07, min = 1.7043e+07, max = 2.9267e+07', '# - flop rate = 0.631562 GFlop/s', '# - bytes moved = 199.843 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.7254 GByte/s', '# - solve arithmetic intensity = 0.231732 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 25.400s; elapsed, 7.673s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 30.550s; elapsed, 7.690s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 25.400s; elapsed, 7.673s, #calls: 1', 'TOTAL : CPU, 25.400s; elapsed, 7.673s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 41.850s; elapsed, 10.474s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.330s; elapsed, 0.120s, #calls: 1', 'time for EIGEN_LDLT: CPU, 51.090s; elapsed, 50.552s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 30.550s; elapsed, 7.690s, #calls: 1', 'TOTAL : CPU, 123.820s; elapsed, 68.837s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 64.410s; elapsed, 8.073s', 'individual call time for EIGEN_LDLT: CPU, 20.740s; elapsed, 19.487s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.430s; elapsed, 0.110s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = CPP -2', '6 = log_2(#threads) + 3', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,121', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.17955', '# - matching time = 0.0581422', '# - symmetrization time = 0.0359781', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,121', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0912349', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 169.261 MB', '# - factor time = 8.43808', '# - factor nonzeros = 21,157,571', '# - factor memory = 169.261 MB', '# - total flops = 5.06937e+10, min = 2.21745e+10, max = 2.85192e+10', '# - flop rate = 6.00773 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.21012e+10', '# --------------------------------------------', '# total = 5.21012e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.88477e-11\trel.res = 1.99751e-15\tbw.error = 5.28761e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0835271', '# - total flops = 4.631e+07, min = 1.7043e+07, max = 2.9267e+07', '# - flop rate = 0.554431 GFlop/s', '# - bytes moved = 199.881 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.39301 GByte/s', '# - solve arithmetic intensity = 0.231688 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 54.290s; elapsed, 8.911s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 70.590s; elapsed, 8.926s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 54.290s; elapsed, 8.911s, #calls: 1', 'TOTAL : CPU, 54.290s; elapsed, 8.911s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 64.410s; elapsed, 8.073s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.430s; elapsed, 0.110s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.740s; elapsed, 19.487s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 70.590s; elapsed, 8.926s, #calls: 1', 'TOTAL : CPU, 156.170s; elapsed, 36.596s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 139.560s; elapsed, 8.753s', 'individual call time for EIGEN_LDLT: CPU, 37.630s; elapsed, 34.892s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.680s; elapsed, 0.093s', 'CPP -2', 'CPP -2# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', '', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,121', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.205065', '# - matching time = 0.104195', '# - symmetrization time = 0.04724', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,121', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.097409', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 169.261 MB', '# - factor time = 8.62634', '# - factor nonzeros = 21,157,571', '# - factor memory = 169.261 MB', '# - total flops = 5.06943e+10, min = 2.21751e+10, max = 2.85192e+10', '# - flop rate = 5.87669 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.21012e+10', '# --------------------------------------------', '# total = 5.21012e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.88318e-11\trel.res = 1.99685e-15\tbw.error = 5.28757e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.10304', '# - total flops = 4.63137e+07, min = 1.70467e+07, max = 2.9267e+07', '# - flop rate = 0.449474 GFlop/s', '# - bytes moved = 200.018 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.94117 GByte/s', '# - solve arithmetic intensity = 0.231548 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 109.000s; elapsed, 9.209s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 144.330s; elapsed, 9.229s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 109.000s; elapsed, 9.209s, #calls: 1', 'TOTAL : CPU, 109.000s; elapsed, 9.209s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 139.560s; elapsed, 8.753s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.680s; elapsed, 0.093s, #calls: 1', 'time for EIGEN_LDLT: CPU, 37.630s; elapsed, 34.892s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 144.330s; elapsed, 9.229s, #calls: 1', 'TOTAL : CPU, 322.200s; elapsed, 52.968s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 19.630s; elapsed, 19.640s', 'individual call time for EIGEN_LDLT: CPU, 19.440s; elapsed, 19.452s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.185s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.168702', '# - matching time = 0.0535901', '# - symmetrization time = 0.0412691', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0673561', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 170.719 MB', '# - factor time = 7.78362', '# - factor nonzeros = 21,339,853', '# - factor memory = 170.719 MB', '# - total flops = 5.15107e+10, min = 1.48212e+07, max = 2.85626e+10', '# - flop rate = 6.61783 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.2862e+10', '# --------------------------------------------', '# total = 5.2862e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.69603e-11\trel.res = 1.5114e-15\tbw.error = 4.67637e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.079797', '# - total flops = 5.12613e+07, min = 105197, max = 2.92123e+07', '# - flop rate = 0.642397 GFlop/s', '# - bytes moved = 180.23 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.2586 GByte/s', '# - solve arithmetic intensity = 0.284422 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.220s; elapsed, 8.218s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.210s; elapsed, 8.217s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.190s; elapsed, 8.203s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.220s; elapsed, 8.222s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.220s; elapsed, 8.218s, #calls: 1', 'TOTAL : CPU, 8.220s; elapsed, 8.218s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.210s; elapsed, 8.217s, #calls: 1', 'TOTAL : CPU, 8.210s; elapsed, 8.217s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.190s; elapsed, 8.203s, #calls: 1', 'TOTAL : CPU, 8.190s; elapsed, 8.203s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 19.630s; elapsed, 19.640s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.185s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.440s; elapsed, 19.452s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.220s; elapsed, 8.222s, #calls: 1', 'TOTAL : CPU, 47.470s; elapsed, 47.498s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 22.560s; elapsed, 11.301s', 'individual call time for EIGEN_LDLT: CPU, 19.320s; elapsed, 19.146s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.150s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.177807', '# - matching time = 0.056633', '# - symmetrization time = 0.041754', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0772901', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 170.719 MB', '# - factor time = 7.72084', '# - factor nonzeros = 21,339,853', '# - factor memory = 170.719 MB', '# - total flops = 5.15107e+10, min = 1.48212e+07, max = 2.85626e+10', '# - flop rate = 6.67165 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.2862e+10', '# --------------------------------------------', '# total = 5.2862e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.69771e-11\trel.res = 1.51208e-15\tbw.error = 4.67141e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.08533', '# - total flops = 5.12668e+07, min = 105197, max = 2.92123e+07', '# - flop rate = 0.600807 GFlop/s', '# - bytes moved = 180.446 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.11468 GByte/s', '# - solve arithmetic intensity = 0.284112 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.540s; elapsed, 8.183s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 14.060s; elapsed, 8.182s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 12.650s; elapsed, 8.170s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.110s; elapsed, 8.188s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.540s; elapsed, 8.183s, #calls: 1', 'TOTAL : CPU, 9.540s; elapsed, 8.183s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 12.650s; elapsed, 8.170s, #calls: 1', 'TOTAL : CPU, 12.650s; elapsed, 8.170s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 14.060s; elapsed, 8.182s, #calls: 1', 'TOTAL : CPU, 14.060s; elapsed, 8.182s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 22.560s; elapsed, 11.301s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.150s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.320s; elapsed, 19.146s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.110s; elapsed, 8.188s, #calls: 1', 'TOTAL : CPU, 58.240s; elapsed, 38.785s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 34.180s; elapsed, 8.582s', 'individual call time for EIGEN_LDLT: CPU, 31.470s; elapsed, 30.940s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.320s; elapsed, 0.121s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.202503', '# - matching time = 0.0815771', '# - symmetrization time = 0.04374', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0870862', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 170.719 MB', '# - factor time = 7.7136', '# - factor nonzeros = 21,339,853', '# - factor memory = 170.719 MB', '# - total flops = 5.15107e+10, min = 1.48226e+07, max = 2.85626e+10', '# - flop rate = 6.67791 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.2862e+10', '# --------------------------------------------', '# total = 5.2862e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.69687e-11\trel.res = 1.51174e-15\tbw.error = 4.67141e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0909681', '# - total flops = 5.12668e+07, min = 105197, max = 2.92123e+07', '# - flop rate = 0.563569 GFlop/s', '# - bytes moved = 180.456 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.98373 GByte/s', '# - solve arithmetic intensity = 0.284096 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 12.260s; elapsed, 8.247s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 26.040s; elapsed, 8.247s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 21.320s; elapsed, 8.234s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 32.300s; elapsed, 8.247s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 12.260s; elapsed, 8.247s, #calls: 1', 'TOTAL : CPU, 12.260s; elapsed, 8.247s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 21.320s; elapsed, 8.234s, #calls: 1', 'TOTAL : CPU, 21.320s; elapsed, 8.234s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 26.040s; elapsed, 8.247s, #calls: 1', 'TOTAL : CPU, 26.040s; elapsed, 8.247s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 34.180s; elapsed, 8.582s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.320s; elapsed, 0.121s, #calls: 1', 'time for EIGEN_LDLT: CPU, 31.470s; elapsed, 30.940s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 32.300s; elapsed, 8.247s, #calls: 1', 'TOTAL : CPU, 98.270s; elapsed, 47.891s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 69.120s; elapsed, 8.667s', 'individual call time for EIGEN_LDLT: CPU, 20.650s; elapsed, 19.369s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.420s; elapsed, 0.103s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.191953', '# - matching time = 0.0693421', '# - symmetrization time = 0.0466621', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.097029', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 170.719 MB', '# - factor time = 8.21027', '# - factor nonzeros = 21,339,853', '# - factor memory = 170.719 MB', '# - total flops = 5.15107e+10, min = 1.48228e+07, max = 2.85626e+10', '# - flop rate = 6.27394 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.2862e+10', '# --------------------------------------------', '# total = 5.2862e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.694e-11\trel.res = 1.51057e-15\tbw.error = 4.67141e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.107361', '# - total flops = 5.12668e+07, min = 105197, max = 2.92123e+07', '# - flop rate = 0.477518 GFlop/s', '# - bytes moved = 180.478 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.68104 GByte/s', '# - solve arithmetic intensity = 0.284062 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 49.340s; elapsed, 8.750s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 35.760s; elapsed, 8.733s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.810s; elapsed, 8.749s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 68.220s; elapsed, 8.756s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 49.340s; elapsed, 8.750s, #calls: 1', 'TOTAL : CPU, 49.340s; elapsed, 8.750s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.810s; elapsed, 8.749s, #calls: 1', 'TOTAL : CPU, 17.810s; elapsed, 8.749s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 35.760s; elapsed, 8.733s, #calls: 1', 'TOTAL : CPU, 35.760s; elapsed, 8.733s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 69.120s; elapsed, 8.667s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.420s; elapsed, 0.103s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.650s; elapsed, 19.369s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 68.220s; elapsed, 8.756s, #calls: 1', 'TOTAL : CPU, 158.410s; elapsed, 36.895s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 2413.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 20.180s; elapsed, 20.203s', 'individual call time for EIGEN_LDLT: CPU, 20.090s; elapsed, 20.102s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.200s; elapsed, 0.192s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,143', '# - number of levels = 66', '# - nd time = 0.19664', '# - matching time = 0.0616531', '# - symmetrization time = 0.029217', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,143', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0707021', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 168.522 MB', '# - factor time = 8.70793', '# - factor nonzeros = 21,065,199', '# - factor memory = 168.522 MB', '# - total flops = 5.03174e+10, min = 1.50222e+07, max = 2.85789e+10', '# - flop rate = 5.77834 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.15536e+10', '# --------------------------------------------', '# total = 5.15536e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.42119e-11\trel.res = 1.80794e-15\tbw.error = 4.5859e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0707011', '# - total flops = 6.23363e+07, min = 92713, max = 2.90586e+07', '# - flop rate = 0.881688 GFlop/s', '# - bytes moved = 166.966 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.36157 GByte/s', '# - solve arithmetic intensity = 0.373348 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.120s; elapsed, 9.153s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 9.120s; elapsed, 9.153s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.150s; elapsed, 9.156s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.130s; elapsed, 9.153s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 9.120s; elapsed, 9.143s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 9.140s; elapsed, 9.155s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 9.130s; elapsed, 9.149s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.150s; elapsed, 9.159s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.130s; elapsed, 9.153s, #calls: 1', 'TOTAL : CPU, 9.130s; elapsed, 9.153s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 9.120s; elapsed, 9.143s, #calls: 1', 'TOTAL : CPU, 9.120s; elapsed, 9.143s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.120s; elapsed, 9.153s, #calls: 1', 'TOTAL : CPU, 9.120s; elapsed, 9.153s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 9.120s; elapsed, 9.153s, #calls: 1', 'TOTAL : CPU, 9.120s; elapsed, 9.153s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 9.130s; elapsed, 9.149s, #calls: 1', 'TOTAL : CPU, 9.130s; elapsed, 9.149s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 9.140s; elapsed, 9.155s, #calls: 1', 'TOTAL : CPU, 9.140s; elapsed, 9.155s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.150s; elapsed, 9.156s, #calls: 1', 'TOTAL : CPU, 9.150s; elapsed, 9.156s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 20.180s; elapsed, 20.203s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.200s; elapsed, 0.192s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.090s; elapsed, 20.102s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.150s; elapsed, 9.159s, #calls: 1', 'TOTAL : CPU, 49.620s; elapsed, 49.657s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 2413.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 22.470s; elapsed, 11.260s', 'individual call time for EIGEN_LDLT: CPU, 20.050s; elapsed, 19.876s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.129s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,143', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.177551', '# - matching time = 0.0605941', '# - symmetrization time = 0.0263829', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,143', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0786829', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 168.522 MB', '# - factor time = 7.7616', '# - factor nonzeros = 21,065,199', '# - factor memory = 168.522 MB', '# - total flops = 5.03174e+10, min = 1.50222e+07, max = 2.85789e+10', '# - flop rate = 6.48287 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.15536e+10', '# --------------------------------------------', '# total = 5.15536e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.40836e-11\trel.res = 1.80269e-15\tbw.error = 4.58556e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0617771', '# - total flops = 6.23363e+07, min = 92713, max = 2.90586e+07', '# - flop rate = 1.00905 GFlop/s', '# - bytes moved = 166.973 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.70282 GByte/s', '# - solve arithmetic intensity = 0.373333 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.640s; elapsed, 8.185s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 9.880s; elapsed, 8.183s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.360s; elapsed, 8.184s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.420s; elapsed, 8.184s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 13.660s; elapsed, 8.183s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 12.010s; elapsed, 8.179s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 11.690s; elapsed, 8.175s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.130s; elapsed, 8.188s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.640s; elapsed, 8.185s, #calls: 1', 'TOTAL : CPU, 9.640s; elapsed, 8.185s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 9.880s; elapsed, 8.183s, #calls: 1', 'TOTAL : CPU, 9.880s; elapsed, 8.183s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.420s; elapsed, 8.184s, #calls: 1', 'TOTAL : CPU, 9.420s; elapsed, 8.184s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.360s; elapsed, 8.184s, #calls: 1', 'TOTAL : CPU, 9.360s; elapsed, 8.184s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 12.010s; elapsed, 8.179s, #calls: 1', 'TOTAL : CPU, 12.010s; elapsed, 8.179s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 13.660s; elapsed, 8.183s, #calls: 1', 'TOTAL : CPU, 13.660s; elapsed, 8.183s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 11.690s; elapsed, 8.175s, #calls: 1', 'TOTAL : CPU, 11.690s; elapsed, 8.175s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 22.470s; elapsed, 11.260s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.129s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.050s; elapsed, 19.876s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.130s; elapsed, 8.188s, #calls: 1', 'TOTAL : CPU, 58.860s; elapsed, 39.453s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 2413.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 31.790s; elapsed, 7.969s', 'individual call time for EIGEN_LDLT: CPU, 20.310s; elapsed, 19.784s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.340s; elapsed, 0.117s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,143', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.208347', '# - matching time = 0.065809', '# - symmetrization time = 0.0293422', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,143', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0798979', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 168.522 MB', '# - factor time = 7.94199', '# - factor nonzeros = 21,065,199', '# - factor memory = 168.522 MB', '# - total flops = 5.03174e+10, min = 1.50234e+07, max = 2.85789e+10', '# - flop rate = 6.33562 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.15536e+10', '# --------------------------------------------', '# total = 5.15536e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.40619e-11\trel.res = 1.8018e-15\tbw.error = 4.58422e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0710061', '# - total flops = 6.23363e+07, min = 92713, max = 2.90586e+07', '# - flop rate = 0.877902 GFlop/s', '# - bytes moved = 166.981 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.35165 GByte/s', '# - solve arithmetic intensity = 0.373313 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 20.540s; elapsed, 8.409s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.540s; elapsed, 8.407s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 13.020s; elapsed, 8.415s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 12.400s; elapsed, 8.415s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 12.110s; elapsed, 8.414s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 13.590s; elapsed, 8.415s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 25.060s; elapsed, 8.415s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 32.960s; elapsed, 8.420s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 20.540s; elapsed, 8.409s, #calls: 1', 'TOTAL : CPU, 20.540s; elapsed, 8.409s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.540s; elapsed, 8.407s, #calls: 1', 'TOTAL : CPU, 17.540s; elapsed, 8.407s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 13.590s; elapsed, 8.415s, #calls: 1', 'TOTAL : CPU, 13.590s; elapsed, 8.415s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 12.110s; elapsed, 8.414s, #calls: 1', 'TOTAL : CPU, 12.110s; elapsed, 8.414s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 13.020s; elapsed, 8.415s, #calls: 1', 'TOTAL : CPU, 13.020s; elapsed, 8.415s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 25.060s; elapsed, 8.415s, #calls: 1', 'TOTAL : CPU, 25.060s; elapsed, 8.415s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 12.400s; elapsed, 8.415s, #calls: 1', 'TOTAL : CPU, 12.400s; elapsed, 8.415s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 31.790s; elapsed, 7.969s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.340s; elapsed, 0.117s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.310s; elapsed, 19.784s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 32.960s; elapsed, 8.420s, #calls: 1', 'TOTAL : CPU, 85.400s; elapsed, 36.290s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 1206.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 19.490s; elapsed, 19.505s', 'individual call time for EIGEN_LDLT: CPU, 19.820s; elapsed, 19.835s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.185s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,143', '# - number of levels = 65', '# - nd time = 0.174265', '# - matching time = 0.0576799', '# - symmetrization time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.0151649', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,143', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0594389', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 168.522 MB', '# - factor time = 7.76313', '# - factor nonzeros = 21,065,199', '# - factor memory = 168.522 MB', '# - total flops = 5.03174e+10, min = 1.38671e+07, max = 2.85337e+10', '# - flop rate = 6.48159 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.13241e+10', '# --------------------------------------------', '# total = 5.13241e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.22003e-11\trel.res = 1.31675e-15\tbw.error = 3.37627e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.043978', '# - total flops = 9.15661e+07, min = 86678, max = 2.87842e+07', '# - flop rate = 2.08209 GFlop/s', '# - bytes moved = 158.412 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.60207 GByte/s', '# - solve arithmetic intensity = 0.578027 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.070s; elapsed, 8.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 8.090s; elapsed, 8.122s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.110s; elapsed, 8.126s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 8.070s; elapsed, 8.126s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 8.080s; elapsed, 8.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 8.100s; elapsed, 8.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 8.120s; elapsed, 8.126s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 8.110s; elapsed, 8.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 8.110s; elapsed, 8.126s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.080s; elapsed, 8.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 8.100s; elapsed, 8.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 8.070s; elapsed, 8.131s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 8.070s; elapsed, 8.129s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 8.070s; elapsed, 8.128s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 8.090s; elapsed, 8.126s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.140s; elapsed, 8.136s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 8.070s; elapsed, 8.128s, #calls: 1', 'TOTAL : CPU, 8.070s; elapsed, 8.128s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.070s; elapsed, 8.127s, #calls: 1', 'TOTAL : CPU, 8.070s; elapsed, 8.127s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 8.090s; elapsed, 8.122s, #calls: 1', 'TOTAL : CPU, 8.090s; elapsed, 8.122s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 8.070s; elapsed, 8.126s, #calls: 1', 'TOTAL : CPU, 8.070s; elapsed, 8.126s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 8.100s; elapsed, 8.127s, #calls: 1', 'TOTAL : CPU, 8.100s; elapsed, 8.127s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 8.080s; elapsed, 8.127s, #calls: 1', 'Exiting profiler', 'TOTAL : CPU, 8.080s; elapsed, 8.127s', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 8.110s; elapsed, 8.127s, #calls: 1', 'TOTAL : CPU, 8.110s; elapsed, 8.127s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 8.100s; elapsed, 8.127s, #calls: 1', 'TOTAL : CPU, 8.100s; elapsed, 8.127s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 8.110s; elapsed, 8.126s, #calls: 1', 'TOTAL : CPU, 8.110s; elapsed, 8.126s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.080s; elapsed, 8.127s, #calls: 1', 'TOTAL : CPU, 8.080s; elapsed, 8.127s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.110s; elapsed, 8.126s, #calls: 1', 'TOTAL : CPU, 8.110s; elapsed, 8.126s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 8.120s; elapsed, 8.126s, #calls: 1', 'TOTAL : CPU, 8.120s; elapsed, 8.126s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 8.070s; elapsed, 8.131s, #calls: 1', 'TOTAL : CPU, 8.070s; elapsed, 8.131s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 8.090s; elapsed, 8.126s, #calls: 1', 'TOTAL : CPU, 8.090s; elapsed, 8.126s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 8.070s; elapsed, 8.129s, #calls: 1', 'TOTAL : CPU, 8.070s; elapsed, 8.129s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 19.490s; elapsed, 19.505s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.185s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.820s; elapsed, 19.835s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.140s; elapsed, 8.136s, #calls: 1', 'TOTAL : CPU, 47.630s; elapsed, 47.662s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 1206.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 23.220s; elapsed, 11.636s', 'individual call time for EIGEN_LDLT: CPU, 20.790s; elapsed, 20.616s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.164s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,143', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.203531', '# - matching time = 0.067708', '# - symmetrization time = 0.015831', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,143', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0727611', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 168.522 MB', '# - factor time = 7.91772', '# - factor nonzeros = 21,065,199', '# - factor memory = 168.522 MB', '# - total flops = 5.03174e+10, min = 1.38671e+07, max = 2.85337e+10', '# - flop rate = 6.35504 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.13241e+10', '# --------------------------------------------', '# total = 5.13241e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.21233e-11\trel.res = 1.3136e-15\tbw.error = 3.37627e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0509582', '# - total flops = 9.15661e+07, min = 86678, max = 2.87842e+07', '# - flop rate = 1.79689 GFlop/s', '# - bytes moved = 158.417 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.10877 GByte/s', '# - solve arithmetic intensity = 0.578006 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 9.450s; elapsed, 8.343s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.940s; elapsed, 8.344s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 14.340s; elapsed, 8.342s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 10.040s; elapsed, 8.342s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 9.450s; elapsed, 8.342s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 10.270s; elapsed, 8.344s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 9.870s; elapsed, 8.343s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 9.790s; elapsed, 8.343s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 12.290s; elapsed, 8.343s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 10.220s; elapsed, 8.340s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 11.120s; elapsed, 8.341s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 11.580s; elapsed, 8.339s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 9.400s; elapsed, 8.344s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.770s; elapsed, 8.342s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.630s; elapsed, 8.343s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.400s; elapsed, 8.349s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 9.870s; elapsed, 8.343s, #calls: 1', 'TOTAL : CPU, 9.870s; elapsed, 8.343s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 9.450s; elapsed, 8.343s, #calls: 1', 'TOTAL : CPU, 9.450s; elapsed, 8.343s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.940s; elapsed, 8.344s, #calls: 1', 'TOTAL : CPU, 9.940s; elapsed, 8.344s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 12.290s; elapsed, 8.343s, #calls: 1', 'TOTAL : CPU, 12.290s; elapsed, 8.343s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 10.220s; elapsed, 8.340s, #calls: 1', 'TOTAL : CPU, 10.220s; elapsed, 8.340s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 14.340s; elapsed, 8.342s, #calls: 1', 'TOTAL : CPU, 14.340s; elapsed, 8.342s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 11.580s; elapsed, 8.339s, #calls: 1', 'TOTAL : CPU, 11.580s; elapsed, 8.339s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 11.120s; elapsed, 8.341s, #calls: 1', 'TOTAL : CPU, 11.120s; elapsed, 8.341s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 9.450s; elapsed, 8.342s, #calls: 1', 'TOTAL : CPU, 9.450s; elapsed, 8.342s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 9.790s; elapsed, 8.343s, #calls: 1', 'TOTAL : CPU, 9.790s; elapsed, 8.343s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.770s; elapsed, 8.342s, #calls: 1', 'TOTAL : CPU, 9.770s; elapsed, 8.342s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 9.400s; elapsed, 8.344s, #calls: 1', 'TOTAL : CPU, 9.400s; elapsed, 8.344s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 10.040s; elapsed, 8.342s, #calls: 1', 'TOTAL : CPU, 10.040s; elapsed, 8.342s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.630s; elapsed, 8.343s, #calls: 1', 'TOTAL : CPU, 9.630s; elapsed, 8.343s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 10.270s; elapsed, 8.344s, #calls: 1', 'TOTAL : CPU, 10.270s; elapsed, 8.344s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 23.220s; elapsed, 11.636s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.164s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.790s; elapsed, 20.616s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.400s; elapsed, 8.349s, #calls: 1', 'TOTAL : CPU, 60.690s; elapsed, 40.765s']
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Deff-/A_strum_5k_omp1_paramslevmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Deff-/b_strum_5k_omp1_paramslevmar.parameter_flags=Deff-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 4.280s; elapsed, 4.277s', 'individual call time for EIGEN_LDLT: CPU, 2.510s; elapsed, 2.506s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.051s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 4.280s; elapsed, 4.277s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.051s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.510s; elapsed, 2.506s, #calls: 1', 'TOTAL : CPU, 6.840s; elapsed, 6.834s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.430s; elapsed, 2.735s', 'individual call time for EIGEN_LDLT: CPU, 2.700s; elapsed, 2.509s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.044s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.430s; elapsed, 2.735s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.044s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.700s; elapsed, 2.509s, #calls: 1', 'TOTAL : CPU, 8.200s; elapsed, 5.288s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.430s; elapsed, 1.892s', 'individual call time for EIGEN_LDLT: CPU, 3.620s; elapsed, 3.076s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.043s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.430s; elapsed, 1.892s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.043s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.620s; elapsed, 3.076s, #calls: 1', 'TOTAL : CPU, 11.160s; elapsed, 5.011s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 14.060s; elapsed, 1.797s', 'individual call time for EIGEN_LDLT: CPU, 4.550s; elapsed, 3.299s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.038s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 14.060s; elapsed, 1.797s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.038s, #calls: 1', 'time for EIGEN_LDLT: CPU, 4.550s; elapsed, 3.299s, #calls: 1', 'TOTAL : CPU, 18.750s; elapsed, 5.133s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 36.060s; elapsed, 2.315s', 'individual call time for EIGEN_LDLT: CPU, 5.510s; elapsed, 2.809s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.039s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 36.060s; elapsed, 2.315s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.039s, #calls: 1', 'time for EIGEN_LDLT: CPU, 5.510s; elapsed, 2.809s, #calls: 1', 'TOTAL : CPU, 41.850s; elapsed, 5.164s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 4.390s; elapsed, 4.397s', 'individual call time for EIGEN_LDLT: CPU, 2.610s; elapsed, 2.604s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.053s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processesCPP -2', '', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,135', '# - number of levels = 77', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.145689', '# - matching time = 0.0309079', '# - symmetrization time = 0.0150032', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,135', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0415602', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.9326 MB', '# - factor time = 2.91968', '# - factor nonzeros = 6,491,577', '# - factor memory = 51.9326 MB', '# - total flops = 7.45429e+09, min = 3.29924e+09, max = 4.15505e+09', '# - flop rate = 2.55312 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.81378e+09', '# --------------------------------------------', '# total = 7.81378e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.4489e-11\trel.res = 1.47627e-15\tbw.error = 3.21475e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.039288', '# - total flops = 1.51891e+07, min = 7.43097e+06, max = 7.75814e+06', '# - flop rate = 0.386609 GFlop/s', '# - bytes moved = 97.4472 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.48033 GByte/s', '# - solve arithmetic intensity = 0.15587 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.190s; elapsed, 3.207s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.210s; elapsed, 3.219s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.190s; elapsed, 3.207s, #calls: 1', 'TOTAL : CPU, 3.190s; elapsed, 3.207s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 4.390s; elapsed, 4.397s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.053s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.610s; elapsed, 2.604s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.210s; elapsed, 3.219s, #calls: 1', 'TOTAL : CPU, 10.260s; elapsed, 10.272s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.820s; elapsed, 2.937s', 'individual call time for EIGEN_LDLT: CPU, 2.690s; elapsed, 2.493s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.046s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,135', '# - number of levels = 77', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.149949', '# - matching time = 0.0315361', '# - symmetrization time = 0.014894', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,135', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.047554', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.9326 MB', '# - factor time = 1.71412', '# - factor nonzeros = 6,491,577', '# - factor memory = 51.9326 MB', '# - total flops = 7.45434e+09, min = 3.29924e+09, max = 4.1551e+09', '# - flop rate = 4.34879 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.81378e+09', '# --------------------------------------------', '# total = 7.81378e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.44115e-11\trel.res = 1.47295e-15\tbw.error = 3.21352e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.032227', '# - total flops = 1.51917e+07, min = 7.43353e+06, max = 7.75814e+06', '# - flop rate = 0.471395 GFlop/s', '# - bytes moved = 97.5548 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.02711 GByte/s', '# - solve arithmetic intensity = 0.155725 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.010s; elapsed, 2.009s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.040s; elapsed, 2.019s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.010s; elapsed, 2.009s, #calls: 1', 'TOTAL : CPU, 4.010s; elapsed, 2.009s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.820s; elapsed, 2.937s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.046s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.690s; elapsed, 2.493s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.040s; elapsed, 2.019s, #calls: 1', 'TOTAL : CPU, 12.620s; elapsed, 7.495s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.190s; elapsed, 1.830s', 'individual call time for EIGEN_LDLT: CPU, 3.110s; elapsed, 2.573s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.038s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,135', '# - number of levels = 77', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.145875', '# - matching time = 0.0327501', '# - symmetrization time = 0.016371', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,135', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.04703', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.9326 MB', '# - factor time = 1.6067', '# - factor nonzeros = 6,491,577', '# - factor memory = 51.9326 MB', '# - total flops = 7.45434e+09, min = 3.29924e+09, max = 4.1551e+09', '# - flop rate = 4.63952 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.81378e+09', '# --------------------------------------------', '# total = 7.81378e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.43531e-11\trel.res = 1.47045e-15\tbw.error = 3.21352e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.040668', '# - total flops = 1.51917e+07, min = 7.43353e+06, max = 7.75814e+06', '# - flop rate = 0.373553 GFlop/s', '# - bytes moved = 97.5688 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.39915 GByte/s', '# - solve arithmetic intensity = 0.155702 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 7.620s; elapsed, 1.914s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 7.680s; elapsed, 1.926s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 7.620s; elapsed, 1.914s, #calls: 1', 'TOTAL : CPU, 7.620s; elapsed, 1.914s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.190s; elapsed, 1.830s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.038s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.110s; elapsed, 2.573s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 7.680s; elapsed, 1.926s, #calls: 1', 'TOTAL : CPU, 18.060s; elapsed, 6.366s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 16.040s; elapsed, 2.038s', 'individual call time for EIGEN_LDLT: CPU, 3.710s; elapsed, 2.425s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.035s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,135', '# - number of levels = 77', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.171064', '# - matching time = 0.0355849', '# - symmetrization time = 0.0161989', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,135', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.050483', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.9326 MB', '# - factor time = 2.25728', '# - factor nonzeros = 6,491,577', '# - factor memory = 51.9326 MB', '# - total flops = 7.45434e+09, min = 3.29924e+09, max = 4.1551e+09', '# - flop rate = 3.30236 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.81378e+09', '# --------------------------------------------', '# total = 7.81378e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.43303e-11\trel.res = 1.46947e-15\tbw.error = 3.21352e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0449791', '# - total flops = 1.51917e+07, min = 7.43353e+06, max = 7.75814e+06', '# - flop rate = 0.33775 GFlop/s', '# - bytes moved = 97.5975 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.16984 GByte/s', '# - solve arithmetic intensity = 0.155656 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 15.920s; elapsed, 2.596s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.850s; elapsed, 2.608s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 15.920s; elapsed, 2.596s, #calls: 1', 'TOTAL : CPU, 15.920s; elapsed, 2.596s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 16.040s; elapsed, 2.038s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.035s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.710s; elapsed, 2.425s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.850s; elapsed, 2.608s, #calls: 1', 'TOTAL : CPU, 40.740s; elapsed, 7.106s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.310s; elapsed, 2.063s', 'individual call time for EIGEN_LDLT: CPU, 7.100s; elapsed, 4.389s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.038s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,135', '# - number of levels = 77', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.192469', '# - matching time = 0.049571', '# - symmetrization time = 0.01757', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,135', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0615411', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.9326 MB', '# - factor time = 1.90784', '# - factor nonzeros = 6,491,577', '# - factor memory = 51.9326 MB', '# - total flops = 7.45449e+09, min = 3.29924e+09, max = 4.15525e+09', '# - flop rate = 3.90729 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.81378e+09', '# --------------------------------------------', '# total = 7.81378e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.41901e-11\trel.res = 1.46347e-15\tbw.error = 3.20616e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0497222', '# - total flops = 1.51917e+07, min = 7.43353e+06, max = 7.75814e+06', '# - flop rate = 0.305531 GFlop/s', '# - bytes moved = 97.6642 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.9642 GByte/s', '# - solve arithmetic intensity = 0.15555 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 33.420s; elapsed, 2.300s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 36.650s; elapsed, 2.314s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 33.420s; elapsed, 2.300s, #calls: 1', 'TOTAL : CPU, 33.420s; elapsed, 2.300s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.310s; elapsed, 2.063s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.038s, #calls: 1', 'time for EIGEN_LDLT: CPU, 7.100s; elapsed, 4.389s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 36.650s; elapsed, 2.314s, #calls: 1', 'TOTAL : CPU, 76.300s; elapsed, 8.805s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 4.740s; elapsed, 4.748s', 'individual call time for EIGEN_LDLT: CPU, 2.660s; elapsed, 2.656s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.062s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,141', '# - number of levels = 73', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.166356', '# - matching time = 0.035315', '# - symmetrization time = 0.0222821', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.038007', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.6883 MB', '# - factor time = 1.94571', '# - factor nonzeros = 6,461,037', '# - factor memory = 51.6883 MB', '# - total flops = 7.39199e+09, min = 916324, max = 3.29543e+09', '# - flop rate = 3.79912 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.73217e+09', '# --------------------------------------------', '# total = 7.73217e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.78316e-11\trel.res = 1.1913e-15\tbw.error = 2.54495e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.039614', '# - total flops = 1.65485e+07, min = 39371, max = 7.71852e+06', '# - flop rate = 0.417744 GFlop/s', '# - bytes moved = 90.5862 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.28672 GByte/s', '# - solve arithmetic intensity = 0.182683 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.240s; elapsed, 2.253s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.250s; elapsed, 2.261s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.250s; elapsed, 2.262s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.250s; elapsed, 2.266s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.240s; elapsed, 2.253s, #calls: 1', 'TOTAL : CPU, 2.240s; elapsed, 2.253s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.250s; elapsed, 2.261s, #calls: 1', 'TOTAL : CPU, 2.250s; elapsed, 2.261s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.250s; elapsed, 2.262s, #calls: 1', 'TOTAL : CPU, 2.250s; elapsed, 2.262s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 4.740s; elapsed, 4.748s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.062s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.660s; elapsed, 2.656s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.250s; elapsed, 2.266s, #calls: 1', 'TOTAL : CPU, 9.710s; elapsed, 9.732s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.770s; elapsed, 2.912s', 'individual call time for EIGEN_LDLT: CPU, 2.870s; elapsed, 2.697s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.051s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,141', '# - number of levels = 73', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.167973', '# - matching time = 0.0343199', '# - symmetrization time = 0.0224059', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.042094', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.6883 MB', '# - factor time = 1.70353', '# - factor nonzeros = 6,461,037', '# - factor memory = 51.6883 MB', '# - total flops = 7.39201e+09, min = 916324, max = 3.29543e+09', '# - flop rate = 4.33923 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.73217e+09', '# --------------------------------------------', '# total = 7.73217e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.70025e-11\trel.res = 1.15582e-15\tbw.error = 2.66061e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0429471', '# - total flops = 1.65485e+07, min = 39371, max = 7.71852e+06', '# - flop rate = 0.385324 GFlop/s', '# - bytes moved = 90.6001 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.10958 GByte/s', '# - solve arithmetic intensity = 0.182654 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.700s; elapsed, 2.034s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.840s; elapsed, 2.034s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.830s; elapsed, 2.026s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.030s; elapsed, 2.039s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.840s; elapsed, 2.034s, #calls: 1', 'TOTAL : CPU, 3.840s; elapsed, 2.034s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.700s; elapsed, 2.034s, #calls: 1', 'TOTAL : CPU, 2.700s; elapsed, 2.034s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.830s; elapsed, 2.026s, #calls: 1', 'TOTAL : CPU, 3.830s; elapsed, 2.026s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.770s; elapsed, 2.912s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.051s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.870s; elapsed, 2.697s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.030s; elapsed, 2.039s, #calls: 1', 'TOTAL : CPU, 12.760s; elapsed, 7.699s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 8.240s; elapsed, 2.092s', 'individual call time for EIGEN_LDLT: CPU, 3.140s; elapsed, 2.607s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.039s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,141', '# - number of levels = 73', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.168116', '# - matching time = 0.0348558', '# - symmetrization time = 0.0209329', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0469329', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.6883 MB', '# - factor time = 1.86214', '# - factor nonzeros = 6,461,037', '# - factor memory = 51.6883 MB', '# - total flops = 7.39202e+09, min = 916780, max = 3.29543e+09', '# - flop rate = 3.96963 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.73217e+09', '# --------------------------------------------', '# total = 7.73217e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.69566e-11\trel.res = 1.15385e-15\tbw.error = 2.65871e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0433009', '# - total flops = 1.65485e+07, min = 39371, max = 7.71852e+06', '# - flop rate = 0.382175 GFlop/s', '# - bytes moved = 90.6178 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.09275 GByte/s', '# - solve arithmetic intensity = 0.182619 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.180s; elapsed, 2.197s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 7.900s; elapsed, 2.197s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 6.700s; elapsed, 2.190s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.650s; elapsed, 2.202s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 7.900s; elapsed, 2.197s, #calls: 1', 'TOTAL : CPU, 7.900s; elapsed, 2.197s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.180s; elapsed, 2.197s, #calls: 1', 'TOTAL : CPU, 4.180s; elapsed, 2.197s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 6.700s; elapsed, 2.190s, #calls: 1', 'TOTAL : CPU, 6.700s; elapsed, 2.190s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 8.240s; elapsed, 2.092s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.039s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.140s; elapsed, 2.607s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.650s; elapsed, 2.202s, #calls: 1', 'TOTAL : CPU, 20.120s; elapsed, 6.940s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 17.010s; elapsed, 2.162s', 'individual call time for EIGEN_LDLT: CPU, 7.200s; elapsed, 5.948s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.041s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,141', '# - number of levels = 73', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.171819', '# - matching time = 0.0376098', '# - symmetrization time = 0.0226271', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0495281', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.6883 MB', '# - factor time = 2.11308', '# - factor nonzeros = 6,461,037', '# - factor memory = 51.6883 MB', '# - total flops = 7.39202e+09, min = 916780, max = 3.29543e+09', '# - flop rate = 3.49823 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.73217e+09', '# --------------------------------------------', '# total = 7.73217e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.76568e-11\trel.res = 1.18382e-15\tbw.error = 2.6511e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0517571', '# - total flops = 1.65485e+07, min = 39371, max = 7.71852e+06', '# - flop rate = 0.319734 GFlop/s', '# - bytes moved = 90.6531 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.75151 GByte/s', '# - solve arithmetic intensity = 0.182548 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 7.260s; elapsed, 2.465s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.410s; elapsed, 2.467s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 13.660s; elapsed, 2.461s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.410s; elapsed, 2.471s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.410s; elapsed, 2.467s, #calls: 1', 'TOTAL : CPU, 16.410s; elapsed, 2.467s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 13.660s; elapsed, 2.461s, #calls: 1', 'TOTAL : CPU, 13.660s; elapsed, 2.461s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 7.260s; elapsed, 2.465s, #calls: 1', 'TOTAL : CPU, 7.260s; elapsed, 2.465s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 17.010s; elapsed, 2.162s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.041s, #calls: 1', 'time for EIGEN_LDLT: CPU, 7.200s; elapsed, 5.948s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.410s; elapsed, 2.471s, #calls: 1', 'TOTAL : CPU, 43.770s; elapsed, 10.622s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 2413.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 4.970s; elapsed, 4.977s', 'individual call time for EIGEN_LDLT: CPU, 2.670s; elapsed, 2.670s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.058s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', 'CPP -2', 'CPP -2', '1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,141', '# - number of levels = 72', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.169586', '# - matching time = 0.0344558', '# - symmetrization time = 0.0159969', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.035197', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.6883 MB', '# - factor time = 1.66368', '# - factor nonzeros = 6,461,037', '# - factor memory = 51.6883 MB', '# - total flops = 7.39199e+09, min = 730831, max = 3.2947e+09', '# - flop rate = 4.44315 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.70381e+09', '# --------------------------------------------', '# total = 7.70381e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.8199e-11\trel.res = 1.20703e-15\tbw.error = 2.54697e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.037612', '# - total flops = 2.11372e+07, min = 25719, max = 7.69055e+06', '# - flop rate = 0.561981 GFlop/s', '# - bytes moved = 85.1253 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.26325 GByte/s', '# - solve arithmetic intensity = 0.248307 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.960s; elapsed, 1.966s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.950s; elapsed, 1.963s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.930s; elapsed, 1.966s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.950s; elapsed, 1.967s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.940s; elapsed, 1.966s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.920s; elapsed, 1.967s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.940s; elapsed, 1.968s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.960s; elapsed, 1.971s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.950s; elapsed, 1.963s, #calls: 1', 'TOTAL : CPU, 1.950s; elapsed, 1.963s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.930s; elapsed, 1.966s, #calls: 1', 'TOTAL : CPU, 1.930s; elapsed, 1.966s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.920s; elapsed, 1.967s, #calls: 1', 'TOTAL : CPU, 1.920s; elapsed, 1.967s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.950s; elapsed, 1.967s, #calls: 1', 'TOTAL : CPU, 1.950s; elapsed, 1.967s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.940s; elapsed, 1.966s, #calls: 1', 'TOTAL : CPU, 1.940s; elapsed, 1.966s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.960s; elapsed, 1.966s, #calls: 1', 'TOTAL : CPU, 1.960s; elapsed, 1.966s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.940s; elapsed, 1.968s, #calls: 1', 'TOTAL : CPU, 1.940s; elapsed, 1.968s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 4.970s; elapsed, 4.977s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.058s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.670s; elapsed, 2.670s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.960s; elapsed, 1.971s, #calls: 1', 'TOTAL : CPU, 9.660s; elapsed, 9.677s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 2413.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.150s; elapsed, 3.101s', 'individual call time for EIGEN_LDLT: CPU, 5.250s; elapsed, 5.079s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.051s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,141', '# - number of levels = 72', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.153727', '# - matching time = 0.0327611', '# - symmetrization time = 0.0188751', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0416169', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.6883 MB', '# - factor time = 1.86923', '# - factor nonzeros = 6,461,037', '# - factor memory = 51.6883 MB', '# - total flops = 7.392e+09, min = 730831, max = 3.2947e+09', '# - flop rate = 3.95456 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.70381e+09', '# --------------------------------------------', '# total = 7.70381e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.81859e-11\trel.res = 1.20647e-15\tbw.error = 2.54697e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0411358', '# - total flops = 2.11372e+07, min = 25719, max = 7.69055e+06', '# - flop rate = 0.51384 GFlop/s', '# - bytes moved = 85.1328 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.06955 GByte/s', '# - solve arithmetic intensity = 0.248285 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.840s; elapsed, 2.176s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.190s; elapsed, 2.176s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 3.620s; elapsed, 2.170s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 3.600s; elapsed, 2.177s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.810s; elapsed, 2.177s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 3.110s; elapsed, 2.176s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 3.660s; elapsed, 2.176s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.280s; elapsed, 2.181s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 3.110s; elapsed, 2.176s, #calls: 1', 'TOTAL : CPU, 3.110s; elapsed, 2.176s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.810s; elapsed, 2.177s, #calls: 1', 'TOTAL : CPU, 2.810s; elapsed, 2.177s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.190s; elapsed, 2.176s, #calls: 1', 'TOTAL : CPU, 4.190s; elapsed, 2.176s', 'Exiting profiler', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 3.600s; elapsed, 2.177s, #calls: 1', 'TOTAL : CPU, 3.600s; elapsed, 2.177s', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 3.620s; elapsed, 2.170s, #calls: 1', 'TOTAL : CPU, 3.620s; elapsed, 2.170s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.840s; elapsed, 2.176s, #calls: 1', 'TOTAL : CPU, 2.840s; elapsed, 2.176s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 3.660s; elapsed, 2.176s, #calls: 1', 'TOTAL : CPU, 3.660s; elapsed, 2.176s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.150s; elapsed, 3.101s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.051s, #calls: 1', 'time for EIGEN_LDLT: CPU, 5.250s; elapsed, 5.079s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.280s; elapsed, 2.181s, #calls: 1', 'TOTAL : CPU, 15.760s; elapsed, 10.412s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 2413.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 12.210s; elapsed, 3.084s', 'individual call time for EIGEN_LDLT: CPU, 3.180s; elapsed, 2.624s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.043s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,141', '# - number of levels = 72', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.175243', '# - matching time = 0.0409341', '# - symmetrization time = 0.0202', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0456691', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.6883 MB', '# - factor time = 1.64532', '# - factor nonzeros = 6,461,037', '# - factor memory = 51.6883 MB', '# - total flops = 7.392e+09, min = 731259, max = 3.2947e+09', '# - flop rate = 4.49275 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.70381e+09', '# --------------------------------------------', '# total = 7.70381e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.81788e-11\trel.res = 1.20616e-15\tbw.error = 2.54697e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0407469', '# - total flops = 2.11372e+07, min = 25719, max = 7.69055e+06', '# - flop rate = 0.518744 GFlop/s', '# - bytes moved = 85.1418 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.08953 GByte/s', '# - solve arithmetic intensity = 0.248259 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 6.360s; elapsed, 1.988s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.090s; elapsed, 1.991s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 7.540s; elapsed, 1.991s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 6.770s; elapsed, 1.989s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 4.820s; elapsed, 1.988s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 6.470s; elapsed, 1.982s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.000s; elapsed, 1.990s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 7.840s; elapsed, 1.993s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 7.540s; elapsed, 1.991s, #calls: 1', 'TOTAL : CPU, 7.540s; elapsed, 1.991s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.090s; elapsed, 1.991s, #calls: 1', 'TOTAL : CPU, 4.090s; elapsed, 1.991s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 6.360s; elapsed, 1.988s, #calls: 1', 'TOTAL : CPU, 6.360s; elapsed, 1.988s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 6.770s; elapsed, 1.989s, #calls: 1', 'TOTAL : CPU, 6.770s; elapsed, 1.989s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 6.470s; elapsed, 1.982s, #calls: 1', 'TOTAL : CPU, 6.470s; elapsed, 1.982s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.000s; elapsed, 1.990s, #calls: 1', 'TOTAL : CPU, 4.000s; elapsed, 1.990s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 4.820s; elapsed, 1.988s, #calls: 1', 'TOTAL : CPU, 4.820s; elapsed, 1.988s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 12.210s; elapsed, 3.084s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.043s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.180s; elapsed, 2.624s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 7.840s; elapsed, 1.993s, #calls: 1', 'TOTAL : CPU, 23.330s; elapsed, 7.743s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 1206.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 4.710s; elapsed, 4.718s', 'individual call time for EIGEN_LDLT: CPU, 2.600s; elapsed, 2.594s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.057s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,141', '# - number of levels = 71', '# - nd time = 0.151932', '# - matching time = 0.0323231', '# - symmetrization time = 0.013088', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0357971', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.6883 MB', '# - factor time = 1.88278', '# - factor nonzeros = 6,461,037', '# - factor memory = 51.6883 MB', '# - total flops = 7.39199e+09, min = 730831, max = 3.28676e+09', '# - flop rate = 3.9261 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.64871e+09', '# --------------------------------------------', '# total = 7.64871e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.4552e-11\trel.res = 1.05092e-15\tbw.error = 2.21425e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.033186', '# - total flops = 2.9672e+07, min = 19684, max = 7.58043e+06', '# - flop rate = 0.894114 GFlop/s', '# - bytes moved = 82.5245 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.48673 GByte/s', '# - solve arithmetic intensity = 0.359554 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.140s; elapsed, 2.159s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.130s; elapsed, 2.159s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.130s; elapsed, 2.159s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.130s; elapsed, 2.162s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 2.130s; elapsed, 2.160s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 2.110s; elapsed, 2.160s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.120s; elapsed, 2.160s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 2.140s; elapsed, 2.159s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 2.140s; elapsed, 2.155s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 2.110s; elapsed, 2.157s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 2.110s; elapsed, 2.159s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.090s; elapsed, 2.158s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 2.090s; elapsed, 2.158s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.140s; elapsed, 2.159s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 2.100s; elapsed, 2.159s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.170s; elapsed, 2.165s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.130s; elapsed, 2.162s, #calls: 1', 'TOTAL : CPU, 2.130s; elapsed, 2.162s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 2.110s; elapsed, 2.160s, #calls: 1', 'TOTAL : CPU, 2.110s; elapsed, 2.160s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 2.140s; elapsed, 2.159s, #calls: 1', 'TOTAL : CPU, 2.140s; elapsed, 2.159s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.130s; elapsed, 2.159s, #calls: 1', 'TOTAL : CPU, 2.130s; elapsed, 2.159s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 2.140s; elapsed, 2.155s, #calls: 1', 'TOTAL : CPU, 2.140s; elapsed, 2.155s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.140s; elapsed, 2.159s, #calls: 1', 'TOTAL : CPU, 2.140s; elapsed, 2.159s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.120s; elapsed, 2.160s, #calls: 1', 'TOTAL : CPU, 2.120s; elapsed, 2.160s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 2.130s; elapsed, 2.160s, #calls: 1', 'TOTAL : CPU, 2.130s; elapsed, 2.160s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.130s; elapsed, 2.159s, #calls: 1', 'TOTAL : CPU, 2.130s; elapsed, 2.159s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.090s; elapsed, 2.158s, #calls: 1', 'TOTAL : CPU, 2.090s; elapsed, 2.158s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.140s; elapsed, 2.159s, #calls: 1', 'TOTAL : CPU, 2.140s; elapsed, 2.159s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 2.090s; elapsed, 2.158s, #calls: 1', 'TOTAL : CPU, 2.090s; elapsed, 2.158s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 2.110s; elapsed, 2.157s, #calls: 1', 'TOTAL : CPU, 2.110s; elapsed, 2.157s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 2.100s; elapsed, 2.159s, #calls: 1', 'TOTAL : CPU, 2.100s; elapsed, 2.159s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 2.110s; elapsed, 2.159s, #calls: 1', 'TOTAL : CPU, 2.110s; elapsed, 2.159s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 4.710s; elapsed, 4.718s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.057s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.600s; elapsed, 2.594s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.170s; elapsed, 2.165s, #calls: 1', 'TOTAL : CPU, 9.530s; elapsed, 9.534s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 1206.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.450s; elapsed, 3.249s', 'individual call time for EIGEN_LDLT: CPU, 2.870s; elapsed, 2.695s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.071s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,141', '# - number of levels = 71', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# - nd time = # ******************************************************************', '0.173838', '# - matching time = 0.0362661', '# - symmetrization time = 0.0147071', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.045975', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.6883 MB', '# - factor time = 1.69256', '# - factor nonzeros = 6,461,037', '# - factor memory = 51.6883 MB', '# - total flops = 7.39199e+09, min = 730831, max = 3.28676e+09', '# - flop rate = 4.36734 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.64871e+09', '# --------------------------------------------', '# total = 7.64871e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.455e-11\trel.res = 1.05084e-15\tbw.error = 2.21425e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0389729', '# - total flops = 2.9672e+07, min = 19684, max = 7.58043e+06', '# - flop rate = 0.761351 GFlop/s', '# - bytes moved = 82.5284 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.11759 GByte/s', '# - solve arithmetic intensity = 0.359537 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.720s; elapsed, 2.024s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.690s; elapsed, 2.025s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.780s; elapsed, 2.024s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 2.900s; elapsed, 2.022s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 3.290s; elapsed, 2.024s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 3.000s; elapsed, 2.021s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 3.520s; elapsed, 2.017s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 3.440s; elapsed, 2.021s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.640s; elapsed, 2.022s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 3.930s; elapsed, 2.022s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 3.600s; elapsed, 2.021s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.650s; elapsed, 2.022s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 2.940s; elapsed, 2.022s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 2.880s; elapsed, 2.022sindividual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 3.170s; elapsed, 2.021s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.000s; elapsed, 2.027s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.690s; elapsed, 2.025s, #calls: 1', 'TOTAL : CPU, 2.690s; elapsed, 2.025s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.720s; elapsed, 2.024s, #calls: 1', 'TOTAL : CPU, 2.720s; elapsed, 2.024s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.780s; elapsed, 2.024s, #calls: 1', 'TOTAL : CPU, 2.780s; elapsed, 2.024s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 3.290s; elapsed, 2.024s, #calls: 1', 'TOTAL : CPU, 3.290s; elapsed, 2.024s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 3.440s; elapsed, 2.021s, #calls: 1', 'TOTAL : CPU, 3.440s; elapsed, 2.021s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 3.520s; elapsed, 2.017s, #calls: 1', 'TOTAL : CPU, 3.520s; elapsed, 2.017s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 3.930s; elapsed, 2.022s, #calls: 1', 'TOTAL : CPU, 3.930s; elapsed, 2.022s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 2.940s; elapsed, 2.022s, #calls: 1', 'TOTAL : CPU, 2.940s; elapsed, 2.022s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 2.900s; elapsed, 2.022s, #calls: 1', 'TOTAL : CPU, 2.900s; elapsed, 2.022s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.640s; elapsed, 2.022s, #calls: 1', 'TOTAL : CPU, 2.640s; elapsed, 2.022s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.650s; elapsed, 2.022s, #calls: 1', 'TOTAL : CPU, 2.650s; elapsed, 2.022s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 3.600s; elapsed, 2.021s, #calls: 1', 'TOTAL : CPU, 3.600s; elapsed, 2.021s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 2.880s; elapsed, 2.022s, #calls: 1', 'TOTAL : CPU, 2.880s; elapsed, 2.022s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 3.000s; elapsed, 2.021s, #calls: 1', 'TOTAL : CPU, 3.000s; elapsed, 2.021s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 3.170s; elapsed, 2.021s, #calls: 1', 'TOTAL : CPU, 3.170s; elapsed, 2.021s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.450s; elapsed, 3.249s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.071s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.870s; elapsed, 2.695s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.000s; elapsed, 2.027s, #calls: 1', 'TOTAL : CPU, 13.420s; elapsed, 8.041s']
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Eta-/A_strum_5k_omp1_paramslevmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Eta-/b_strum_5k_omp1_paramslevmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '17309 17309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 4.190s; elapsed, 4.200s', 'individual call time for EIGEN_LDLT: CPU, 2.520s; elapsed, 2.509s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.050s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 4.190s; elapsed, 4.200s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.050s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.520s; elapsed, 2.509s, #calls: 1', 'TOTAL : CPU, 6.760s; elapsed, 6.759s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '17309 17309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.380s; elapsed, 2.714s', 'individual call time for EIGEN_LDLT: CPU, 2.770s; elapsed, 2.584s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.066s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.380s; elapsed, 2.714s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.066s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.770s; elapsed, 2.584s, #calls: 1', 'TOTAL : CPU, 8.260s; elapsed, 5.364s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '17309 17309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 9.400s; elapsed, 2.384s', 'individual call time for EIGEN_LDLT: CPU, 3.350s; elapsed, 2.806s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.037s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 9.400s; elapsed, 2.384s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.037s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.350s; elapsed, 2.806s, #calls: 1', 'TOTAL : CPU, 12.840s; elapsed, 5.227s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '17309 17309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 15.870s; elapsed, 2.022s', 'individual call time for EIGEN_LDLT: CPU, 4.910s; elapsed, 3.619s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.037s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 15.870s; elapsed, 2.022s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.037s, #calls: 1', 'time for EIGEN_LDLT: CPU, 4.910s; elapsed, 3.619s, #calls: 1', 'TOTAL : CPU, 20.920s; elapsed, 5.678s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '17309 17309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 36.750s; elapsed, 2.347s', 'individual call time for EIGEN_LDLT: CPU, 6.330s; elapsed, 3.595s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.036s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 36.750s; elapsed, 2.347s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.036s, #calls: 1', 'time for EIGEN_LDLT: CPU, 6.330s; elapsed, 3.595s, #calls: 1', 'TOTAL : CPU, 43.300s; elapsed, 5.978s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '17309 8654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 4.370s; elapsed, 4.382s', 'individual call time for EIGEN_LDLT: CPU, 2.600s; elapsed, 2.606s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.052s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 17,309', '# - number of nonzeros = 249,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,087', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.1261', '# - matching time = 0.030179', '# - symmetrization time = 0.0141439', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,087', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.041594', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 53.0181 MB', '# - factor time = 2.81571', '# - factor nonzeros = 6,627,267', '# - factor memory = 53.0181 MB', '# - total flops = 7.33115e+09, min = 3.62189e+09, max = 3.70926e+09', '# - flop rate = 2.60366 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.68091e+09', '# --------------------------------------------', '# total = 7.68091e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.3352e-11\trel.res = 1.4276e-15\tbw.error = 3.28895e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.035306', '# - total flops = 1.53008e+07, min = 6.57394e+06, max = 8.72688e+06', '# - flop rate = 0.433377 GFlop/s', '# - bytes moved = 92.1272 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.60939 GByte/s', '# - solve arithmetic intensity = 0.166083 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.060s; elapsed, 3.073s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.090s; elapsed, 3.085s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.060s; elapsed, 3.073s, #calls: 1', 'TOTAL : CPU, 3.060s; elapsed, 3.073s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 4.370s; elapsed, 4.382s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.052s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.600s; elapsed, 2.606s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.090s; elapsed, 3.085s, #calls: 1', 'TOTAL : CPU, 10.110s; elapsed, 10.125s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '17309 8654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.310s; elapsed, 2.675s', 'individual call time for EIGEN_LDLT: CPU, 2.820s; elapsed, 2.638s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.046s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 17,309', '# - number of nonzeros = 249,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,087', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.130841', '# - matching time = 0.0312269', '# - symmetrization time = 0.0159578', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,087', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0484161', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 53.0181 MB', '# - factor time = 1.77197', '# - factor nonzeros = 6,627,267', '# - factor memory = 53.0181 MB', '# - total flops = 7.3312e+09, min = 3.62194e+09, max = 3.70926e+09', '# - flop rate = 4.13732 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.68091e+09', '# --------------------------------------------', '# total = 7.68091e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.3097e-11\trel.res = 1.41669e-15\tbw.error = 3.2689e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0360379', '# - total flops = 1.53038e+07, min = 6.57691e+06, max = 8.72688e+06', '# - flop rate = 0.424658 GFlop/s', '# - bytes moved = 92.2479 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.55974 GByte/s', '# - solve arithmetic intensity = 0.165899 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.020s; elapsed, 2.048s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.110s; elapsed, 2.060s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.020s; elapsed, 2.048s, #calls: 1', 'TOTAL : CPU, 4.020s; elapsed, 2.048s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.310s; elapsed, 2.675s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.046s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.820s; elapsed, 2.638s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.110s; elapsed, 2.060s, #calls: 1', 'TOTAL : CPU, 12.310s; elapsed, 7.419s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '17309 8654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 10.950s; elapsed, 2.767s', 'individual call time for EIGEN_LDLT: CPU, 4.100s; elapsed, 3.540s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.042s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 17,309', '# - number of nonzeros = 249,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,087', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.123603', '# - matching time = 0.031127', '# - symmetrization time = 0.0144041', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,087', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0463271', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 53.0181 MB', '# - factor time = 2.24457', '# - factor nonzeros = 6,627,267', '# - factor memory = 53.0181 MB', '# - total flops = 7.3312e+09, min = 3.62194e+09, max = 3.70926e+09', '# - flop rate = 3.26619 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.68091e+09', '# --------------------------------------------', '# total = 7.68091e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.30224e-11\trel.res = 1.41349e-15\tbw.error = 3.26735e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0390739', '# - total flops = 1.53038e+07, min = 6.57691e+06, max = 8.72688e+06', '# - flop rate = 0.391662 GFlop/s', '# - bytes moved = 92.2597 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.36116 GByte/s', '# - solve arithmetic intensity = 0.165877 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.030s; elapsed, 2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.100s; elapsed, 2.525s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.030s; elapsed, 2.513s, #calls: 1', 'TOTAL : CPU, 8.030s; elapsed, 2.513s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 10.950s; elapsed, 2.767s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.042s, #calls: 1', 'time for EIGEN_LDLT: CPU, 4.100s; elapsed, 3.540s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.100s; elapsed, 2.525s, #calls: 1', 'TOTAL : CPU, 25.240s; elapsed, 8.874s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '17309 8654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 17.540s; elapsed, 2.228s', 'individual call time for EIGEN_LDLT: CPU, 4.430s; elapsed, 3.129s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.038s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processesCPP -2', '', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 17,309', '# - number of nonzeros = 249,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,087', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.128883', '# - matching time = 0.0334818', '# - symmetrization time = 0.0149758', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,087', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0500679', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 53.0181 MB', '# - factor time = 1.94183', '# - factor nonzeros = 6,627,267', '# - factor memory = 53.0181 MB', '# - total flops = 7.33121e+09, min = 3.62194e+09, max = 3.70926e+09', '# - flop rate = 3.77541 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.68091e+09', '# --------------------------------------------', '# total = 7.68091e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.30087e-11\trel.res = 1.4129e-15\tbw.error = 3.26735e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0372891', '# - total flops = 1.53038e+07, min = 6.57691e+06, max = 8.72688e+06', '# - flop rate = 0.410409 GFlop/s', '# - bytes moved = 92.2841 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.47482 GByte/s', '# - solve arithmetic intensity = 0.165833 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 14.640s; elapsed, 2.221s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.860s; elapsed, 2.234s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 14.640s; elapsed, 2.221s, #calls: 1', 'TOTAL : CPU, 14.640s; elapsed, 2.221s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 17.540s; elapsed, 2.228s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.038s, #calls: 1', 'time for EIGEN_LDLT: CPU, 4.430s; elapsed, 3.129s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.860s; elapsed, 2.234s, #calls: 1', 'TOTAL : CPU, 39.970s; elapsed, 7.629s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '17309 8654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 33.660s; elapsed, 2.152s', 'individual call time for EIGEN_LDLT: CPU, 6.880s; elapsed, 4.158s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.040s', 'CPP -2', '# Initializing STRUMPACKCPP -2', '', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 17,309', '# - number of nonzeros = 249,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,087', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.14408', '# - matching time = 0.0367179', '# - symmetrization time = 0.0176201', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,087', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0636098', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 53.0181 MB', '# - factor time = 1.79586', '# - factor nonzeros = 6,627,267', '# - factor memory = 53.0181 MB', '# - total flops = 7.33139e+09, min = 3.62212e+09, max = 3.70926e+09', '# - flop rate = 4.08239 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.68091e+09', '# --------------------------------------------', '# total = 7.68091e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.29008e-11\trel.res = 1.40828e-15\tbw.error = 3.26812e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.050818', '# - total flops = 1.53038e+07, min = 6.57691e+06, max = 8.72688e+06', '# - flop rate = 0.301149 GFlop/s', '# - bytes moved = 92.3391 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.81706 GByte/s', '# - solve arithmetic intensity = 0.165735 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 31.560s; elapsed, 2.128s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 34.110s; elapsed, 2.139s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 31.560s; elapsed, 2.128s, #calls: 1', 'TOTAL : CPU, 31.560s; elapsed, 2.128s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 33.660s; elapsed, 2.152s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.040s, #calls: 1', 'time for EIGEN_LDLT: CPU, 6.880s; elapsed, 4.158s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 34.110s; elapsed, 2.139s, #calls: 1', 'TOTAL : CPU, 74.900s; elapsed, 8.490s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '17309 4327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 4.670s; elapsed, 4.672s', 'individual call time for EIGEN_LDLT: CPU, 2.690s; elapsed, 2.685s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.058s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 17,309', '# - number of nonzeros = 249,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,087', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.145503', '# - matching time = 0.0334229', '# - symmetrization time = 0.0190492', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,087', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.039644', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 53.0181 MB', '# - factor time = 1.79257', '# - factor nonzeros = 6,627,267', '# - factor memory = 53.0181 MB', '# - total flops = 7.33115e+09, min = 3.72336e+06, max = 3.70554e+09', '# - flop rate = 4.08973 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.66768e+09', '# --------------------------------------------', '# total = 7.66768e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.91374e-11\trel.res = 1.24719e-15\tbw.error = 2.48962e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0350709', '# - total flops = 1.64897e+07, min = 63729, max = 8.66016e+06', '# - flop rate = 0.470182 GFlop/s', '# - bytes moved = 86.6401 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.47043 GByte/s', '# - solve arithmetic intensity = 0.190324 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.070s; elapsed, 2.079s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.050s; elapsed, 2.077s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.070s; elapsed, 2.070s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.080s; elapsed, 2.082s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.070s; elapsed, 2.079s, #calls: 1', 'TOTAL : CPU, 2.070s; elapsed, 2.079s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.070s; elapsed, 2.070s, #calls: 1', 'TOTAL : CPU, 2.070s; elapsed, 2.070s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.050s; elapsed, 2.077s, #calls: 1', 'TOTAL : CPU, 2.050s; elapsed, 2.077s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 4.670s; elapsed, 4.672s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.058s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.690s; elapsed, 2.685s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.080s; elapsed, 2.082s, #calls: 1', 'TOTAL : CPU, 9.500s; elapsed, 9.497s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '17309 4327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.210s; elapsed, 2.628s', 'individual call time for EIGEN_LDLT: CPU, 2.750s; elapsed, 2.569s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.046s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 4 MPI processesCPP -2', 'CPP -2', 'CPP -2', '', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 17,309', '# - number of nonzeros = 249,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,087', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.130406', '# - matching time = 0.0314019', '# - symmetrization time = 0.0193281', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,087', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0407379', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 53.0181 MB', '# - factor time = 1.95598', '# - factor nonzeros = 6,627,267', '# - factor memory = 53.0181 MB', '# - total flops = 7.33116e+09, min = 3.72336e+06, max = 3.70554e+09', '# - flop rate = 3.74808 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.66768e+09', '# --------------------------------------------', '# total = 7.66768e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.89755e-11\trel.res = 1.24027e-15\tbw.error = 2.42638e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.041337', '# - total flops = 1.64897e+07, min = 63729, max = 8.66016e+06', '# - flop rate = 0.398909 GFlop/s', '# - bytes moved = 86.6493 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.09617 GByte/s', '# - solve arithmetic intensity = 0.190304 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.870s; elapsed, 2.237s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.070s; elapsed, 2.235s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.790s; elapsed, 2.228s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.420s; elapsed, 2.240s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.070s; elapsed, 2.235s, #calls: 1', 'TOTAL : CPU, 4.070s; elapsed, 2.235s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.870s; elapsed, 2.237s, #calls: 1', 'TOTAL : CPU, 2.870s; elapsed, 2.237s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.790s; elapsed, 2.228s, #calls: 1', 'TOTAL : CPU, 3.790s; elapsed, 2.228s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.210s; elapsed, 2.628s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.046s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.750s; elapsed, 2.569s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.420s; elapsed, 2.240s, #calls: 1', 'TOTAL : CPU, 12.450s; elapsed, 7.484s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '17309 4327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.360s; elapsed, 1.876s', 'individual call time for EIGEN_LDLT: CPU, 6.640s; elapsed, 6.100s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.043s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 17,309', '# - number of nonzeros = 249,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,087', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.129768', '# - matching time = 0.032598', '# - symmetrization time = 0.0193801', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,087', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0464342', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 53.0181 MB', '# - factor time = 1.72388', '# - factor nonzeros = 6,627,267', '# - factor memory = 53.0181 MB', '# - total flops = 7.33117e+09, min = 3.72505e+06, max = 3.70554e+09', '# - flop rate = 4.25272 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.66768e+09', '# --------------------------------------------', '# total = 7.66768e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.89543e-11\trel.res = 1.23936e-15\tbw.error = 2.42638e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0433669', '# - total flops = 1.64897e+07, min = 63729, max = 8.66016e+06', '# - flop rate = 0.380237 GFlop/s', '# - bytes moved = 86.6618 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.99834 GByte/s', '# - solve arithmetic intensity = 0.190276 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.880s; elapsed, 2.013s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 7.420s; elapsed, 2.013s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 6.520s; elapsed, 2.005s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 7.950s; elapsed, 2.017s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 7.420s; elapsed, 2.013s, #calls: 1', 'TOTAL : CPU, 7.420s; elapsed, 2.013s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.880s; elapsed, 2.013s, #calls: 1', 'TOTAL : CPU, 3.880s; elapsed, 2.013s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 6.520s; elapsed, 2.005s, #calls: 1', 'TOTAL : CPU, 6.520s; elapsed, 2.005s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.360s; elapsed, 1.876s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.043s, #calls: 1', 'time for EIGEN_LDLT: CPU, 6.640s; elapsed, 6.100s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 7.950s; elapsed, 2.017s, #calls: 1', 'TOTAL : CPU, 22.050s; elapsed, 10.035s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '17309 4327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 16.230s; elapsed, 2.068s', 'individual call time for EIGEN_LDLT: CPU, 3.950s; elapsed, 2.649s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.039s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using CPP -2', 'CPP -2', 'CPP -2', '4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 17,309', '# - number of nonzeros = 249,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,087', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.145592', '# - matching time = 0.0366342', '# - symmetrization time = 0.023905', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,087', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0573509', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 53.0181 MB', '# - factor time = 1.78539', '# - factor nonzeros = 6,627,267', '# - factor memory = 53.0181 MB', '# - total flops = 7.33117e+09, min = 3.7252e+06, max = 3.70554e+09', '# - flop rate = 4.10621 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.66768e+09', '# --------------------------------------------', '# total = 7.66768e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.89215e-11\trel.res = 1.23796e-15\tbw.error = 2.42602e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.048136', '# - total flops = 1.64897e+07, min = 63729, max = 8.66016e+06', '# - flop rate = 0.342565 GFlop/s', '# - bytes moved = 86.6868 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.80087 GByte/s', '# - solve arithmetic intensity = 0.190222 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 7.060s; elapsed, 2.117s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 15.690s; elapsed, 2.117s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 13.010s; elapsed, 2.110s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.650s; elapsed, 2.122s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 7.060s; elapsed, 2.117s, #calls: 1', 'TOTAL : CPU, 7.060s; elapsed, 2.117s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 15.690s; elapsed, 2.117s, #calls: 1', 'TOTAL : CPU, 15.690s; elapsed, 2.117s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 13.010s; elapsed, 2.110s, #calls: 1', 'TOTAL : CPU, 13.010s; elapsed, 2.110s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 16.230s; elapsed, 2.068s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.039s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.950s; elapsed, 2.649s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.650s; elapsed, 2.122s, #calls: 1', 'TOTAL : CPU, 36.970s; elapsed, 6.878s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '17309 2163.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 4.470s; elapsed, 4.469s', 'individual call time for EIGEN_LDLT: CPU, 2.620s; elapsed, 2.626s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.056s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 17,309', '# - number of nonzeros = 249,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,087', '# - number of levels = 66', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.129871', '# - matching time = 0.0303349', '# - symmetrization time = 0.02282', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,087', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.035038', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 53.0181 MB', '# - factor time = 1.91071', '# - factor nonzeros = 6,627,267', '# - factor memory = 53.0181 MB', '# - total flops = 7.33115e+09, min = 3.56378e+06, max = 3.70197e+09', '# - flop rate = 3.83686 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.63859e+09', '# --------------------------------------------', '# total = 7.63859e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.81301e-11\trel.res = 1.20408e-15\tbw.error = 2.38827e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0389578', '# - total flops = 2.05202e+07, min = 52652, max = 8.60455e+06', '# - flop rate = 0.526728 GFlop/s', '# - bytes moved = 81.8271 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.1004 GByte/s', '# - solve arithmetic intensity = 0.250775 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.140s; elapsed, 2.183s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.170s; elapsed, 2.184s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.170s; elapsed, 2.182s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.150s; elapsed, 2.174s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.150s; elapsed, 2.183s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.180s; elapsed, 2.184s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.160s; elapsed, 2.183s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.190s; elapsed, 2.187s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.140s; elapsed, 2.183s, #calls: 1', 'TOTAL : CPU, 2.140s; elapsed, 2.183s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.170s; elapsed, 2.182s, #calls: 1', 'TOTAL : CPU, 2.170s; elapsed, 2.182s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.180s; elapsed, 2.184s, #calls: 1', 'TOTAL : CPU, 2.180s; elapsed, 2.184s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.170s; elapsed, 2.184s, #calls: 1', 'TOTAL : CPU, 2.170s; elapsed, 2.184s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.150s; elapsed, 2.174s, #calls: 1', 'TOTAL : CPU, 2.150s; elapsed, 2.174s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.150s; elapsed, 2.183s, #calls: 1', 'TOTAL : CPU, 2.150s; elapsed, 2.183s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.160s; elapsed, 2.183s, #calls: 1', 'TOTAL : CPU, 2.160s; elapsed, 2.183s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 4.470s; elapsed, 4.469s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.056s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.620s; elapsed, 2.626s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.190s; elapsed, 2.187s, #calls: 1', 'TOTAL : CPU, 9.330s; elapsed, 9.338s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '17309 2163.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.450s; elapsed, 2.758s', 'individual call time for EIGEN_LDLT: CPU, 3.030s; elapsed, 2.834s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.048s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 17,309', '# - number of nonzeros = 249,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,087', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.134082', '# - matching time = 0.0329502', '# - symmetrization time = 0.0245631', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,087', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.044343', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 53.0181 MB', '# - factor time = 1.83034', '# - factor nonzeros = 6,627,267', '# - factor memory = 53.0181 MB', '# - total flops = 7.33116e+09, min = 3.56378e+06, max = 3.70197e+09', '# - flop rate = 4.00535 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.63859e+09', '# --------------------------------------------', '# total = 7.63859e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.81094e-11\trel.res = 1.20319e-15\tbw.error = 2.38936e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0450761', '# - total flops = 2.05202e+07, min = 52652, max = 8.60455e+06', '# - flop rate = 0.455234 GFlop/s', '# - bytes moved = 81.8342 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.81547 GByte/s', '# - solve arithmetic intensity = 0.250753 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.800s; elapsed, 2.130s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 3.520s; elapsed, 2.120s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 3.590s; elapsed, 2.129s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.980s; elapsed, 2.129s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 3.050s; elapsed, 2.129s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 3.670s; elapsed, 2.129s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.770s; elapsed, 2.129s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.210s; elapsed, 2.134s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 3.520s; elapsed, 2.120s, #calls: 1', 'TOTAL : CPU, 3.520s; elapsed, 2.120s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.800s; elapsed, 2.130s, #calls: 1', 'TOTAL : CPU, 2.800s; elapsed, 2.130s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 3.590s; elapsed, 2.129s, #calls: 1', 'TOTAL : CPU, 3.590s; elapsed, 2.129s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 3.050s; elapsed, 2.129s, #calls: 1', 'TOTAL : CPU, 3.050s; elapsed, 2.129s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.770s; elapsed, 2.129s, #calls: 1', 'TOTAL : CPU, 2.770s; elapsed, 2.129s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.980s; elapsed, 2.129s, #calls: 1', 'TOTAL : CPU, 3.980s; elapsed, 2.129s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 3.670s; elapsed, 2.129s, #calls: 1', 'TOTAL : CPU, 3.670s; elapsed, 2.129s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.450s; elapsed, 2.758s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.048s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.030s; elapsed, 2.834s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.210s; elapsed, 2.134s, #calls: 1', 'TOTAL : CPU, 12.760s; elapsed, 7.774s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '17309 2163.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 8.580s; elapsed, 2.176s', 'individual call time for EIGEN_LDLT: CPU, 3.020s; elapsed, 2.469s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.040s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 8 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 17,309', '# - number of nonzeros = 249,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,087', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.147006', '# - matching time = 0.0353072', '# - symmetrization time = 0.023737', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,087', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0469329', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 53.0181 MB', '# - factor time = 1.92219', '# - factor nonzeros = 6,627,267', '# - factor memory = 53.0181 MB', '# - total flops = 7.33116e+09, min = 3.56586e+06, max = 3.70197e+09', '# - flop rate = 3.81397 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.63859e+09', '# --------------------------------------------', '# total = 7.63859e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.14314e-11\trel.res = 1.34539e-15\tbw.error = 2.39921e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0487611', '# - total flops = 2.05202e+07, min = 52652, max = 8.60455e+06', '# - flop rate = 0.420831 GFlop/s', '# - bytes moved = 81.8434 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.67846 GByte/s', '# - solve arithmetic intensity = 0.250725 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.310s; elapsed, 2.241s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.160s; elapsed, 2.241s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 7.690s; elapsed, 2.242s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 6.650s; elapsed, 2.242s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 5.030s; elapsed, 2.241s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 6.490s; elapsed, 2.242s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 6.400s; elapsed, 2.233s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.880s; elapsed, 2.245s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.310s; elapsed, 2.241s, #calls: 1', 'TOTAL : CPU, 4.310s; elapsed, 2.241s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 7.690s; elapsed, 2.242s, #calls: 1', 'TOTAL : CPU, 7.690s; elapsed, 2.242s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 5.030s; elapsed, 2.241s, #calls: 1', 'TOTAL : CPU, 5.030s; elapsed, 2.241s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.160s; elapsed, 2.241s, #calls: 1', 'TOTAL : CPU, 4.160s; elapsed, 2.241s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 6.490s; elapsed, 2.242s, #calls: 1', 'TOTAL : CPU, 6.490s; elapsed, 2.242s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 6.650s; elapsed, 2.242s, #calls: 1', 'TOTAL : CPU, 6.650s; elapsed, 2.242s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 6.400s; elapsed, 2.233s, #calls: 1', 'TOTAL : CPU, 6.400s; elapsed, 2.233s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 8.580s; elapsed, 2.176s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.040s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.020s; elapsed, 2.469s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.880s; elapsed, 2.245s, #calls: 1', 'TOTAL : CPU, 20.580s; elapsed, 6.931s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '17309 1081.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 4.940s; elapsed, 4.942s', 'individual call time for EIGEN_LDLT: CPU, 3.220s; elapsed, 3.219s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.063s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 17,309', '# - number of nonzeros = 249,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,087', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.144272', '# - matching time = 0.0349131', '# - symmetrization time = 0.0130441', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,087', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0379288', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 53.0181 MB', '# - factor time = 1.88906', '# - factor nonzeros = 6,627,267', '# - factor memory = 53.0181 MB', '# - total flops = 7.33115e+09, min = 3.56378e+06, max = 3.68612e+09', '# - flop rate = 3.88084 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.58349e+09', '# --------------------------------------------', '# total = 7.58349e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.40797e-11\trel.res = 1.03071e-15\tbw.error = 2.36644e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0317309', '# - total flops = 2.77955e+07, min = 47242, max = 8.4104e+06', '# - flop rate = 0.875975 GFlop/s', '# - bytes moved = 78.9378 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.48773 GByte/s', '# - solve arithmetic intensity = 0.352118 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.100s; elapsed, 2.162s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 2.150s; elapsed, 2.164s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 2.120s; elapsed, 2.164s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 2.150s; elapsed, 2.165s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 2.150s; elapsed, 2.164s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.140s; elapsed, 2.163s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.120s; elapsed, 2.164s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.100s; elapsed, 2.163s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.160s; elapsed, 2.163s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 2.100s; elapsed, 2.160s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 2.120s; elapsed, 2.158s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.160s; elapsed, 2.163s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.130s; elapsed, 2.164s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 2.130s; elapsed, 2.163s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 2.110s; elapsed, 2.164s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.160s; elapsed, 2.169s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 2.150s; elapsed, 2.165s, #calls: 1', 'TOTAL : CPU, 2.150s; elapsed, 2.165s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 2.150s; elapsed, 2.164s, #calls: 1', 'TOTAL : CPU, 2.150s; elapsed, 2.164s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 2.150s; elapsed, 2.164s, #calls: 1', 'TOTAL : CPU, 2.150s; elapsed, 2.164s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.120s; elapsed, 2.164s, #calls: 1', 'TOTAL : CPU, 2.120s; elapsed, 2.164s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.100s; elapsed, 2.163s, #calls: 1', 'TOTAL : CPU, 2.100s; elapsed, 2.163s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 2.100s; elapsed, 2.160s, #calls: 1', 'TOTAL : CPU, 2.100s; elapsed, 2.160s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 2.120s; elapsed, 2.158s, #calls: 1', 'TOTAL : CPU, 2.120s; elapsed, 2.158s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 2.110s; elapsed, 2.164s, #calls: 1', 'TOTAL : CPU, 2.110s; elapsed, 2.164s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.160s; elapsed, 2.163s, #calls: 1', 'TOTAL : CPU, 2.160s; elapsed, 2.163s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.130s; elapsed, 2.164s, #calls: 1', 'TOTAL : CPU, 2.130s; elapsed, 2.164s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.140s; elapsed, 2.163s, #calls: 1', 'TOTAL : CPU, 2.140s; elapsed, 2.163s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.160s; elapsed, 2.163s, #calls: 1', 'TOTAL : CPU, 2.160s; elapsed, 2.163s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.100s; elapsed, 2.162s, #calls: 1', 'TOTAL : CPU, 2.100s; elapsed, 2.162s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 2.120s; elapsed, 2.164s, #calls: 1', 'TOTAL : CPU, 2.120s; elapsed, 2.164s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 2.130s; elapsed, 2.163s, #calls: 1', 'TOTAL : CPU, 2.130s; elapsed, 2.163s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 4.940s; elapsed, 4.942s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.063s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.220s; elapsed, 3.219s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.160s; elapsed, 2.169s, #calls: 1', 'TOTAL : CPU, 10.380s; elapsed, 10.393s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '17309 1081.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.780s; elapsed, 2.916s', 'individual call time for EIGEN_LDLT: CPU, 2.710s; elapsed, 2.530s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.048s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threadsCPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 16 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 17,309', '# - number of nonzeros = 249,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,087', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.149898', '# - matching time = 0.0356231', '# - symmetrization time = 0.0138478', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,087', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.045547', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 53.0181 MB', '# - factor time = 1.70579', '# - factor nonzeros = 6,627,267', '# - factor memory = 53.0181 MB', '# - total flops = 7.33116e+09, min = 3.56378e+06, max = 3.68612e+09', '# - flop rate = 4.29781 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.58349e+09', '# --------------------------------------------', '# total = 7.58349e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.63605e-11\trel.res = 1.12833e-15\tbw.error = 2.10167e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.033108', '# - total flops = 2.77955e+07, min = 47242, max = 8.4104e+06', '# - flop rate = 0.839539 GFlop/s', '# - bytes moved = 78.9432 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.38442 GByte/s', '# - solve arithmetic intensity = 0.352094 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.750s; elapsed, 2.000s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.670s; elapsed, 1.999s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.620s; elapsed, 1.997sindividual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 2.860s; elapsed, 1.997s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.590s; elapsed, 1.997s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 2.830s; elapsed, 1.999s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 3.440s; elapsed, 1.998s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 3.420s; elapsed, 1.993s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.710s; elapsed, 1.998s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.620s; elapsed, 1.998s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 3.810s; elapsed, 1.997s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 3.090s; elapsed, 1.998s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 3.380s; elapsed, 1.994s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 2.940s; elapsed, 1.998s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 3.430s; elapsed, 1.998s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.950s; elapsed, 2.005s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.750s; elapsed, 2.000s, #calls: 1', 'TOTAL : CPU, 2.750s; elapsed, 2.000s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 2.940s; elapsed, 1.998s, #calls: 1', 'TOTAL : CPU, 2.940s; elapsed, 1.998s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.670s; elapsed, 1.999s, #calls: 1', 'TOTAL : CPU, 2.670s; elapsed, 1.999s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.710s; elapsed, 1.998s, #calls: 1', 'TOTAL : CPU, 2.710s; elapsed, 1.998s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 2.860s; elapsed, 1.997s, #calls: 1', 'TOTAL : CPU, 2.860s; elapsed, 1.997s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 3.380s; elapsed, 1.994s, #calls: 1', 'TOTAL : CPU, 3.380s; elapsed, 1.994s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 2.830s; elapsed, 1.999s, #calls: 1', 'TOTAL : CPU, 2.830s; elapsed, 1.999s', 'Exiting profiler', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 3.440s; elapsed, 1.998s, #calls: 1', 'TOTAL : CPU, 3.440s; elapsed, 1.998s', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 3.430s; elapsed, 1.998s, #calls: 1', 'TOTAL : CPU, 3.430s; elapsed, 1.998s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 3.810s; elapsed, 1.997s, #calls: 1', 'TOTAL : CPU, 3.810s; elapsed, 1.997s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 3.420s; elapsed, 1.993s, #calls: 1', 'TOTAL : CPU, 3.420s; elapsed, 1.993s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.620s; elapsed, 1.998s, #calls: 1', 'TOTAL : CPU, 2.620s; elapsed, 1.998s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.590s; elapsed, 1.997s, #calls: 1', 'TOTAL : CPU, 2.590s; elapsed, 1.997s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.620s; elapsed, 1.997s, #calls: 1', 'TOTAL : CPU, 2.620s; elapsed, 1.997s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 3.090s; elapsed, 1.998s, #calls: 1', 'TOTAL : CPU, 3.090s; elapsed, 1.998s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.780s; elapsed, 2.916s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.048s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.710s; elapsed, 2.530s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.950s; elapsed, 2.005s, #calls: 1', 'TOTAL : CPU, 12.520s; elapsed, 7.499s']
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-/A_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-/b_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 24.060s; elapsed, 24.074s', 'individual call time for EIGEN_LDLT: CPU, 18.820s; elapsed, 18.838s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.169s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 24.060s; elapsed, 24.074s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.169s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.820s; elapsed, 18.838s, #calls: 1', 'TOTAL : CPU, 43.050s; elapsed, 43.082s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 30.320s; elapsed, 15.219s', 'individual call time for EIGEN_LDLT: CPU, 19.210s; elapsed, 19.031s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.162s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 30.320s; elapsed, 15.219s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.162s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.210s; elapsed, 19.031s, #calls: 1', 'TOTAL : CPU, 49.800s; elapsed, 34.412s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 38.720s; elapsed, 9.720s', 'individual call time for EIGEN_LDLT: CPU, 19.430s; elapsed, 18.877s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.117s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 38.720s; elapsed, 9.720s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.117s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.430s; elapsed, 18.877s, #calls: 1', 'TOTAL : CPU, 58.450s; elapsed, 28.715s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 80.890s; elapsed, 10.165s', 'individual call time for EIGEN_LDLT: CPU, 20.280s; elapsed, 19.021s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.390s; elapsed, 0.097s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 80.890s; elapsed, 10.165s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.390s; elapsed, 0.097s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.280s; elapsed, 19.021s, #calls: 1', 'TOTAL : CPU, 101.560s; elapsed, 29.283s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 166.980s; elapsed, 10.497s', 'individual call time for EIGEN_LDLT: CPU, 21.380s; elapsed, 18.683s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.740s; elapsed, 0.088s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 166.980s; elapsed, 10.497s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.740s; elapsed, 0.088s, #calls: 1', 'time for EIGEN_LDLT: CPU, 21.380s; elapsed, 18.683s, #calls: 1', 'TOTAL : CPU, 189.100s; elapsed, 29.267s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 24.790s; elapsed, 24.814s', 'individual call time for EIGEN_LDLT: CPU, 19.390s; elapsed, 19.405s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.179s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,781', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.226218', '# - matching time = 0.0563948', '# - symmetrization time = 0.0344679', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,781', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.074439', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 160.156 MB', '# - factor time = 16.616', '# - factor nonzeros = 20,019,517', '# - factor memory = 160.156 MB', '# - total flops = 5.00254e+10, min = 2.29344e+10, max = 2.7091e+10', '# - flop rate = 3.01068 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.23886e+10', '# --------------------------------------------', '# total = 5.23886e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.51073e-11\trel.res = 1.84455e-15\tbw.error = 4.16326e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0946271', '# - total flops = 4.47923e+07, min = 2.05301e+07, max = 2.42622e+07', '# - flop rate = 0.473356 GFlop/s', '# - bytes moved = 229.689 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.42731 GByte/s', '# - solve arithmetic intensity = 0.195013 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.110s; elapsed, 17.132s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.130s; elapsed, 17.147s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.110s; elapsed, 17.132s, #calls: 1', 'TOTAL : CPU, 17.110s; elapsed, 17.132s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 24.790s; elapsed, 24.814s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.179s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.390s; elapsed, 19.405s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.130s; elapsed, 17.147s, #calls: 1', 'TOTAL : CPU, 61.490s; elapsed, 61.544s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 29.670s; elapsed, 14.881s', 'individual call time for EIGEN_LDLT: CPU, 32.380s; elapsed, 32.222s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.176s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,781', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,# - nd time = ', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.217317', '# - matching time = 0.0619159', '# - symmetrization time = 0.0342321', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,781', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.076571', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 160.156 MB', '# - factor time = 9.66358', '# - factor nonzeros = 20,019,517', '# - factor memory = 160.156 MB', '# - total flops = 5.00256e+10, min = 2.29344e+10, max = 2.70912e+10', '# - flop rate = 5.17672 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.23886e+10', '# --------------------------------------------', '# total = 5.23886e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.46879e-11\trel.res = 1.8274e-15\tbw.error = 4.16242e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0888779', '# - total flops = 4.481e+07, min = 2.05478e+07, max = 2.42622e+07', '# - flop rate = 0.504175 GFlop/s', '# - bytes moved = 230.398 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.59229 GByte/s', '# - solve arithmetic intensity = 0.19449 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 19.650s; elapsed, 10.176s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.240s; elapsed, 10.189s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 19.650s; elapsed, 10.176s, #calls: 1', 'TOTAL : CPU, 19.650s; elapsed, 10.176s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 29.670s; elapsed, 14.881s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.176s, #calls: 1', 'time for EIGEN_LDLT: CPU, 32.380s; elapsed, 32.222s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.240s; elapsed, 10.189s, #calls: 1', 'TOTAL : CPU, 82.580s; elapsed, 57.469s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 38.960s; elapsed, 9.794s', 'individual call time for EIGEN_LDLT: CPU, 19.690s; elapsed, 19.168s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.113s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,781', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.211068', '# - matching time = 0.0545471', '# - symmetrization time = 0.0341558', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,781', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0801709', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 160.156 MB', '# - factor time = 9.53991', '# - factor nonzeros = 20,019,517', '# - factor memory = 160.156 MB', '# - total flops = 5.00256e+10, min = 2.29344e+10, max = 2.70912e+10', '# - flop rate = 5.24383 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.23886e+10', '# --------------------------------------------', '# total = 5.23886e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.46826e-11\trel.res = 1.82718e-15\tbw.error = 4.16509e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.096158', '# - total flops = 4.481e+07, min = 2.05478e+07, max = 2.42622e+07', '# - flop rate = 0.466004 GFlop/s', '# - bytes moved = 230.416 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.39622 GByte/s', '# - solve arithmetic intensity = 0.194475 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 34.310s; elapsed, 10.050s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 39.940s; elapsed, 10.065s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 34.310s; elapsed, 10.050s, #calls: 1', 'TOTAL : CPU, 34.310s; elapsed, 10.050s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 38.960s; elapsed, 9.794s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.113s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.690s; elapsed, 19.168s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 39.940s; elapsed, 10.065s, #calls: 1', 'TOTAL : CPU, 98.880s; elapsed, 39.140s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 76.380s; elapsed, 9.621s', 'individual call time for EIGEN_LDLT: CPU, 20.560s; elapsed, 19.329s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.410s; elapsed, 0.093s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,781', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.218543', '# - matching time = 0.057996', '# - symmetrization time = 0.040771', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,781', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0839651', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 160.156 MB', '# - factor time = 9.67384', '# - factor nonzeros = 20,019,517', '# - factor memory = 160.156 MB', '# - total flops = 5.00256e+10, min = 2.29344e+10, max = 2.70912e+10', '# - flop rate = 5.17123 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.23886e+10', '# --------------------------------------------', '# total = 5.23886e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.47005e-11\trel.res = 1.82791e-15\tbw.error = 4.16543e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.121101', '# - total flops = 4.481e+07, min = 2.05478e+07, max = 2.42622e+07', '# - flop rate = 0.370022 GFlop/s', '# - bytes moved = 230.453 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.90298 GByte/s', '# - solve arithmetic intensity = 0.194443 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 67.660s; elapsed, 10.232s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 81.200s; elapsed, 10.251s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 67.660s; elapsed, 10.232s, #calls: 1', 'TOTAL : CPU, 67.660s; elapsed, 10.232s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 76.380s; elapsed, 9.621s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.410s; elapsed, 0.093s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.560s; elapsed, 19.329s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 81.200s; elapsed, 10.251s, #calls: 1', 'TOTAL : CPU, 178.550s; elapsed, 39.294s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 189.190s; elapsed, 11.888s', 'individual call time for EIGEN_LDLT: CPU, 22.050s; elapsed, 19.352s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.710s; elapsed, 0.089s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,781', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.24396', '# - matching time = 0.068167', '# - symmetrization time = 0.0411301', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,781', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0925391', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 160.156 MB', '# - factor time = 10.9859', '# - factor nonzeros = 20,019,517', '# - factor memory = 160.156 MB', '# - total flops = 5.00263e+10, min = 2.29344e+10, max = 2.70919e+10', '# - flop rate = 4.55368 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.23886e+10', '# --------------------------------------------', '# total = 5.23886e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.46636e-11\trel.res = 1.82641e-15\tbw.error = 4.16509e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0951681', '# - total flops = 4.48137e+07, min = 2.05515e+07, max = 2.42622e+07', '# - flop rate = 0.47089 GFlop/s', '# - bytes moved = 230.586 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.42293 GByte/s', '# - solve arithmetic intensity = 0.194347 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 155.330s; elapsed, 11.577s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 183.140s; elapsed, 11.592s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 155.330s; elapsed, 11.577s, #calls: 1', 'TOTAL : CPU, 155.330s; elapsed, 11.577s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 189.190s; elapsed, 11.888s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.710s; elapsed, 0.089s, #calls: 1', 'time for EIGEN_LDLT: CPU, 22.050s; elapsed, 19.352s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 183.140s; elapsed, 11.592s, #calls: 1', 'TOTAL : CPU, 395.090s; elapsed, 42.921s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 25.620s; elapsed, 25.647s', 'individual call time for EIGEN_LDLT: CPU, 19.380s; elapsed, 19.390s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.188s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,763', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.230615', '# - matching time = 0.0568919', '# - symmetrization time = 0.022572', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,763', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0617738', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 162.795 MB', '# - factor time = 9.74542', '# - factor nonzeros = 20,349,413', '# - factor memory = 162.795 MB', '# - total flops = 5.13294e+10, min = 2.45595e+06, max = 2.28929e+10', '# - flop rate = 5.26703 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.36486e+10', '# --------------------------------------------', '# total = 5.36486e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.31158e-11\trel.res = 1.76311e-15\tbw.error = 4.50443e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0648479', '# - total flops = 5.20842e+07, min = 54847, max = 2.42155e+07', '# - flop rate = 0.803175 GFlop/s', '# - bytes moved = 202.404 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.1212 GByte/s', '# - solve arithmetic intensity = 0.257329 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.170s; elapsed, 10.199s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.190s; elapsed, 10.201s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.180s; elapsed, 10.193s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.200s; elapsed, 10.207s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.180s; elapsed, 10.193s, #calls: 1', 'TOTAL : CPU, 10.180s; elapsed, 10.193s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.190s; elapsed, 10.201s, #calls: 1', 'TOTAL : CPU, 10.190s; elapsed, 10.201s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.170s; elapsed, 10.199s, #calls: 1', 'TOTAL : CPU, 10.170s; elapsed, 10.199s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 25.620s; elapsed, 25.647s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.188s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.380s; elapsed, 19.390s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.200s; elapsed, 10.207s, #calls: 1', 'TOTAL : CPU, 55.380s; elapsed, 55.432s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 29.330s; elapsed, 14.733s', 'individual call time for EIGEN_LDLT: CPU, 19.220s; elapsed, 19.045s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.132s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,763', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.229134', '# - matching time = 0.0568249', '# - symmetrization time = 0.0270479', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,763', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.073278', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 162.795 MB', '# - factor time = 9.63242', '# - factor nonzeros = 20,349,413', '# - factor memory = 162.795 MB', '# - total flops = 5.13295e+10, min = 2.45595e+06, max = 2.28929e+10', '# - flop rate = 5.32882 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.36486e+10', '# --------------------------------------------', '# total = 5.36486e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.1948e-11\trel.res = 1.71536e-15\tbw.error = 4.50638e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.066623', '# - total flops = 5.20895e+07, min = 54847, max = 2.42155e+07', '# - flop rate = 0.781856 GFlop/s', '# - bytes moved = 202.615 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.04122 GByte/s', '# - solve arithmetic intensity = 0.257087 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 15.870s; elapsed, 10.106s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.420s; elapsed, 10.109s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.970s; elapsed, 10.102s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.970s; elapsed, 10.116s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.420s; elapsed, 10.109s, #calls: 1', 'TOTAL : CPU, 11.420s; elapsed, 10.109s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.970s; elapsed, 10.102s, #calls: 1', 'TOTAL : CPU, 18.970s; elapsed, 10.102s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 15.870s; elapsed, 10.106s, #calls: 1', 'TOTAL : CPU, 15.870s; elapsed, 10.106s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 29.330s; elapsed, 14.733s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.132s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.220s; elapsed, 19.045s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.970s; elapsed, 10.116s, #calls: 1', 'TOTAL : CPU, 68.730s; elapsed, 44.026s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 39.460s; elapsed, 9.925s', 'individual call time for EIGEN_LDLT: CPU, 19.830s; elapsed, 19.291s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.115s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,763', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.231708', '# - matching time = 0.058223', '# - symmetrization time = 0.0290191', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,763', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0720448', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 162.795 MB', '# - factor time = 9.5864', '# - factor nonzeros = 20,349,413', '# - factor memory = 162.795 MB', '# - total flops = 5.13295e+10, min = 2.45658e+06, max = 2.28929e+10', '# - flop rate = 5.35441 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.36486e+10', '# --------------------------------------------', '# total = 5.36486e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.19742e-11\trel.res = 1.71643e-15\tbw.error = 4.50833e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0679989', '# - total flops = 5.20895e+07, min = 54847, max = 2.42155e+07', '# - flop rate = 0.766035 GFlop/s', '# - bytes moved = 202.628 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.97987 GByte/s', '# - solve arithmetic intensity = 0.25707 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 33.240s; elapsed, 10.061s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 24.920s; elapsed, 10.065s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 13.690s; elapsed, 10.066s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 39.610s; elapsed, 10.073s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 33.240s; elapsed, 10.061s, #calls: 1', 'TOTAL : CPU, 33.240s; elapsed, 10.061s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 13.690s; elapsed, 10.066s, #calls: 1', 'TOTAL : CPU, 13.690s; elapsed, 10.066s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 24.920s; elapsed, 10.065s, #calls: 1', 'TOTAL : CPU, 24.920s; elapsed, 10.065s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 39.460s; elapsed, 9.925s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.115s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.830s; elapsed, 19.291s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 39.610s; elapsed, 10.073s, #calls: 1', 'TOTAL : CPU, 99.200s; elapsed, 39.404s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 88.360s; elapsed, 11.199s', 'individual call time for EIGEN_LDLT: CPU, 20.130s; elapsed, 18.904s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.107s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', 'CPP -2', 'CPP -2', '8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,763', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.251237', '# - matching time = 0.067874', '# - symmetrization time = 0.028312', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,763', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.083693', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 162.795 MB', '# - factor time = 10.3587', '# - factor nonzeros = 20,349,413', '# - factor memory = 162.795 MB', '# - total flops = 5.13295e+10, min = 2.45658e+06, max = 2.28929e+10', '# - flop rate = 4.95521 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.36486e+10', '# --------------------------------------------', '# total = 5.36486e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.19328e-11\trel.res = 1.71474e-15\tbw.error = 4.50833e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0798702', '# - total flops = 5.20895e+07, min = 54847, max = 2.42155e+07', '# - flop rate = 0.652177 GFlop/s', '# - bytes moved = 202.654 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.5373 GByte/s', '# - solve arithmetic intensity = 0.257036 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 48.270s; elapsed, 10.892s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 68.930s; elapsed, 10.888s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 19.640s; elapsed, 10.894s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 85.580s; elapsed, 10.901s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 68.930s; elapsed, 10.888s, #calls: 1', 'TOTAL : CPU, 68.930s; elapsed, 10.888s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 48.270s; elapsed, 10.892s, #calls: 1', 'TOTAL : CPU, 48.270s; elapsed, 10.892s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 19.640s; elapsed, 10.894s, #calls: 1', 'TOTAL : CPU, 19.640s; elapsed, 10.894s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 88.360s; elapsed, 11.199s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.107s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.130s; elapsed, 18.904s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 85.580s; elapsed, 10.901s, #calls: 1', 'TOTAL : CPU, 194.540s; elapsed, 41.110s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 2913.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 25.820s; elapsed, 25.845s', 'individual call time for EIGEN_LDLT: CPU, 18.870s; elapsed, 18.878s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.189s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,761', '# - number of levels = 130', '# - nd time = 0.226691# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '', '# - matching time = 0.0579801', '# - symmetrization time = 0.0214732', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,761', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0543609', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 163.769 MB', '# - factor time = 9.98133', '# - factor nonzeros = 20,471,129', '# - factor memory = 163.769 MB', '# - total flops = 5.19135e+10, min = 2.03243e+06, max = 2.28029e+10', '# - flop rate = 5.20106 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.41139e+10', '# --------------------------------------------', '# total = 5.41139e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.69221e-11\trel.res = 1.50984e-15\tbw.error = 3.89873e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0675311', '# - total flops = 6.8532e+07, min = 38492, max = 2.40848e+07', '# - flop rate = 1.01482 GFlop/s', '# - bytes moved = 190.4 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.81945 GByte/s', '# - solve arithmetic intensity = 0.359936 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 10.390s; elapsed, 10.426s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.390s; elapsed, 10.423s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 10.400s; elapsed, 10.418s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.400s; elapsed, 10.424s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.380s; elapsed, 10.424s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.400s; elapsed, 10.424s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.410s; elapsed, 10.419s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.420s; elapsed, 10.430s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.400s; elapsed, 10.424s, #calls: 1', 'TOTAL : CPU, 10.400s; elapsed, 10.424s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 10.390s; elapsed, 10.426s, #calls: 1', 'TOTAL : CPU, 10.390s; elapsed, 10.426s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 10.400s; elapsed, 10.418s, #calls: 1', 'TOTAL : CPU, 10.400s; elapsed, 10.418s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.410s; elapsed, 10.419s, #calls: 1', 'TOTAL : CPU, 10.410s; elapsed, 10.419s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.400s; elapsed, 10.424s, #calls: 1', 'TOTAL : CPU, 10.400s; elapsed, 10.424s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.390s; elapsed, 10.423s, #calls: 1', 'TOTAL : CPU, 10.390s; elapsed, 10.423s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.380s; elapsed, 10.424s, #calls: 1', 'TOTAL : CPU, 10.380s; elapsed, 10.424s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 25.820s; elapsed, 25.845s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.189s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.870s; elapsed, 18.878s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.420s; elapsed, 10.430s, #calls: 1', 'TOTAL : CPU, 55.300s; elapsed, 55.340s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 2913.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.600s; elapsed, 16.339s', 'individual call time for EIGEN_LDLT: CPU, 19.430s; elapsed, 19.249s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.134s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,761', '# - number of levels = 130', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.251934', '# - matching time = 0.064342', '# - symmetrization time = 0.026175', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,761', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0743341', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 163.769 MB', '# - factor time = 10.1306', '# - factor nonzeros = 20,471,129', '# - factor memory = 163.769 MB', '# - total flops = 5.19135e+10, min = 2.03243e+06, max = 2.28029e+10', '# - flop rate = 5.12445 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.41139e+10', '# --------------------------------------------', '# total = 5.41139e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.71507e-11\trel.res = 1.51918e-15\tbw.error = 3.89716e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0607901', '# - total flops = 6.8532e+07, min = 38492, max = 2.40848e+07', '# - flop rate = 1.12736 GFlop/s', '# - bytes moved = 190.413 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.13231 GByte/s', '# - solve arithmetic intensity = 0.359911 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 12.290s; elapsed, 10.621s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 14.530s; elapsed, 10.632s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 15.630s; elapsed, 10.625s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.860s; elapsed, 10.630s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 18.810s; elapsed, 10.630s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 12.460s; elapsed, 10.630s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 11.690s; elapsed, 10.625s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.970s; elapsed, 10.635s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 12.290s; elapsed, 10.621s, #calls: 1', 'TOTAL : CPU, 12.290s; elapsed, 10.621s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 15.630s; elapsed, 10.625s, #calls: 1', 'TOTAL : CPU, 15.630s; elapsed, 10.625s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 14.530s; elapsed, 10.632s, #calls: 1', 'TOTAL : CPU, 14.530s; elapsed, 10.632s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 18.810s; elapsed, 10.630s, #calls: 1', 'TOTAL : CPU, 18.810s; elapsed, 10.630s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 12.460s; elapsed, 10.630s, #calls: 1', 'TOTAL : CPU, 12.460s; elapsed, 10.630s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 11.690s; elapsed, 10.625s, #calls: 1', 'TOTAL : CPU, 11.690s; elapsed, 10.625s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.860s; elapsed, 10.630s, #calls: 1', 'TOTAL : CPU, 11.860s; elapsed, 10.630s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.600s; elapsed, 16.339s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.134s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.430s; elapsed, 19.249s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.970s; elapsed, 10.635s, #calls: 1', 'TOTAL : CPU, 73.210s; elapsed, 46.358s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 2913.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 42.160s; elapsed, 10.650s', 'individual call time for EIGEN_LDLT: CPU, 21.940s; elapsed, 21.469s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.350s; elapsed, 0.131s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,761', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.250029', '# - matching time = 0.066143', '# - symmetrization time = 0.032233', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,761', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0823579', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 163.769 MB', '# - factor time = 10.4576', '# - factor nonzeros = 20,471,129', '# - factor memory = 163.769 MB', '# - total flops = 5.19135e+10, min = 2.03308e+06, max = 2.28029e+10', '# - flop rate = 4.96419 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.41139e+10', '# --------------------------------------------', '# total = 5.41139e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.71433e-11\trel.res = 1.51888e-15\tbw.error = 3.89716e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0783091', '# - total flops = 6.8532e+07, min = 38492, max = 2.40848e+07', '# - flop rate = 0.875148 GFlop/s', '# - bytes moved = 190.43 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.43178 GByte/s', '# - solve arithmetic intensity = 0.35988 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 26.260s; elapsed, 10.984s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 16.670s; elapsed, 10.991s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 16.060s; elapsed, 10.982s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 21.080s; elapsed, 10.993s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 14.840s; elapsed, 10.991s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 14.370s; elapsed, 10.991s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 36.750s; elapsed, 10.991s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 43.170s; elapsed, 10.997s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 16.670s; elapsed, 10.991s, #calls: 1', 'TOTAL : CPU, 16.670s; elapsed, 10.991s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 26.260s; elapsed, 10.984s, #calls: 1', 'TOTAL : CPU, 26.260s; elapsed, 10.984s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 16.060s; elapsed, 10.982s, #calls: 1', 'TOTAL : CPU, 16.060s; elapsed, 10.982s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 14.840s; elapsed, 10.991s, #calls: 1', 'TOTAL : CPU, 14.840s; elapsed, 10.991s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 21.080s; elapsed, 10.993s, #calls: 1', 'TOTAL : CPU, 21.080s; elapsed, 10.993s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 14.370s; elapsed, 10.991s, #calls: 1', 'TOTAL : CPU, 14.370s; elapsed, 10.991s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 36.750s; elapsed, 10.991s, #calls: 1', 'TOTAL : CPU, 36.750s; elapsed, 10.991s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 42.160s; elapsed, 10.650s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.350s; elapsed, 0.131s, #calls: 1', 'time for EIGEN_LDLT: CPU, 21.940s; elapsed, 21.469s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 43.170s; elapsed, 10.997s, #calls: 1', 'TOTAL : CPU, 107.620s; elapsed, 43.246s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 1456.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 26.610s; elapsed, 26.637s', 'individual call time for EIGEN_LDLT: CPU, 20.100s; elapsed, 20.121s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.191s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,761', '# - number of levels = 129', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.247728', '# - matching time = 0.063144', '# - symmetrization time = 0.0182321', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,761', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0620339', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 163.769 MB', '# - factor time = 10.3851', '# - factor nonzeros = 20,471,129', '# - factor memory = 163.769 MB', '# - total flops = 5.19135e+10, min = 2.03243e+06, max = 2.27913e+10', '# - flop rate = 4.99882 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.39135e+10', '# --------------------------------------------', '# total = 5.39135e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.04218e-11\trel.res = 1.24402e-15\tbw.error = 3.17516e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0472679', '# - total flops = 1.09026e+08, min = 31207, max = 2.39472e+07', '# - flop rate = 2.30654 GFlop/s', '# - bytes moved = 181.908 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.84844 GByte/s', '# - solve arithmetic intensity = 0.599346 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.780s; elapsed, 10.835s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 10.810s; elapsed, 10.838s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 10.780s; elapsed, 10.838s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 10.800s; elapsed, 10.838s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.830s; elapsed, 10.838s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 10.800s; elapsed, 10.831s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 10.770s; elapsed, 10.837s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 10.810s; elapsed, 10.837s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 10.810s; elapsed, 10.832s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.780s; elapsed, 10.837s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 10.770s; elapsed, 10.839s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 10.810s; elapsed, 10.837s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 10.820s; elapsed, 10.834s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.810s; elapsed, 10.838s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.770s; elapsed, 10.837s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.820s; elapsed, 10.843s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 10.810s; elapsed, 10.838s, #calls: 1', 'TOTAL : CPU, 10.810s; elapsed, 10.838s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 10.800s; elapsed, 10.831s, #calls: 1', 'TOTAL : CPU, 10.800s; elapsed, 10.831s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 10.800s; elapsed, 10.838s, #calls: 1', 'TOTAL : CPU, 10.800s; elapsed, 10.838s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.830s; elapsed, 10.838s, #calls: 1', 'TOTAL : CPU, 10.830s; elapsed, 10.838s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 10.770s; elapsed, 10.839s, #calls: 1', 'TOTAL : CPU, 10.770s; elapsed, 10.839s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.780s; elapsed, 10.837s, #calls: 1', 'TOTAL : CPU, 10.780s; elapsed, 10.837s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 10.810s; elapsed, 10.837s, #calls: 1', 'TOTAL : CPU, 10.810s; elapsed, 10.837s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 10.810s; elapsed, 10.837s, #calls: 1', 'TOTAL : CPU, 10.810s; elapsed, 10.837s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 10.770s; elapsed, 10.837s, #calls: 1', 'TOTAL : CPU, 10.770s; elapsed, 10.837s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.770s; elapsed, 10.837s, #calls: 1', 'TOTAL : CPU, 10.770s; elapsed, 10.837s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.780s; elapsed, 10.835s, #calls: 1', 'TOTAL : CPU, 10.780s; elapsed, 10.835s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 10.810s; elapsed, 10.832s, #calls: 1', 'TOTAL : CPU, 10.810s; elapsed, 10.832s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 10.780s; elapsed, 10.838s, #calls: 1', 'TOTAL : CPU, 10.780s; elapsed, 10.838s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 10.820s; elapsed, 10.834s, #calls: 1', 'TOTAL : CPU, 10.820s; elapsed, 10.834s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.810s; elapsed, 10.838s, #calls: 1', 'TOTAL : CPU, 10.810s; elapsed, 10.838s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 26.610s; elapsed, 26.637s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.191s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.100s; elapsed, 20.121s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.820s; elapsed, 10.843s, #calls: 1', 'TOTAL : CPU, 57.710s; elapsed, 57.792s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 1456.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 30.310s; elapsed, 15.226s', 'individual call time for EIGEN_LDLT: CPU, 19.680s; elapsed, 19.512s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.142s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 16 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,761', '# - number of levels = 129', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.266644', '# - matching time = 0.073559', '# - symmetrization time = 0.018481', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,761', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.074456', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 163.769 MB', '# - factor time = 10.0739', '# - factor nonzeros = 20,471,129', '# - factor memory = 163.769 MB', '# - total flops = 5.19135e+10, min = 2.03243e+06, max = 2.27913e+10', '# - flop rate = 5.15326 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.39135e+10', '# --------------------------------------------', '# total = 5.39135e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.01963e-11\trel.res = 1.2348e-15\tbw.error = 3.17394e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.053853', '# - total flops = 1.09026e+08, min = 31207, max = 2.39472e+07', '# - flop rate = 2.0245 GFlop/s', '# - bytes moved = 181.917 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.37803 GByte/s', '# - solve arithmetic intensity = 0.599314 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 13.830s; elapsed, 10.587s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 12.370s; elapsed, 10.583s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 11.620s; elapsed, 10.584s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 11.960s; elapsed, 10.578s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 15.020s; elapsed, 10.579s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 12.210s; elapsed, 10.584s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 12.020s; elapsed, 10.579s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 13.720s; elapsed, 10.585s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 11.770s; elapsed, 10.582s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 11.880s; elapsed, 10.583s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 11.550s; elapsed, 10.584s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 12.070s; elapsed, 10.584s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 18.500s; elapsed, 10.582s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 12.440s; elapsed, 10.584s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 11.620s; elapsed, 10.582s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.860s; elapsed, 10.589s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 13.830s; elapsed, 10.587s, #calls: 1', 'TOTAL : CPU, 13.830s; elapsed, 10.587s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 15.020s; elapsed, 10.579s, #calls: 1', 'TOTAL : CPU, 15.020s; elapsed, 10.579s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 11.550s; elapsed, 10.584s, #calls: 1', 'TOTAL : CPU, 11.550s; elapsed, 10.584s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 11.620s; elapsed, 10.584s, #calls: 1', 'TOTAL : CPU, 11.620s; elapsed, 10.584s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 12.370s; elapsed, 10.583s, #calls: 1', 'TOTAL : CPU, 12.370s; elapsed, 10.583s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 12.020s; elapsed, 10.579s, #calls: 1', 'TOTAL : CPU, 12.020s; elapsed, 10.579s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 12.070s; elapsed, 10.584s, #calls: 1', 'TOTAL : CPU, 12.070s; elapsed, 10.584s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 11.880s; elapsed, 10.583s, #calls: 1', 'TOTAL : CPU, 11.880s; elapsed, 10.583s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 12.210s; elapsed, 10.584s, #calls: 1', 'TOTAL : CPU, 12.210s; elapsed, 10.584s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 11.960s; elapsed, 10.578s, #calls: 1', 'TOTAL : CPU, 11.960s; elapsed, 10.578s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 12.440s; elapsed, 10.584s, #calls: 1', 'TOTAL : CPU, 12.440s; elapsed, 10.584s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 13.720s; elapsed, 10.585s, #calls: 1', 'TOTAL : CPU, 13.720s; elapsed, 10.585s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 11.620s; elapsed, 10.582s, #calls: 1', 'TOTAL : CPU, 11.620s; elapsed, 10.582s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 11.770s; elapsed, 10.582s, #calls: 1', 'TOTAL : CPU, 11.770s; elapsed, 10.582sExiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 18.500s; elapsed, 10.582s, #calls: 1', 'TOTAL : CPU, 18.500s; elapsed, 10.582s', '', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 30.310s; elapsed, 15.226s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.142s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.680s; elapsed, 19.512s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.860s; elapsed, 10.589s, #calls: 1', 'TOTAL : CPU, 71.080s; elapsed, 45.468s']
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-/A_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-/b_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.660s; elapsed, 5.667s', 'individual call time for EIGEN_LDLT: CPU, 2.470s; elapsed, 2.472s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.053s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.660s; elapsed, 5.667s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.053s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.470s; elapsed, 2.472s, #calls: 1', 'TOTAL : CPU, 8.180s; elapsed, 8.191s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.620s; elapsed, 3.811s', 'individual call time for EIGEN_LDLT: CPU, 2.620s; elapsed, 2.446s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.049s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.620s; elapsed, 3.811s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.049s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.620s; elapsed, 2.446s, #calls: 1', 'TOTAL : CPU, 10.320s; elapsed, 6.306s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 9.250s; elapsed, 2.317s', 'individual call time for EIGEN_LDLT: CPU, 5.290s; elapsed, 4.744s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.046s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 9.250s; elapsed, 2.317s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.046s, #calls: 1', 'time for EIGEN_LDLT: CPU, 5.290s; elapsed, 4.744s, #calls: 1', 'TOTAL : CPU, 14.650s; elapsed, 7.107s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 21.680s; elapsed, 2.719s', 'individual call time for EIGEN_LDLT: CPU, 6.450s; elapsed, 5.228s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.041s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 21.680s; elapsed, 2.719s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.041s, #calls: 1', 'time for EIGEN_LDLT: CPU, 6.450s; elapsed, 5.228s, #calls: 1', 'TOTAL : CPU, 28.300s; elapsed, 7.987s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 36.630s; elapsed, 2.297s', 'individual call time for EIGEN_LDLT: CPU, 8.480s; elapsed, 5.838s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.041s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 36.630s; elapsed, 2.297s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.041s, #calls: 1', 'time for EIGEN_LDLT: CPU, 8.480s; elapsed, 5.838s, #calls: 1', 'TOTAL : CPU, 45.400s; elapsed, 8.175s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.930s; elapsed, 5.926s', 'individual call time for EIGEN_LDLT: CPU, 2.460s; elapsed, 2.467s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.053s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,139', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.183663', '# - matching time = 0.032547', '# - symmetrization time = 0.016258', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.043304', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.2791 MB', '# - factor time = 4.05603', '# - factor nonzeros = 6,034,893', '# - factor memory = 48.2791 MB', '# - total flops = 7.11168e+09, min = 2.51306e+09, max = 4.59862e+09', '# - flop rate = 1.75336 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.7315e+09', '# --------------------------------------------', '# total = 7.7315e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.18684e-11\trel.res = 1.36409e-15\tbw.error = 3.68017e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0481272', '# - total flops = 1.45529e+07, min = 6.25352e+06, max = 8.29941e+06', '# - flop rate = 0.302385 GFlop/s', '# - bytes moved = 107.14 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.22619 GByte/s', '# - solve arithmetic intensity = 0.13583 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.390s; elapsed, 4.402s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.410s; elapsed, 4.413s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.390s; elapsed, 4.402s, #calls: 1', 'TOTAL : CPU, 4.390s; elapsed, 4.402s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.930s; elapsed, 5.926s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.053s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.460s; elapsed, 2.467s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.410s; elapsed, 4.413s, #calls: 1', 'TOTAL : CPU, 12.850s; elapsed, 12.860s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.720s; elapsed, 3.860s', 'individual call time for EIGEN_LDLT: CPU, 2.720s; elapsed, 2.550s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.049s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,139', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.189001', '# - matching time = 0.0323069', '# - symmetrization time = 0.0165331', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0420849', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.2791 MB', '# - factor time = 2.23998', '# - factor nonzeros = 6,034,893', '# - factor memory = 48.2791 MB', '# - total flops = 7.11174e+09, min = 2.51306e+09, max = 4.59868e+09', '# - flop rate = 3.17492 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.7315e+09', '# --------------------------------------------', '# total = 7.7315e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.16628e-11\trel.res = 1.3553e-15\tbw.error = 3.67931e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0438221', '# - total flops = 1.45583e+07, min = 6.25352e+06, max = 8.30482e+06', '# - flop rate = 0.332215 GFlop/s', '# - bytes moved = 107.353 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.44976 GByte/s', '# - solve arithmetic intensity = 0.135611 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 5.150s; elapsed, 2.594s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 5.120s; elapsed, 2.604s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 5.150s; elapsed, 2.594s, #calls: 1', 'TOTAL : CPU, 5.150s; elapsed, 2.594s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.720s; elapsed, 3.860s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.049s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.720s; elapsed, 2.550s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 5.120s; elapsed, 2.604s, #calls: 1', 'TOTAL : CPU, 15.640s; elapsed, 9.063s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 9.880s; elapsed, 2.476s', 'individual call time for EIGEN_LDLT: CPU, 3.110s; elapsed, 2.585s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.044s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,139', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.189412', '# - matching time = 0.032264', '# - symmetrization time = 0.0180559', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0430522', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.2791 MB', '# - factor time = 2.12122', '# - factor nonzeros = 6,034,893', '# - factor memory = 48.2791 MB', '# - total flops = 7.11174e+09, min = 2.51306e+09, max = 4.59868e+09', '# - flop rate = 3.35266 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.7315e+09', '# --------------------------------------------', '# total = 7.7315e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.16386e-11\trel.res = 1.35426e-15\tbw.error = 3.67931e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.048625', '# - total flops = 1.45583e+07, min = 6.25352e+06, max = 8.30482e+06', '# - flop rate = 0.2994 GFlop/s', '# - bytes moved = 107.363 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.20798 GByte/s', '# - solve arithmetic intensity = 0.135599 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.840s; elapsed, 2.485s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.290s; elapsed, 2.495s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.840s; elapsed, 2.485s, #calls: 1', 'TOTAL : CPU, 9.840s; elapsed, 2.485s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 9.880s; elapsed, 2.476s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.044s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.110s; elapsed, 2.585s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.290s; elapsed, 2.495s, #calls: 1', 'TOTAL : CPU, 22.390s; elapsed, 7.600s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 18.690s; elapsed, 2.344s', 'individual call time for EIGEN_LDLT: CPU, 4.850s; elapsed, 3.577s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.043s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,139', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.177027', '# - matching time = 0.03299', '# - symmetrization time = 0.020232', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0554481', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.2791 MB', '# - factor time = 2.16919', '# - factor nonzeros = 6,034,893', '# - factor memory = 48.2791 MB', '# - total flops = 7.11175e+09, min = 2.51306e+09, max = 4.59868e+09', '# - flop rate = 3.27853 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.7315e+09', '# --------------------------------------------', '# total = 7.7315e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.167e-11\trel.res = 1.3556e-15\tbw.error = 3.67931e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.058609', '# - total flops = 1.45583e+07, min = 6.25352e+06, max = 8.30482e+06', '# - flop rate = 0.248398 GFlop/s', '# - bytes moved = 107.383 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.8322 GByte/s', '# - solve arithmetic intensity = 0.135573 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 20.290s; elapsed, 2.552s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.480s; elapsed, 2.564s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 20.290s; elapsed, 2.552s, #calls: 1', 'TOTAL : CPU, 20.290s; elapsed, 2.552s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 18.690s; elapsed, 2.344s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.043s, #calls: 1', 'time for EIGEN_LDLT: CPU, 4.850s; elapsed, 3.577s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.480s; elapsed, 2.564s, #calls: 1', 'TOTAL : CPU, 44.170s; elapsed, 8.527s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 38.320s; elapsed, 2.409s', 'individual call time for EIGEN_LDLT: CPU, 5.580s; elapsed, 3.002s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.044s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,139', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.206053', '# - matching time = 0.0501931', '# - symmetrization time = 0.021385', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0629201', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.2791 MB', '# - factor time = 2.13356', '# - factor nonzeros = 6,034,893', '# - factor memory = 48.2791 MB', '# - total flops = 7.11193e+09, min = 2.51306e+09, max = 4.59887e+09', '# - flop rate = 3.33337 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.7315e+09', '# --------------------------------------------', '# total = 7.7315e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.18243e-11\trel.res = 1.36221e-15\tbw.error = 3.74487e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0608232', '# - total flops = 1.45583e+07, min = 6.25352e+06, max = 8.30482e+06', '# - flop rate = 0.239355 GFlop/s', '# - bytes moved = 107.429 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.76625 GByte/s', '# - solve arithmetic intensity = 0.135516 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 40.120s; elapsed, 2.572s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 40.380s; elapsed, 2.576s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 40.120s; elapsed, 2.572s, #calls: 1', 'TOTAL : CPU, 40.120s; elapsed, 2.572s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 38.320s; elapsed, 2.409s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.044s, #calls: 1', 'time for EIGEN_LDLT: CPU, 5.580s; elapsed, 3.002s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 40.380s; elapsed, 2.576s, #calls: 1', 'TOTAL : CPU, 84.590s; elapsed, 8.031s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.300s; elapsed, 6.301s', 'individual call time for EIGEN_LDLT: CPU, 4.790s; elapsed, 4.791s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.056s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,139', '# - number of levels = 131', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.178712', '# - matching time = 0.0305891', '# - symmetrization time = 0.0179451', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0355392', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.2791 MB', '# - factor time = 3.02934', '# - factor nonzeros = 6,034,893', '# - factor memory = 48.2791 MB', '# - total flops = 7.11168e+09, min = 8.93654e+08, max = 2.51306e+09', '# - flop rate = 2.3476 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.71747e+09', '# --------------------------------------------', '# total = 7.71747e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.72392e-11\trel.res = 1.16595e-15\tbw.error = 2.55499e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.041023', '# - total flops = 1.9286e+07, min = 3.82026e+06, max = 6.22438e+06', '# - flop rate = 0.470126 GFlop/s', '# - bytes moved = 94.649 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.30722 GByte/s', '# - solve arithmetic intensity = 0.203763 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.340s; elapsed, 3.348s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.340s; elapsed, 3.343s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.350s; elapsed, 3.353s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.350s; elapsed, 3.356s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.340s; elapsed, 3.348s, #calls: 1', 'TOTAL : CPU, 3.340s; elapsed, 3.348s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.350s; elapsed, 3.353s, #calls: 1', 'TOTAL : CPU, 3.350s; elapsed, 3.353s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.340s; elapsed, 3.343s, #calls: 1', 'TOTAL : CPU, 3.340s; elapsed, 3.343s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.300s; elapsed, 6.301s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.056s, #calls: 1', 'time for EIGEN_LDLT: CPU, 4.790s; elapsed, 4.791s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.350s; elapsed, 3.356s, #calls: 1', 'TOTAL : CPU, 14.500s; elapsed, 14.504s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.750s; elapsed, 3.881s', 'individual call time for EIGEN_LDLT: CPU, 2.750s; elapsed, 2.568s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.050s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,139', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.185792', '# - matching time = 0.033582', '# - symmetrization time = 0.018688', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0437679', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.2791 MB', '# - factor time = 1.90523', '# - factor nonzeros = 6,034,893', '# - factor memory = 48.2791 MB', '# - total flops = 7.11169e+09, min = 8.93654e+08, max = 2.51306e+09', '# - flop rate = 3.73271 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.71747e+09', '# --------------------------------------------', '# total = 7.71747e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.71856e-11\trel.res = 1.16365e-15\tbw.error = 2.55499e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0351601', '# - total flops = 1.9286e+07, min = 3.82026e+06, max = 6.22438e+06', '# - flop rate = 0.548519 GFlop/s', '# - bytes moved = 94.654 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.69209 GByte/s', '# - solve arithmetic intensity = 0.203752 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.720s; elapsed, 2.245s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.450s; elapsed, 2.243s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.960s; elapsed, 2.235s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.470s; elapsed, 2.248s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.450s; elapsed, 2.243s, #calls: 1', 'TOTAL : CPU, 4.450s; elapsed, 2.243s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.960s; elapsed, 2.235s, #calls: 1', 'TOTAL : CPU, 3.960s; elapsed, 2.235s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.720s; elapsed, 2.245s, #calls: 1', 'TOTAL : CPU, 3.720s; elapsed, 2.245s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.750s; elapsed, 3.881s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.050s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.750s; elapsed, 2.568s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.470s; elapsed, 2.248s, #calls: 1', 'TOTAL : CPU, 15.050s; elapsed, 8.747s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 9.470s; elapsed, 2.387s', 'individual call time for EIGEN_LDLT: CPU, 3.740s; elapsed, 3.220s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.054s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,139', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.211134', '# - matching time = 0.0458491', '# - symmetrization time = 0.0200331', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0436361', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.2791 MB', '# - factor time = 2.11703', '# - factor nonzeros = 6,034,893', '# - factor memory = 48.2791 MB', '# - total flops = 7.11169e+09, min = 8.93654e+08, max = 2.51306e+09', '# - flop rate = 3.35927 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.71747e+09', '# --------------------------------------------', '# total = 7.71747e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.71718e-11\trel.res = 1.16306e-15\tbw.error = 2.55499e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0419059', '# - total flops = 1.9286e+07, min = 3.82026e+06, max = 6.22438e+06', '# - flop rate = 0.460221 GFlop/s', '# - bytes moved = 94.6611 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.2589 GByte/s', '# - solve arithmetic intensity = 0.203737 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 6.650s; elapsed, 2.507s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.360s; elapsed, 2.497s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.850s; elapsed, 2.504s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.720s; elapsed, 2.510s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.360s; elapsed, 2.497s, #calls: 1', 'TOTAL : CPU, 8.360s; elapsed, 2.497s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 6.650s; elapsed, 2.507s, #calls: 1', 'TOTAL : CPU, 6.650s; elapsed, 2.507s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.850s; elapsed, 2.504s, #calls: 1', 'TOTAL : CPU, 9.850s; elapsed, 2.504s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 9.470s; elapsed, 2.387s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.054s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.740s; elapsed, 3.220s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.720s; elapsed, 2.510s, #calls: 1', 'TOTAL : CPU, 23.050s; elapsed, 8.170s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 18.120s; elapsed, 2.270s', 'individual call time for EIGEN_LDLT: CPU, 3.720s; elapsed, 2.486s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.042s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,139', '# - number of levels = 131', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.205088', '# - matching time = 0.0374961', '# - symmetrization time = 0.020402', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0452731', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.2791 MB', '# - factor time = 2.12288', '# - factor nonzeros = 6,034,893', '# - factor memory = 48.2791 MB', '# - total flops = 7.11169e+09, min = 8.93654e+08, max = 2.51306e+09', '# - flop rate = 3.35002 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.71747e+09', '# --------------------------------------------', '# total = 7.71747e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.71495e-11\trel.res = 1.1621e-15\tbw.error = 2.55499e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0447309', '# - total flops = 1.9286e+07, min = 3.82026e+06, max = 6.22438e+06', '# - flop rate = 0.431155 GFlop/s', '# - bytes moved = 94.6754 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.11655 GByte/s', '# - solve arithmetic intensity = 0.203706 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 12.370s; elapsed, 2.500s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 15.620s; elapsed, 2.492s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 19.630s; elapsed, 2.498s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.750s; elapsed, 2.504s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 15.620s; elapsed, 2.492s, #calls: 1', 'TOTAL : CPU, 15.620s; elapsed, 2.492s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 19.630s; elapsed, 2.498s, #calls: 1', 'TOTAL : CPU, 19.630s; elapsed, 2.498s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 12.370s; elapsed, 2.500s, #calls: 1', 'TOTAL : CPU, 12.370s; elapsed, 2.500s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 18.120s; elapsed, 2.270s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.042s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.720s; elapsed, 2.486s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.750s; elapsed, 2.504s, #calls: 1', 'TOTAL : CPU, 41.750s; elapsed, 7.302s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 2913.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.290s; elapsed, 6.306s', 'individual call time for EIGEN_LDLT: CPU, 2.640s; elapsed, 2.634s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.066s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 8 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,141', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.210765', '# - matching time = 0.035332', '# - symmetrization time = 0.0226789', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0359058', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.8475 MB', '# - factor time = 2.63446', '# - factor nonzeros = 6,230,935', '# - factor memory = 49.8475 MB', '# - total flops = 7.56445e+09, min = 227903, max = 2.51361e+09', '# - flop rate = 2.87135 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.14808e+09', '# --------------------------------------------', '# total = 8.14808e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.58238e-11\trel.res = 1.5334e-15\tbw.error = 2.79918e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0461409', '# - total flops = 2.27636e+07, min = 23029, max = 6.19462e+06', '# - flop rate = 0.493349 GFlop/s', '# - bytes moved = 95.0808 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.06066 GByte/s', '# - solve arithmetic intensity = 0.239413 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.970s; elapsed, 3.006s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.970s; elapsed, 3.004s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.970s; elapsed, 3.005s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.960s; elapsed, 3.007s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.990s; elapsed, 3.003s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.980s; elapsed, 2.995s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.970s; elapsed, 3.005s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.010s; elapsed, 3.009s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.970s; elapsed, 3.005s, #calls: 1', 'TOTAL : CPU, 2.970s; elapsed, 3.005s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.960s; elapsed, 3.007s, #calls: 1', 'TOTAL : CPU, 2.960s; elapsed, 3.007s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.990s; elapsed, 3.003s, #calls: 1', 'TOTAL : CPU, 2.990s; elapsed, 3.003s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.980s; elapsed, 2.995s, #calls: 1', 'TOTAL : CPU, 2.980s; elapsed, 2.995s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.970s; elapsed, 3.006s, #calls: 1', 'TOTAL : CPU, 2.970s; elapsed, 3.006s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.970s; elapsed, 3.005s, #calls: 1', 'TOTAL : CPU, 2.970s; elapsed, 3.005s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.970s; elapsed, 3.004s, #calls: 1', 'TOTAL : CPU, 2.970s; elapsed, 3.004s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.290s; elapsed, 6.306s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.066s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.640s; elapsed, 2.634s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.010s; elapsed, 3.009s, #calls: 1', 'TOTAL : CPU, 12.010s; elapsed, 12.015s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 2913.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.670s; elapsed, 3.840s', 'individual call time for EIGEN_LDLT: CPU, 2.770s; elapsed, 2.580s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.049s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,141', '# - number of levels = 130', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.186872', '# - matching time = 0.0331202', '# - symmetrization time = 0.0211761', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.039819', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.8475 MB', '# - factor time = 2.29259', '# - factor nonzeros = 6,230,935', '# - factor memory = 49.8475 MB', '# - total flops = 7.56446e+09, min = 227903, max = 2.51361e+09', '# - flop rate = 3.29953 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.14808e+09', '# --------------------------------------------', '# total = 8.14808e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.57235e-11\trel.res = 1.52911e-15\tbw.error = 2.80249e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0422561', '# - total flops = 2.27636e+07, min = 23029, max = 6.19462e+06', '# - flop rate = 0.538705 GFlop/s', '# - bytes moved = 95.089 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.2503 GByte/s', '# - solve arithmetic intensity = 0.239392 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.450s; elapsed, 2.640s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 5.160s; elapsed, 2.640s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 3.670s; elapsed, 2.641s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 4.230s; elapsed, 2.641s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.410s; elapsed, 2.640s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 3.410s; elapsed, 2.641s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 4.390s; elapsed, 2.632s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 5.230s; elapsed, 2.645s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 3.410s; elapsed, 2.641s, #calls: 1', 'TOTAL : CPU, 3.410s; elapsed, 2.641s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 5.160s; elapsed, 2.640s, #calls: 1', 'TOTAL : CPU, 5.160s; elapsed, 2.640s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 4.230s; elapsed, 2.641s, #calls: 1', 'TOTAL : CPU, 4.230s; elapsed, 2.641s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.410s; elapsed, 2.640s, #calls: 1', 'TOTAL : CPU, 3.410s; elapsed, 2.640s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 3.670s; elapsed, 2.641s, #calls: 1', 'TOTAL : CPU, 3.670s; elapsed, 2.641s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 4.390s; elapsed, 2.632s, #calls: 1', 'TOTAL : CPU, 4.390s; elapsed, 2.632s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.450s; elapsed, 2.640s, #calls: 1', 'TOTAL : CPU, 3.450s; elapsed, 2.640s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.670s; elapsed, 3.840s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.049s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.770s; elapsed, 2.580s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 5.230s; elapsed, 2.645s, #calls: 1', 'TOTAL : CPU, 15.740s; elapsed, 9.114s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 2913.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 15.160s; elapsed, 3.819s', 'individual call time for EIGEN_LDLT: CPU, 3.370s; elapsed, 2.847s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.048s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3CPP -2', 'CPP -2', 'CPP -2', '', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,141', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.234132', '# - matching time = 0.0391309', '# - symmetrization time = 0.0235999', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0450461', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.8475 MB', '# - factor time = 2.41375', '# - factor nonzeros = 6,230,935', '# - factor memory = 49.8475 MB', '# - total flops = 7.56446e+09, min = 228737, max = 2.51361e+09', '# - flop rate = 3.13391 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.14808e+09', '# --------------------------------------------', '# total = 8.14808e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.5714e-11\trel.res = 1.5287e-15\tbw.error = 2.80816e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.049768', '# - total flops = 2.27636e+07, min = 23029, max = 6.19462e+06', '# - flop rate = 0.457394 GFlop/s', '# - bytes moved = 95.0992 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.91085 GByte/s', '# - solve arithmetic intensity = 0.239366 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 7.160s; elapsed, 2.833s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 5.950s; elapsed, 2.832s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 5.170s; elapsed, 2.831s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 8.520s; elapsed, 2.822s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 5.370s; elapsed, 2.831s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 5.280s; elapsed, 2.831s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.820s; elapsed, 2.831s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 11.010s; elapsed, 2.836s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 5.170s; elapsed, 2.831s, #calls: 1', 'TOTAL : CPU, 5.170s; elapsed, 2.831s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 8.520s; elapsed, 2.822s, #calls: 1', 'TOTAL : CPU, 8.520s; elapsed, 2.822s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 7.160s; elapsed, 2.833s, #calls: 1', 'TOTAL : CPU, 7.160s; elapsed, 2.833s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 5.950s; elapsed, 2.832s, #calls: 1', 'TOTAL : CPU, 5.950s; elapsed, 2.832s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 5.280s; elapsed, 2.831s, #calls: 1', 'TOTAL : CPU, 5.280s; elapsed, 2.831s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.820s; elapsed, 2.831s, #calls: 1', 'TOTAL : CPU, 10.820s; elapsed, 2.831s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 5.370s; elapsed, 2.831s, #calls: 1', 'TOTAL : CPU, 5.370s; elapsed, 2.831s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 15.160s; elapsed, 3.819s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.048s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.370s; elapsed, 2.847s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 11.010s; elapsed, 2.836s, #calls: 1', 'TOTAL : CPU, 29.660s; elapsed, 9.550s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 1456.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.440s; elapsed, 6.444s', 'individual call time for EIGEN_LDLT: CPU, 2.660s; elapsed, 2.661s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.067s', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,135', '# - number of levels = 129', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.20999', '# - matching time = 0.0366721', '# - symmetrization time = 0.0154288', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,135', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0342059', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.9364 MB', '# - factor time = 2.07525', '# - factor nonzeros = 6,117,051', '# - factor memory = 48.9364 MB', '# - total flops = 7.3195e+09, min = 381751, max = 2.51086e+09', '# - flop rate = 3.52705 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.85078e+09', '# --------------------------------------------', '# total = 7.85078e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.63666e-11\trel.res = 1.12859e-15\tbw.error = 2.13979e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.037606', '# - total flops = 3.59797e+07, min = 18684, max = 6.128e+06', '# - flop rate = 0.956754 GFlop/s', '# - bytes moved = 90.6956 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.41173 GByte/s', '# - solve arithmetic intensity = 0.396708 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 2.410s; elapsed, 2.426s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.390s; elapsed, 2.424s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.380s; elapsed, 2.425s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.410s; elapsed, 2.423s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 2.360s; elapsed, 2.418s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 2.370s; elapsed, 2.426s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 2.380s; elapsed, 2.424s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.360s; elapsed, 2.424s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.380s; elapsed, 2.423s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.350s; elapsed, 2.425s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 2.360s; elapsed, 2.424s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 2.390s; elapsed, 2.421s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 2.420s; elapsed, 2.426s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 2.390s; elapsed, 2.424s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.410s; elapsed, 2.424s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.410s; elapsed, 2.430s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 2.370s; elapsed, 2.426s, #calls: 1', 'TOTAL : CPU, 2.370s; elapsed, 2.426s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 2.410s; elapsed, 2.426s, #calls: 1', 'TOTAL : CPU, 2.410s; elapsed, 2.426s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.380s; elapsed, 2.425s, #calls: 1', 'TOTAL : CPU, 2.380s; elapsed, 2.425s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.350s; elapsed, 2.425s, #calls: 1', 'TOTAL : CPU, 2.350s; elapsed, 2.425s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.360s; elapsed, 2.424s, #calls: 1', 'TOTAL : CPU, 2.360s; elapsed, 2.424s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 2.380s; elapsed, 2.424s, #calls: 1', 'TOTAL : CPU, 2.380s; elapsed, 2.424s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 2.420s; elapsed, 2.426s, #calls: 1', 'TOTAL : CPU, 2.420s; elapsed, 2.426s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 2.360s; elapsed, 2.424s, #calls: 1', 'TOTAL : CPU, 2.360s; elapsed, 2.424s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.390s; elapsed, 2.424s, #calls: 1', 'TOTAL : CPU, 2.390s; elapsed, 2.424s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 2.360s; elapsed, 2.418s, #calls: 1', 'TOTAL : CPU, 2.360s; elapsed, 2.418s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 2.390s; elapsed, 2.421s, #calls: 1', 'TOTAL : CPU, 2.390s; elapsed, 2.421s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.410s; elapsed, 2.423s, #calls: 1', 'TOTAL : CPU, 2.410s; elapsed, 2.423s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 2.390s; elapsed, 2.424s, #calls: 1', 'TOTAL : CPU, 2.390s; elapsed, 2.424s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.380s; elapsed, 2.423s, #calls: 1', 'TOTAL : CPU, 2.380s; elapsed, 2.423s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.410s; elapsed, 2.424s, #calls: 1', 'TOTAL : CPU, 2.410s; elapsed, 2.424s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.440s; elapsed, 6.444s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.067s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.660s; elapsed, 2.661s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.410s; elapsed, 2.430s, #calls: 1', 'TOTAL : CPU, 11.580s; elapsed, 11.601s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 1456.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 8.240s; elapsed, 4.132s', 'individual call time for EIGEN_LDLT: CPU, 2.780s; elapsed, 2.604s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.061s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,135', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# - nd time = # ******************************************************************', '0.220307', '# - matching time = 0.045186', '# - symmetrization time = 0.0156379', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,135', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0388739', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.9364 MB', '# - factor time = 2.13793', '# - factor nonzeros = 6,117,051', '# - factor memory = 48.9364 MB', '# - total flops = 7.3195e+09, min = 381751, max = 2.51086e+09', '# - flop rate = 3.42364 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.85078e+09', '# --------------------------------------------', '# total = 7.85078e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.6397e-11\trel.res = 1.1299e-15\tbw.error = 2.13979e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0360532', '# - total flops = 3.59797e+07, min = 18684, max = 6.128e+06', '# - flop rate = 0.997962 GFlop/s', '# - bytes moved = 90.7028 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.51581 GByte/s', '# - solve arithmetic intensity = 0.396677 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.380s; elapsed, 2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 3.220s; elapsed, 2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 3.490s; elapsed, 2.509s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 3.970s; elapsed, 2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 3.980s; elapsed, 2.514s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.340s; elapsed, 2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.310s; elapsed, 2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 4.880s; elapsed, 2.515s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 3.450s; elapsed, 2.514s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 4.010s; elapsed, 2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 3.240s; elapsed, 2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 4.300s; elapsed, 2.511s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 3.310s; elapsed, 2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 3.330s; elapsed, 2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 3.280s; elapsed, 2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.890s; elapsed, 2.519s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 4.880s; elapsed, 2.515s, #calls: 1', 'TOTAL : CPU, 4.880s; elapsed, 2.515s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 3.970s; elapsed, 2.513s, #calls: 1', 'TOTAL : CPU, 3.970s; elapsed, 2.513s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 3.220s; elapsed, 2.513s, #calls: 1', 'TOTAL : CPU, 3.220s; elapsed, 2.513s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 3.980s; elapsed, 2.514s, #calls: 1', 'TOTAL : CPU, 3.980s; elapsed, 2.514s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 3.310s; elapsed, 2.513s, #calls: 1', 'TOTAL : CPU, 3.310s; elapsed, 2.513s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.380s; elapsed, 2.513s, #calls: 1', 'TOTAL : CPU, 3.380s; elapsed, 2.513s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.340s; elapsed, 2.513s, #calls: 1', 'TOTAL : CPU, 3.340s; elapsed, 2.513s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 3.330s; elapsed, 2.513s, #calls: 1', 'TOTAL : CPU, 3.330s; elapsed, 2.513s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 3.450s; elapsed, 2.514s, #calls: 1', 'TOTAL : CPU, 3.450s; elapsed, 2.514s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 3.240s; elapsed, 2.513s, #calls: 1', 'TOTAL : CPU, 3.240s; elapsed, 2.513s', 'Exiting profiler', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 4.300s; elapsed, 2.511s, #calls: 1', 'TOTAL : CPU, 4.300s; elapsed, 2.511s', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.310s; elapsed, 2.513s, #calls: 1', 'TOTAL : CPU, 3.310s; elapsed, 2.513s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 4.010s; elapsed, 2.513s, #calls: 1', 'TOTAL : CPU, 4.010s; elapsed, 2.513s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 3.490s; elapsed, 2.509s, #calls: 1', 'TOTAL : CPU, 3.490s; elapsed, 2.509s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 3.280s; elapsed, 2.513s, #calls: 1', 'TOTAL : CPU, 3.280s; elapsed, 2.513s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 8.240s; elapsed, 4.132s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.061s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.780s; elapsed, 2.604s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.890s; elapsed, 2.519s, #calls: 1', 'TOTAL : CPU, 15.990s; elapsed, 9.316s']
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-/A_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-/b_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.620s; elapsed, 5.636s', 'individual call time for EIGEN_LDLT: CPU, 2.510s; elapsed, 2.499s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.056s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.620s; elapsed, 5.636s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.056s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.510s; elapsed, 2.499s, #calls: 1', 'TOTAL : CPU, 8.180s; elapsed, 8.191s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.840s; elapsed, 3.420s', 'individual call time for EIGEN_LDLT: CPU, 2.640s; elapsed, 2.463s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.045s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.840s; elapsed, 3.420s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.045s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.640s; elapsed, 2.463s, #calls: 1', 'TOTAL : CPU, 9.560s; elapsed, 5.928s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 9.130s; elapsed, 2.289s', 'individual call time for EIGEN_LDLT: CPU, 3.130s; elapsed, 2.598s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.042s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 9.130s; elapsed, 2.289s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.042s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.130s; elapsed, 2.598s, #calls: 1', 'TOTAL : CPU, 12.360s; elapsed, 4.929s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 19.060s; elapsed, 2.387s', 'individual call time for EIGEN_LDLT: CPU, 6.800s; elapsed, 5.530s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.042s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 19.060s; elapsed, 2.387s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.042s, #calls: 1', 'time for EIGEN_LDLT: CPU, 6.800s; elapsed, 5.530s, #calls: 1', 'TOTAL : CPU, 26.000s; elapsed, 7.959s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 43.340s; elapsed, 2.718s', 'individual call time for EIGEN_LDLT: CPU, 7.950s; elapsed, 5.299s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.038s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 43.340s; elapsed, 2.718s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.038s, #calls: 1', 'time for EIGEN_LDLT: CPU, 7.950s; elapsed, 5.299s, #calls: 1', 'TOTAL : CPU, 51.550s; elapsed, 8.054s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.630s; elapsed, 5.627s', 'individual call time for EIGEN_LDLT: CPU, 2.420s; elapsed, 2.423s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.055s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,927', '# - number of levels = 130', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.161156', '# - matching time = 0.029036', '# - symmetrization time = 0.015054', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,927', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0424871', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.1376 MB', '# - factor time = 3.92202', '# - factor nonzeros = 6,142,201', '# - factor memory = 49.1376 MB', '# - total flops = 7.46369e+09, min = 2.94438e+09, max = 4.51931e+09', '# - flop rate = 1.90302 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.06944e+09', '# --------------------------------------------', '# total = 8.06944e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.6413e-11\trel.res = 1.55862e-15\tbw.error = 3.62098e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0474091', '# - total flops = 1.47533e+07, min = 6.98046e+06, max = 7.77288e+06', '# - flop rate = 0.311193 GFlop/s', '# - bytes moved = 105.81 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.23185 GByte/s', '# - solve arithmetic intensity = 0.139433 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.230s; elapsed, 4.237s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.250s; elapsed, 4.247s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.230s; elapsed, 4.237s, #calls: 1', 'TOTAL : CPU, 4.230s; elapsed, 4.237s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.630s; elapsed, 5.627s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.055s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.420s; elapsed, 2.423s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.250s; elapsed, 4.247s, #calls: 1', 'TOTAL : CPU, 12.350s; elapsed, 12.352s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.090s; elapsed, 3.549s', 'individual call time for EIGEN_LDLT: CPU, 2.760s; elapsed, 2.577s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.046s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processesCPP -2', '', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,927', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.173755', '# - matching time = 0.031755', '# - symmetrization time = 0.0164971', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,927', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0449631', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.1376 MB', '# - factor time = 3.3272', '# - factor nonzeros = 6,142,201', '# - factor memory = 49.1376 MB', '# - total flops = 7.46376e+09, min = 2.94438e+09, max = 4.51938e+09', '# - flop rate = 2.24325 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.06944e+09', '# --------------------------------------------', '# total = 8.06944e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.67124e-11\trel.res = 1.57144e-15\tbw.error = 3.63192e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0456979', '# - total flops = 1.47589e+07, min = 6.98046e+06, max = 7.77839e+06', '# - flop rate = 0.322965 GFlop/s', '# - bytes moved = 106.027 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.32017 GByte/s', '# - solve arithmetic intensity = 0.139199 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 6.790s; elapsed, 3.666s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 7.350s; elapsed, 3.676s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 6.790s; elapsed, 3.666s, #calls: 1', 'TOTAL : CPU, 6.790s; elapsed, 3.666s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.090s; elapsed, 3.549s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.046s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.760s; elapsed, 2.577s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 7.350s; elapsed, 3.676s, #calls: 1', 'TOTAL : CPU, 17.270s; elapsed, 9.848s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 9.840s; elapsed, 2.468s', 'individual call time for EIGEN_LDLT: CPU, 3.070s; elapsed, 2.529s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.043s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using CPP -2', '2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,927', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.177125', '# - matching time = 0.032584', '# - symmetrization time = 0.0162432', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,927', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0469749', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.1376 MB', '# - factor time = 2.17574', '# - factor nonzeros = 6,142,201', '# - factor memory = 49.1376 MB', '# - total flops = 7.46376e+09, min = 2.94438e+09, max = 4.51938e+09', '# - flop rate = 3.43045 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.06944e+09', '# --------------------------------------------', '# total = 8.06944e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.6699e-11\trel.res = 1.57086e-15\tbw.error = 3.63192e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.053381', '# - total flops = 1.47589e+07, min = 6.98046e+06, max = 7.77839e+06', '# - flop rate = 0.276482 GFlop/s', '# - bytes moved = 106.038 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.98644 GByte/s', '# - solve arithmetic intensity = 0.139184 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.090s; elapsed, 2.531s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.090s; elapsed, 2.541s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.090s; elapsed, 2.531s, #calls: 1', 'TOTAL : CPU, 10.090s; elapsed, 2.531s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 9.840s; elapsed, 2.468s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.043s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.070s; elapsed, 2.529s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.090s; elapsed, 2.541s, #calls: 1', 'TOTAL : CPU, 23.100s; elapsed, 7.582s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 19.660s; elapsed, 2.462s', 'individual call time for EIGEN_LDLT: CPU, 7.270s; elapsed, 6.028s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.042s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,927', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.19543', '# - matching time = 0.036809', '# - symmetrization time = 0.019227', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,927', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0593681', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.1376 MB', '# - factor time = 2.39285', '# - factor nonzeros = 6,142,201', '# - factor memory = 49.1376 MB', '# - total flops = 7.46376e+09, min = 2.94438e+09, max = 4.51938e+09', '# - flop rate = 3.1192 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.06944e+09', '# --------------------------------------------', '# total = 8.06944e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.66517e-11\trel.res = 1.56884e-15\tbw.error = 3.63192e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0481601', '# - total flops = 1.47589e+07, min = 6.98046e+06, max = 7.77839e+06', '# - flop rate = 0.306454 GFlop/s', '# - bytes moved = 106.061 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.20225 GByte/s', '# - solve arithmetic intensity = 0.139155 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.970s; elapsed, 2.790s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.140s; elapsed, 2.801s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.970s; elapsed, 2.790s, #calls: 1', 'TOTAL : CPU, 18.970s; elapsed, 2.790s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 19.660s; elapsed, 2.462s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.042s, #calls: 1', 'time for EIGEN_LDLT: CPU, 7.270s; elapsed, 6.028s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.140s; elapsed, 2.801s, #calls: 1', 'TOTAL : CPU, 49.230s; elapsed, 11.333s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 41.400s; elapsed, 2.597s', 'individual call time for EIGEN_LDLT: CPU, 7.940s; elapsed, 5.317s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.040s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,927', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.215598', '# - matching time = 0.0398321', '# - symmetrization time = 0.0203738', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,927', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0602891', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.1376 MB', '# - factor time = 2.16204', '# - factor nonzeros = 6,142,201', '# - factor memory = 49.1376 MB', '# - total flops = 7.46395e+09, min = 2.94438e+09, max = 4.51957e+09', '# - flop rate = 3.45227 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.06944e+09', '# --------------------------------------------', '# total = 8.06944e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.65568e-11\trel.res = 1.56478e-15\tbw.error = 3.57131e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.061873', '# - total flops = 1.47589e+07, min = 6.98046e+06, max = 7.77839e+06', '# - flop rate = 0.238535 GFlop/s', '# - bytes moved = 106.109 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.71494 GByte/s', '# - solve arithmetic intensity = 0.139092 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 40.160s; elapsed, 2.593s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 40.890s; elapsed, 2.603s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 40.160s; elapsed, 2.593s, #calls: 1', 'TOTAL : CPU, 40.160s; elapsed, 2.593s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 41.400s; elapsed, 2.597s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.040s, #calls: 1', 'time for EIGEN_LDLT: CPU, 7.940s; elapsed, 5.317s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 40.890s; elapsed, 2.603s, #calls: 1', 'TOTAL : CPU, 90.500s; elapsed, 10.557s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.250s; elapsed, 6.255s', 'individual call time for EIGEN_LDLT: CPU, 2.640s; elapsed, 2.643s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.060s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,931', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.191609', '# - matching time = 0.0349221', '# - symmetrization time = 0.0150561', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,931', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.035259', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.3058 MB', '# - factor time = 2.74773', '# - factor nonzeros = 6,163,223', '# - factor memory = 49.3058 MB', '# - total flops = 7.53221e+09, min = 865688, max = 2.95202e+09', '# - flop rate = 2.74125 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.13207e+09', '# --------------------------------------------', '# total = 8.13207e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.02608e-11\trel.res = 1.29528e-15\tbw.error = 2.43099e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0377111', '# - total flops = 1.6493e+07, min = 43055, max = 6.92318e+06', '# - flop rate = 0.43735 GFlop/s', '# - bytes moved = 98.2063 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.60417 GByte/s', '# - solve arithmetic intensity = 0.167942 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.050s; elapsed, 3.070s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.060s; elapsed, 3.074s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.050s; elapsed, 3.072s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.070s; elapsed, 3.079s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.050s; elapsed, 3.072s, #calls: 1', 'TOTAL : CPU, 3.050s; elapsed, 3.072s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.050s; elapsed, 3.070s, #calls: 1', 'TOTAL : CPU, 3.050s; elapsed, 3.070s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.060s; elapsed, 3.074s, #calls: 1', 'TOTAL : CPU, 3.060s; elapsed, 3.074s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.250s; elapsed, 6.255s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.060s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.640s; elapsed, 2.643s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.070s; elapsed, 3.079s, #calls: 1', 'TOTAL : CPU, 12.020s; elapsed, 12.037s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.760s; elapsed, 3.888s', 'individual call time for EIGEN_LDLT: CPU, 2.900s; elapsed, 2.718s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.050s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,931', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.# - nd time = ', '# ******************************************************************', '0.187901', '# - matching time = 0.0358179', '# - symmetrization time = 0.0150061', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,931', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0474441', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.3058 MB', '# - factor time = 2.06362', '# - factor nonzeros = 6,163,223', '# - factor memory = 49.3058 MB', '# - total flops = 7.53223e+09, min = 865688, max = 2.95202e+09', '# - flop rate = 3.65001 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.13207e+09', '# --------------------------------------------', '# total = 8.13207e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.01024e-11\trel.res = 1.2885e-15\tbw.error = 2.41537e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0345149', '# - total flops = 1.64951e+07, min = 43055, max = 6.92318e+06', '# - flop rate = 0.477913 GFlop/s', '# - bytes moved = 98.2807 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.84749 GByte/s', '# - solve arithmetic intensity = 0.167837 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.430s; elapsed, 2.405s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.760s; elapsed, 2.406s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.090s; elapsed, 2.410s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.760s; elapsed, 2.414s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.430s; elapsed, 2.405s, #calls: 1', 'TOTAL : CPU, 4.430s; elapsed, 2.405s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.760s; elapsed, 2.406s, #calls: 1', 'TOTAL : CPU, 4.760s; elapsed, 2.406s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.090s; elapsed, 2.410s, #calls: 1', 'TOTAL : CPU, 3.090s; elapsed, 2.410s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.760s; elapsed, 3.888s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.050s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.900s; elapsed, 2.718s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.760s; elapsed, 2.414s, #calls: 1', 'TOTAL : CPU, 15.490s; elapsed, 9.070s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 12.170s; elapsed, 3.051s', 'individual call time for EIGEN_LDLT: CPU, 3.270s; elapsed, 2.743s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.047s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,931', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.20429', '# - matching time = 0.0361559', '# - symmetrization time = 0.015136', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,931', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0423629', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.3058 MB', '# - factor time = 3.18081', '# - factor nonzeros = 6,163,223', '# - factor memory = 49.3058 MB', '# - total flops = 7.53223e+09, min = 866384, max = 2.95202e+09', '# - flop rate = 2.36803 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.13207e+09', '# --------------------------------------------', '# total = 8.13207e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.01136e-11\trel.res = 1.28898e-15\tbw.error = 2.43099e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.04076', '# - total flops = 1.64951e+07, min = 43055, max = 6.92318e+06', '# - flop rate = 0.404689 GFlop/s', '# - bytes moved = 98.2891 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.41141 GByte/s', '# - solve arithmetic intensity = 0.167822 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.240s; elapsed, 3.535s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 5.600s; elapsed, 3.539s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 11.360s; elapsed, 3.536s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 14.010s; elapsed, 3.544s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 5.600s; elapsed, 3.539s, #calls: 1', 'TOTAL : CPU, 5.600s; elapsed, 3.539s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.240s; elapsed, 3.535s, #calls: 1', 'TOTAL : CPU, 9.240s; elapsed, 3.535s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 11.360s; elapsed, 3.536s, #calls: 1', 'TOTAL : CPU, 11.360s; elapsed, 3.536s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 12.170s; elapsed, 3.051s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.047s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.270s; elapsed, 2.743s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 14.010s; elapsed, 3.544s, #calls: 1', 'TOTAL : CPU, 29.560s; elapsed, 9.385s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 18.260s; elapsed, 2.288s', 'individual call time for EIGEN_LDLT: CPU, 3.810s; elapsed, 2.632s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.045s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,931', '# - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.195347', '# - matching time = 0.039413', '# - symmetrization time = 0.026557', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,931', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.045253', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.3058 MB', '# - factor time = 2.50829', '# - factor nonzeros = 6,163,223', '# - factor memory = 49.3058 MB', '# - total flops = 7.53224e+09, min = 866384, max = 2.95202e+09', '# - flop rate = 3.00294 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.13207e+09', '# --------------------------------------------', '# total = 8.13207e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.01093e-11\trel.res = 1.2888e-15\tbw.error = 2.43099e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.048475', '# - total flops = 1.64951e+07, min = 43055, max = 6.92318e+06', '# - flop rate = 0.340281 GFlop/s', '# - bytes moved = 98.3065 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.02798 GByte/s', '# - solve arithmetic intensity = 0.167793 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 7.600s; elapsed, 2.883s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 20.670s; elapsed, 2.882s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.630s; elapsed, 2.883s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.500s; elapsed, 2.890s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 7.600s; elapsed, 2.883s, #calls: 1', 'TOTAL : CPU, 7.600s; elapsed, 2.883s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.630s; elapsed, 2.883s, #calls: 1', 'TOTAL : CPU, 16.630s; elapsed, 2.883s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 20.670s; elapsed, 2.882s, #calls: 1', 'TOTAL : CPU, 20.670s; elapsed, 2.882s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 18.260s; elapsed, 2.288s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.045s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.810s; elapsed, 2.632s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.500s; elapsed, 2.890s, #calls: 1', 'TOTAL : CPU, 44.760s; elapsed, 7.854s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 2663.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.100s; elapsed, 6.102s', 'individual call time for EIGEN_LDLT: CPU, 2.580s; elapsed, 2.580s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.055s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,945', '# - number of levels = 126', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.177324', '# - matching time = 0.0320971', '# - symmetrization time = 0.0155351', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,945', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.033618', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.269 MB', '# - factor time = 2.34061', '# - factor nonzeros = 6,158,621', '# - factor memory = 49.269 MB', '# - total flops = 7.49292e+09, min = 698726, max = 2.95246e+09', '# - flop rate = 3.20126 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.05389e+09', '# --------------------------------------------', '# total = 8.05389e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.57236e-11\trel.res = 1.10107e-15\tbw.error = 2.56229e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0352139', '# - total flops = 2.07556e+07, min = 27135, max = 6.90876e+06', '# - flop rate = 0.589414 GFlop/s', '# - bytes moved = 93.7947 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.66357 GByte/s', '# - solve arithmetic intensity = 0.221288 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.600s; elapsed, 2.647s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.610s; elapsed, 2.644s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.620s; elapsed, 2.640s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.610s; elapsed, 2.646s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.630s; elapsed, 2.647s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.630s; elapsed, 2.646s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.640s; elapsed, 2.647s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.650s; elapsed, 2.652s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.610s; elapsed, 2.644s, #calls: 1', 'TOTAL : CPU, 2.610s; elapsed, 2.644s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.620s; elapsed, 2.640s, #calls: 1', 'TOTAL : CPU, 2.620s; elapsed, 2.640s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.630s; elapsed, 2.647s, #calls: 1', 'TOTAL : CPU, 2.630s; elapsed, 2.647s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.600s; elapsed, 2.647s, #calls: 1', 'TOTAL : CPU, 2.600s; elapsed, 2.647s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.610s; elapsed, 2.646s, #calls: 1', 'TOTAL : CPU, 2.610s; elapsed, 2.646s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.630s; elapsed, 2.646s, #calls: 1', 'TOTAL : CPU, 2.630s; elapsed, 2.646s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.640s; elapsed, 2.647s, #calls: 1', 'TOTAL : CPU, 2.640s; elapsed, 2.647s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.100s; elapsed, 6.102s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.055s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.580s; elapsed, 2.580s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.650s; elapsed, 2.652s, #calls: 1', 'TOTAL : CPU, 11.380s; elapsed, 11.389s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 2663.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.080s; elapsed, 3.546s', 'individual call time for EIGEN_LDLT: CPU, 2.730s; elapsed, 2.552s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.055s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,945', '# - number of levels = 126', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.196749', '# - matching time = 0.0356281', '# - symmetrization time = 0.01385', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,945', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0362802', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.269 MB', '# - factor time = 2.1096', '# - factor nonzeros = 6,158,621', '# - factor memory = 49.269 MB', '# - total flops = 7.49293e+09, min = 698726, max = 2.95246e+09', '# - flop rate = 3.55183 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.05389e+09', '# --------------------------------------------', '# total = 8.05389e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.55859e-11\trel.res = 1.09518e-15\tbw.error = 2.50926e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.034878', '# - total flops = 2.07556e+07, min = 27135, max = 6.90876e+06', '# - flop rate = 0.595091 GFlop/s', '# - bytes moved = 93.8014 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.68941 GByte/s', '# - solve arithmetic intensity = 0.221272 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 3.960s; elapsed, 2.446s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 3.300s; elapsed, 2.446s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 3.230s; elapsed, 2.443s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 4.230s; elapsed, 2.441s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.070s; elapsed, 2.446s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.070s; elapsed, 2.445s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.770s; elapsed, 2.445s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.840s; elapsed, 2.450s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 3.960s; elapsed, 2.446s, #calls: 1', 'TOTAL : CPU, 3.960s; elapsed, 2.446s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 3.300s; elapsed, 2.446s, #calls: 1', 'TOTAL : CPU, 3.300s; elapsed, 2.446s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 4.230s; elapsed, 2.441s, #calls: 1', 'TOTAL : CPU, 4.230s; elapsed, 2.441s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.070s; elapsed, 2.446s, #calls: 1', 'TOTAL : CPU, 3.070s; elapsed, 2.446s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.070s; elapsed, 2.445s, #calls: 1', 'TOTAL : CPU, 3.070s; elapsed, 2.445s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 3.230s; elapsed, 2.443s, #calls: 1', 'TOTAL : CPU, 3.230s; elapsed, 2.443s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.770s; elapsed, 2.445s, #calls: 1', 'TOTAL : CPU, 4.770s; elapsed, 2.445s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.080s; elapsed, 3.546s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.055s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.730s; elapsed, 2.552s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.840s; elapsed, 2.450s, #calls: 1', 'TOTAL : CPU, 14.740s; elapsed, 8.604s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 2663.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 11.350s; elapsed, 2.840s', 'individual call time for EIGEN_LDLT: CPU, 3.660s; elapsed, 3.133s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.059s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,945', '# - number of levels = 126', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.212622', '# - matching time = 0.0406101', '# - symmetrization time = 0.018033', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,945', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0448229', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.269 MB', '# - factor time = 2.62631', '# - factor nonzeros = 6,158,621', '# - factor memory = 49.269 MB', '# - total flops = 7.49293e+09, min = 699330, max = 2.95246e+09', '# - flop rate = 2.85303 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.05389e+09', '# --------------------------------------------', '# total = 8.05389e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.55917e-11\trel.res = 1.09543e-15\tbw.error = 2.50926e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.039459', '# - total flops = 2.07556e+07, min = 27135, max = 6.90876e+06', '# - flop rate = 0.526004 GFlop/s', '# - bytes moved = 93.8099 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.3774 GByte/s', '# - solve arithmetic intensity = 0.221252 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 7.380s; elapsed, 3.003s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 5.020s; elapsed, 3.003s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 5.930s; elapsed, 3.004s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 5.770s; elapsed, 3.000s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 8.870s; elapsed, 2.998s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.500s; elapsed, 3.003s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.940s; elapsed, 3.003s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 11.820s; elapsed, 3.008s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 7.380s; elapsed, 3.003s, #calls: 1', 'TOTAL : CPU, 7.380s; elapsed, 3.003s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 5.020s; elapsed, 3.003s, #calls: 1', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 5.770s; elapsed, 3.000s, #calls: 1', 'TOTAL : CPU, 5.020s; elapsed, 3.003s', 'TOTAL : CPU, 5.770s; elapsed, 3.000s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 5.930s; elapsed, 3.004s, #calls: 1', 'TOTAL : CPU, 5.930s; elapsed, 3.004s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.500s; elapsed, 3.003s, #calls: 1', 'TOTAL : CPU, 10.500s; elapsed, 3.003s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 8.870s; elapsed, 2.998s, #calls: 1', 'TOTAL : CPU, 8.870s; elapsed, 2.998s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.940s; elapsed, 3.003s, #calls: 1', 'TOTAL : CPU, 4.940s; elapsed, 3.003s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 11.350s; elapsed, 2.840s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.059s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.660s; elapsed, 3.133s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 11.820s; elapsed, 3.008s, #calls: 1', 'TOTAL : CPU, 26.940s; elapsed, 9.040s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 1331.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.570s; elapsed, 6.577s', 'individual call time for EIGEN_LDLT: CPU, 2.570s; elapsed, 2.576s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.066s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,953', '# - number of levels = 125', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.178816', '# - matching time = 0.0337219', '# - symmetrization time = 0.0145042', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,953', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0358319', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.1334 MB', '# - factor time = 2.17968', '# - factor nonzeros = 6,141,669', '# - factor memory = 49.1334 MB', '# - total flops = 7.46606e+09, min = 582404, max = 2.95054e+09', '# - flop rate = 3.42531 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.98609e+09', '# --------------------------------------------', '# total = 7.98609e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.31462e-11\trel.res = 9.90751e-16\tbw.error = 2.15469e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0373371', '# - total flops = 3.13629e+07, min = 19616, max = 6.85515e+06', '# - flop rate = 0.839994 GFlop/s', '# - bytes moved = 90.1412 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.41426 GByte/s', '# - solve arithmetic intensity = 0.347931 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.480s; elapsed, 2.491s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.460s; elapsed, 2.494s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.430s; elapsed, 2.493s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 2.460s; elapsed, 2.491s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.460s; elapsed, 2.493s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.450s; elapsed, 2.491s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 2.480s; elapsed, 2.490s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 2.450s; elapsed, 2.489s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 2.470s; elapsed, 2.487s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 2.440s; elapsed, 2.492s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 2.460s; elapsed, 2.493s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.440s; elapsed, 2.492s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 2.490s; elapsed, 2.495s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 2.490s; elapsed, 2.492s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.440s; elapsed, 2.491s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.490s; elapsed, 2.497s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.460s; elapsed, 2.494s, #calls: 1', 'TOTAL : CPU, 2.460s; elapsed, 2.494s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 2.490s; elapsed, 2.495s, #calls: 1', 'TOTAL : CPU, 2.490s; elapsed, 2.495s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.460s; elapsed, 2.493s, #calls: 1', 'TOTAL : CPU, 2.460s; elapsed, 2.493s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.430s; elapsed, 2.493s, #calls: 1', 'TOTAL : CPU, 2.430s; elapsed, 2.493s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 2.470s; elapsed, 2.487s, #calls: 1', 'TOTAL : CPU, 2.470s; elapsed, 2.487s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 2.460s; elapsed, 2.491s, #calls: 1', 'TOTAL : CPU, 2.460s; elapsed, 2.491s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.480s; elapsed, 2.491s, #calls: 1', 'TOTAL : CPU, 2.480s; elapsed, 2.491s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 2.450s; elapsed, 2.489s, #calls: 1', 'TOTAL : CPU, 2.450s; elapsed, 2.489s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.450s; elapsed, 2.491s, #calls: 1', 'TOTAL : CPU, 2.450s; elapsed, 2.491s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.440s; elapsed, 2.492s, #calls: 1', 'TOTAL : CPU, 2.440s; elapsed, 2.492s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 2.480s; elapsed, 2.490s, #calls: 1', 'TOTAL : CPU, 2.480s; elapsed, 2.490s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.440s; elapsed, 2.491s, #calls: 1', 'TOTAL : CPU, 2.440s; elapsed, 2.491s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 2.490s; elapsed, 2.492s, #calls: 1', 'TOTAL : CPU, 2.490s; elapsed, 2.492s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 2.440s; elapsed, 2.492s, #calls: 1', 'TOTAL : CPU, 2.440s; elapsed, 2.492s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 2.460s; elapsed, 2.493s, #calls: 1', 'TOTAL : CPU, 2.460s; elapsed, 2.493s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.570s; elapsed, 6.577s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.066s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.570s; elapsed, 2.576s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.490s; elapsed, 2.497s, #calls: 1', 'TOTAL : CPU, 11.700s; elapsed, 11.716s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 1331.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.730s; elapsed, 3.883s', 'individual call time for EIGEN_LDLT: CPU, 2.860s; elapsed, 2.676s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.063s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 253,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,953', '# - number of levels = 125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.205187', '# - matching time = 0.036953', '# - symmetrization time = 0.016541', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,953', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0437949', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.1334 MB', '# - factor time = 2.32146', '# - factor nonzeros = 6,141,669', '# - factor memory = 49.1334 MB', '# - total flops = 7.46607e+09, min = 582404, max = 2.95054e+09', '# - flop rate = 3.21611 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.98609e+09', '# --------------------------------------------', '# total = 7.98609e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.31206e-11\trel.res = 9.89653e-16\tbw.error = 2.15469e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.039819', '# - total flops = 3.13629e+07, min = 19616, max = 6.85515e+06', '# - flop rate = 0.787637 GFlop/s', '# - bytes moved = 90.1488 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.26397 GByte/s', '# - solve arithmetic intensity = 0.347902 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.350s; elapsed, 2.682s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.400s; elapsed, 2.684s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 3.670s; elapsed, 2.682s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 3.310s; elapsed, 2.682s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 3.810s; elapsed, 2.682s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 3.300s; elapsed, 2.684s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 5.040s; elapsed, 2.683s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.360s; elapsed, 2.682s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 3.550s; elapsed, 2.683s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 3.590s; elapsed, 2.684s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 4.640s; elapsed, 2.679s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 3.630s; elapsed, 2.687s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 3.890s; elapsed, 2.680s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 4.200s; elapsed, 2.686s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 3.960s; elapsed, 2.688s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 5.320s; elapsed, 2.691s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.400s; elapsed, 2.684s, #calls: 1', 'TOTAL : CPU, 3.400s; elapsed, 2.684s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 3.810s; elapsed, 2.682s, #calls: 1', 'TOTAL : CPU, 3.810s; elapsed, 2.682s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 3.300s; elapsed, 2.684s, #calls: 1', 'TOTAL : CPU, 3.300s; elapsed, 2.684s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.350s; elapsed, 2.682s, #calls: 1', 'TOTAL : CPU, 3.350s; elapsed, 2.682s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 3.630s; elapsed, 2.687s, #calls: 1', 'TOTAL : CPU, 3.630s; elapsed, 2.687s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 3.550s; elapsed, 2.683s, #calls: 1', 'TOTAL : CPU, 3.550s; elapsed, 2.683s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 3.670s; elapsed, 2.682s, #calls: 1', 'TOTAL : CPU, 3.670s; elapsed, 2.682s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.360s; elapsed, 2.682s, #calls: 1', 'TOTAL : CPU, 3.360s; elapsed, 2.682s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 3.890s; elapsed, 2.680s, #calls: 1', 'TOTAL : CPU, 3.890s; elapsed, 2.680s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 3.960s; elapsed, 2.688s, #calls: 1', 'TOTAL : CPU, 3.960s; elapsed, 2.688s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 4.640s; elapsed, 2.679s, #calls: 1', 'TOTAL : CPU, 4.640s; elapsed, 2.679s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 5.040s; elapsed, 2.683s, #calls: 1', 'TOTAL : CPU, 5.040s; elapsed, 2.683s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 3.310s; elapsed, 2.682s, #calls: 1', 'TOTAL : CPU, 3.310s; elapsed, 2.682s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 3.590s; elapsed, 2.684s, #calls: 1Exiting profiler', '', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 4.200s; elapsed, 2.686s, #calls: 1', 'TOTAL : CPU, 4.200s; elapsed, 2.686s', 'TOTAL : CPU, 3.590s; elapsed, 2.684s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.730s; elapsed, 3.883s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.063s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.860s; elapsed, 2.676s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 5.320s; elapsed, 2.691s, #calls: 1', 'TOTAL : CPU, 16.000s; elapsed, 9.313s']
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/A_strum_5k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/b_strum_5k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 19.030s; elapsed, 19.054s', 'individual call time for EIGEN_LDLT: CPU, 18.510s; elapsed, 18.517s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.168s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 19.030s; elapsed, 19.054s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.168s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.510s; elapsed, 18.517s, #calls: 1', 'TOTAL : CPU, 37.700s; elapsed, 37.739s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 22.710s; elapsed, 11.372s', 'individual call time for EIGEN_LDLT: CPU, 19.000s; elapsed, 18.829s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.169s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 22.710s; elapsed, 11.372s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.169s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.000s; elapsed, 18.829s, #calls: 1', 'TOTAL : CPU, 41.990s; elapsed, 30.371s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.300s; elapsed, 8.107s', 'individual call time for EIGEN_LDLT: CPU, 19.110s; elapsed, 18.578s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.098s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.300s; elapsed, 8.107s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.098s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.110s; elapsed, 18.578s, #calls: 1', 'TOTAL : CPU, 51.680s; elapsed, 26.784s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 62.960s; elapsed, 7.903s', 'individual call time for EIGEN_LDLT: CPU, 19.810s; elapsed, 18.560s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.086s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 62.960s; elapsed, 7.903s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.086s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.810s; elapsed, 18.560s, #calls: 1', 'TOTAL : CPU, 83.150s; elapsed, 26.549s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 146.930s; elapsed, 9.218s', 'individual call time for EIGEN_LDLT: CPU, 21.240s; elapsed, 18.576s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.740s; elapsed, 0.093s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 146.930s; elapsed, 9.218s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.740s; elapsed, 0.093s, #calls: 1', 'time for EIGEN_LDLT: CPU, 21.240s; elapsed, 18.576s, #calls: 1', 'TOTAL : CPU, 168.910s; elapsed, 27.887s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 19.210s; elapsed, 19.206s', 'individual call time for EIGEN_LDLT: CPU, 18.550s; elapsed, 18.563s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.173s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,179', '# - number of levels = 85', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# - nd time = # ******************************************************************', '0.184921', '# - matching time = 0.054353', '# - symmetrization time = 0.0323091', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,179', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.069479', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 166.554 MB', '# - factor time = 11.46', '# - factor nonzeros = 20,819,237', '# - factor memory = 166.554 MB', '# - total flops = 5.08732e+10, min = 2.52921e+10, max = 2.55811e+10', '# - flop rate = 4.43921 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.24078e+10', '# --------------------------------------------', '# total = 5.24078e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.98285e-11\trel.res = 2.03761e-15\tbw.error = 5.23477e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0683091', '# - total flops = 4.59385e+07, min = 1.93991e+07, max = 2.65394e+07', '# - flop rate = 0.67251 GFlop/s', '# - bytes moved = 213.256 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.12193 GByte/s', '# - solve arithmetic intensity = 0.215415 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.870s; elapsed, 11.890s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 11.900s; elapsed, 11.906s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.870s; elapsed, 11.890s, #calls: 1', 'TOTAL : CPU, 11.870s; elapsed, 11.890s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 19.210s; elapsed, 19.206s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.173s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.550s; elapsed, 18.563s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 11.900s; elapsed, 11.906s, #calls: 1', 'TOTAL : CPU, 49.830s; elapsed, 49.849s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 25.500s; elapsed, 12.799s', 'individual call time for EIGEN_LDLT: CPU, 19.390s; elapsed, 19.225s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.158s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,179', '# - number of levels = 85', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.201731', '# - matching time = 0.057472', '# - symmetrization time = 0.03443', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,179', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0810959', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 166.554 MB', '# - factor time = 9.50401', '# - factor nonzeros = 20,819,237', '# - factor memory = 166.554 MB', '# - total flops = 5.08734e+10, min = 2.52923e+10, max = 2.55811e+10', '# - flop rate = 5.35284 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.24078e+10', '# --------------------------------------------', '# total = 5.24078e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.74997e-11\trel.res = 1.94238e-15\tbw.error = 5.22417e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.074331', '# - total flops = 4.59555e+07, min = 1.94161e+07, max = 2.65394e+07', '# - flop rate = 0.618254 GFlop/s', '# - bytes moved = 213.935 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.87814 GByte/s', '# - solve arithmetic intensity = 0.21481 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.100s; elapsed, 9.981s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.870s; elapsed, 9.997s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.100s; elapsed, 9.981s, #calls: 1', 'TOTAL : CPU, 18.100s; elapsed, 9.981s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 25.500s; elapsed, 12.799s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.158s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.390s; elapsed, 19.225s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.870s; elapsed, 9.997s, #calls: 1', 'TOTAL : CPU, 65.040s; elapsed, 42.179s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 34.770s; elapsed, 8.728s', 'individual call time for EIGEN_LDLT: CPU, 19.750s; elapsed, 19.230s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.350s; elapsed, 0.125s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,179', '# - number of levels = 85', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.199896', '# - matching time = 0.058301', '# - symmetrization time = 0.0341711', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,179', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0828421', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 166.554 MB', '# - factor time = 9.03816', '# - factor nonzeros = 20,819,237', '# - factor memory = 166.554 MB', '# - total flops = 5.08734e+10, min = 2.52923e+10, max = 2.55811e+10', '# - flop rate = 5.62874 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.24078e+10', '# --------------------------------------------', '# total = 5.24078e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.748e-11\trel.res = 1.94158e-15\tbw.error = 5.22417e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.076231', '# - total flops = 4.59555e+07, min = 1.94161e+07, max = 2.65394e+07', '# - flop rate = 0.602845 GFlop/s', '# - bytes moved = 213.953 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.80663 GByte/s', '# - solve arithmetic intensity = 0.214793 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 30.510s; elapsed, 9.518s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 37.820s; elapsed, 9.534s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 30.510s; elapsed, 9.518s, #calls: 1', 'TOTAL : CPU, 30.510s; elapsed, 9.518s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 34.770s; elapsed, 8.728s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.350s; elapsed, 0.125s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.750s; elapsed, 19.230s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 37.820s; elapsed, 9.534s, #calls: 1', 'TOTAL : CPU, 92.690s; elapsed, 37.618s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 64.490s; elapsed, 8.093s', 'individual call time for EIGEN_LDLT: CPU, 24.740s; elapsed, 23.464s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.430s; elapsed, 0.111s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,179', '# - number of levels = 85', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.202268', '# - matching time = 0.0629671', '# - symmetrization time = 0.0384412', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,179', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0920711', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 166.554 MB', '# - factor time = 8.22144', '# - factor nonzeros = 20,819,237', '# - factor memory = 166.554 MB', '# - total flops = 5.08734e+10, min = 2.52923e+10, max = 2.55811e+10', '# - flop rate = 6.18789 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.24078e+10', '# --------------------------------------------', '# total = 5.24078e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.74504e-11\trel.res = 1.94037e-15\tbw.error = 5.22288e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0949609', '# - total flops = 4.59555e+07, min = 1.94161e+07, max = 2.65394e+07', '# - flop rate = 0.483941 GFlop/s', '# - bytes moved = 213.988 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.25343 GByte/s', '# - solve arithmetic intensity = 0.214758 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 57.450s; elapsed, 8.746s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 69.210s; elapsed, 8.761s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 57.450s; elapsed, 8.746s, #calls: 1', 'TOTAL : CPU, 57.450s; elapsed, 8.746s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 64.490s; elapsed, 8.093s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.430s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_LDLT: CPU, 24.740s; elapsed, 23.464s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 69.210s; elapsed, 8.761s, #calls: 1', 'TOTAL : CPU, 158.870s; elapsed, 40.428s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 136.010s; elapsed, 8.543s', 'individual call time for EIGEN_LDLT: CPU, 25.030s; elapsed, 22.292s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.840s; elapsed, 0.109s', 'CPP -2', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,179', '# - number of levels = 85', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.218258', '# - matching time = 0.102226', '# - symmetrization time = 0.042341', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,179', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.124054', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 166.554 MB', '# - factor time = 9.45199', '# - factor nonzeros = 20,819,237', '# - factor memory = 166.554 MB', '# - total flops = 5.08741e+10, min = 2.5293e+10, max = 2.55811e+10', '# - flop rate = 5.38236 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.24078e+10', '# --------------------------------------------', '# total = 5.24078e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.74347e-11\trel.res = 1.93972e-15\tbw.error = 5.22513e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0965509', '# - total flops = 4.59592e+07, min = 1.94198e+07, max = 2.65394e+07', '# - flop rate = 0.47601 GFlop/s', '# - bytes moved = 214.118 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.21767 GByte/s', '# - solve arithmetic intensity = 0.214644 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 117.960s; elapsed, 10.077s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 158.530s; elapsed, 10.085s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 117.960s; elapsed, 10.077s, #calls: 1', 'TOTAL : CPU, 117.960s; elapsed, 10.077s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 136.010s; elapsed, 8.543s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.840s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_LDLT: CPU, 25.030s; elapsed, 22.292s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 158.530s; elapsed, 10.085s, #calls: 1', 'TOTAL : CPU, 320.410s; elapsed, 41.029s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 19.800s; elapsed, 19.822s', 'individual call time for EIGEN_LDLT: CPU, 19.100s; elapsed, 19.110s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.176s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,197', '# - number of levels = 84', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.201171', '# - matching time = 0.0566521', '# - symmetrization time = 0.0360639', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0653849', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 167.054 MB', '# - factor time = 8.48796', '# - factor nonzeros = 20,881,753', '# - factor memory = 167.054 MB', '# - total flops = 5.11312e+10, min = 4.5447e+06, max = 2.57031e+10', '# - flop rate = 6.02396 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.26108e+10', '# --------------------------------------------', '# total = 5.26108e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.81723e-11\trel.res = 1.56096e-15\tbw.error = 4.2043e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.066628', '# - total flops = 5.19342e+07, min = 62299, max = 2.6526e+07', '# - flop rate = 0.779465 GFlop/s', '# - bytes moved = 187.257 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.81049 GByte/s', '# - solve arithmetic intensity = 0.277341 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.900s; elapsed, 8.922s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.910s; elapsed, 8.932s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.920s; elapsed, 8.932s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.930s; elapsed, 8.938s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.920s; elapsed, 8.932s, #calls: 1', 'TOTAL : CPU, 8.920s; elapsed, 8.932s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.910s; elapsed, 8.932s, #calls: 1', 'TOTAL : CPU, 8.910s; elapsed, 8.932s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.900s; elapsed, 8.922s, #calls: 1', 'TOTAL : CPU, 8.900s; elapsed, 8.922s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 19.800s; elapsed, 19.822s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.176s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.100s; elapsed, 19.110s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.930s; elapsed, 8.938s, #calls: 1', 'TOTAL : CPU, 48.000s; elapsed, 48.046s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 23.850s; elapsed, 11.946s', 'individual call time for EIGEN_LDLT: CPU, 19.790s; elapsed, 19.627s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.171s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,197', '# - number of levels = 84', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.202519', '# - matching time = 0.0591428', '# - symmetrization time = 0.0387259', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0718439', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 167.054 MB', '# - factor time = 8.61287', '# - factor nonzeros = 20,881,753', '# - factor memory = 167.054 MB', '# - total flops = 5.11312e+10, min = 4.5447e+06, max = 2.57031e+10', '# - flop rate = 5.93661 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.26108e+10', '# --------------------------------------------', '# total = 5.26108e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.79151e-11\trel.res = 1.55044e-15\tbw.error = 4.15674e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.073236', '# - total flops = 5.19403e+07, min = 62299, max = 2.6526e+07', '# - flop rate = 0.709218 GFlop/s', '# - bytes moved = 187.497 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.56018 GByte/s', '# - solve arithmetic intensity = 0.277019 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.410s; elapsed, 9.086s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 14.510s; elapsed, 9.073s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 15.900s; elapsed, 9.081s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.910s; elapsed, 9.090s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.410s; elapsed, 9.086s, #calls: 1', 'TOTAL : CPU, 10.410s; elapsed, 9.086s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 14.510s; elapsed, 9.073s, #calls: 1', 'TOTAL : CPU, 14.510s; elapsed, 9.073s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 15.900s; elapsed, 9.081s, #calls: 1', 'TOTAL : CPU, 15.900s; elapsed, 9.081s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 23.850s; elapsed, 11.946s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.171s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.790s; elapsed, 19.627s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.910s; elapsed, 9.090s, #calls: 1', 'TOTAL : CPU, 61.850s; elapsed, 40.835s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.710s; elapsed, 8.225s', 'individual call time for EIGEN_LDLT: CPU, 19.240s; elapsed, 18.720s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.330s; elapsed, 0.127s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,197', '# - number of levels = 84', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.218998', '# - matching time = 0.0876551', '# - symmetrization time = 0.0404172', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0862811', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 167.054 MB', '# - factor time = 8.28437', '# - factor nonzeros = 20,881,753', '# - factor memory = 167.054 MB', '# - total flops = 5.11312e+10, min = 4.54523e+06, max = 2.57031e+10', '# - flop rate = 6.17201 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.26108e+10', '# --------------------------------------------', '# total = 5.26108e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.7877e-11\trel.res = 1.54889e-15\tbw.error = 4.15674e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0826938', '# - total flops = 5.19403e+07, min = 62299, max = 2.6526e+07', '# - flop rate = 0.628104 GFlop/s', '# - bytes moved = 187.511 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.26754 GByte/s', '# - solve arithmetic intensity = 0.276998 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 12.700s; elapsed, 8.825s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 27.910s; elapsed, 8.825s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 21.090s; elapsed, 8.818s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 34.630s; elapsed, 8.827s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 12.700s; elapsed, 8.825s, #calls: 1', 'TOTAL : CPU, 12.700s; elapsed, 8.825s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 27.910s; elapsed, 8.825s, #calls: 1', 'TOTAL : CPU, 27.910s; elapsed, 8.825s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 21.090s; elapsed, 8.818s, #calls: 1', 'TOTAL : CPU, 21.090s; elapsed, 8.818s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.710s; elapsed, 8.225s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.330s; elapsed, 0.127s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.240s; elapsed, 18.720s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 34.630s; elapsed, 8.827s, #calls: 1', 'TOTAL : CPU, 86.910s; elapsed, 35.898s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 76.110s; elapsed, 9.602s', 'individual call time for EIGEN_LDLT: CPU, 20.360s; elapsed, 19.127s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.109s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,197', '# - number of levels = 84', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.219856', '# - matching time = 0.0639889', '# - symmetrization time = 0.0398591', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0895171', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 167.054 MB', '# - factor time = 9.04502', '# - factor nonzeros = 20,881,753', '# - factor memory = 167.054 MB', '# - total flops = 5.11312e+10, min = 4.54523e+06, max = 2.57031e+10', '# - flop rate = 5.65297 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.26108e+10', '# --------------------------------------------', '# total = 5.26108e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.7875e-11\trel.res = 1.54881e-15\tbw.error = 4.15674e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0935569', '# - total flops = 5.19403e+07, min = 62299, max = 2.6526e+07', '# - flop rate = 0.555174 GFlop/s', '# - bytes moved = 187.54 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.00455 GByte/s', '# - solve arithmetic intensity = 0.276956 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 40.140s; elapsed, 9.573s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 54.500s; elapsed, 9.583s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.150s; elapsed, 9.582s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 74.930s; elapsed, 9.591s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 54.500s; elapsed, 9.583s, #calls: 1', 'TOTAL : CPU, 54.500s; elapsed, 9.583s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.150s; elapsed, 9.582s, #calls: 1', 'TOTAL : CPU, 18.150s; elapsed, 9.582s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 40.140s; elapsed, 9.573s, #calls: 1', 'TOTAL : CPU, 40.140s; elapsed, 9.573s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 76.110s; elapsed, 9.602s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.360s; elapsed, 19.127s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 74.930s; elapsed, 9.591s, #calls: 1', 'TOTAL : CPU, 171.900s; elapsed, 38.429s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 2663.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 20.310s; elapsed, 20.333s', 'individual call time for EIGEN_LDLT: CPU, 19.440s; elapsed, 19.452s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.171s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,171', '# - number of levels = 84', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.197948', '# - matching time = 0.0574601', '# - symmetrization time = 0.0289381', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0666449', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 166.866 MB', '# - factor time = 8.49045', '# - factor nonzeros = 20,858,227', '# - factor memory = 166.866 MB', '# - total flops = 5.09765e+10, min = 3.60814e+06, max = 2.56131e+10', '# - flop rate = 6.00398 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.23661e+10', '# --------------------------------------------', '# total = 5.23661e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.69923e-11\trel.res = 1.51271e-15\tbw.error = 4.03287e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.061552', '# - total flops = 6.58429e+07, min = 44454, max = 2.64127e+07', '# - flop rate = 1.06971 GFlop/s', '# - bytes moved = 176.451 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.8667 GByte/s', '# - solve arithmetic intensity = 0.373151 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 8.890s; elapsed, 8.919s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.910s; elapsed, 8.921s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.890s; elapsed, 8.920s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.870s; elapsed, 8.919s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 8.900s; elapsed, 8.912s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 8.890s; elapsed, 8.920s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 8.910s; elapsed, 8.919s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.920s; elapsed, 8.927s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.910s; elapsed, 8.921s, #calls: 1', 'TOTAL : CPU, 8.910s; elapsed, 8.921s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.890s; elapsed, 8.920s, #calls: 1', 'TOTAL : CPU, 8.890s; elapsed, 8.920s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 8.910s; elapsed, 8.919s, #calls: 1', 'TOTAL : CPU, 8.910s; elapsed, 8.919s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 8.890s; elapsed, 8.919s, #calls: 1', 'TOTAL : CPU, 8.890s; elapsed, 8.919s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 8.900s; elapsed, 8.912s, #calls: 1', 'TOTAL : CPU, 8.900s; elapsed, 8.912s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 8.890s; elapsed, 8.920s, #calls: 1', 'TOTAL : CPU, 8.890s; elapsed, 8.920s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.870s; elapsed, 8.919s, #calls: 1', 'TOTAL : CPU, 8.870s; elapsed, 8.919s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 20.310s; elapsed, 20.333s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.171s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.440s; elapsed, 19.452s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.920s; elapsed, 8.927s, #calls: 1', 'TOTAL : CPU, 48.840s; elapsed, 48.883s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 2663.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 25.160s; elapsed, 12.626s', 'individual call time for EIGEN_LDLT: CPU, 26.160s; elapsed, 25.992s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.148s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,171', '# - number of levels = 84', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.23073', '# - matching time = 0.0668161', '# - symmetrization time = 0.033504', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0781732', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 166.866 MB', '# - factor time = 8.83507', '# - factor nonzeros = 20,858,227', '# - factor memory = 166.866 MB', '# - total flops = 5.09765e+10, min = 3.60814e+06, max = 2.56131e+10', '# - flop rate = 5.76979 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.23661e+10', '# --------------------------------------------', '# total = 5.23661e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.61403e-11\trel.res = 1.47787e-15\tbw.error = 4.03122e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0719712', '# - total flops = 6.58429e+07, min = 44454, max = 2.64127e+07', '# - flop rate = 0.914851 GFlop/s', '# - bytes moved = 176.462 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.45184 GByte/s', '# - solve arithmetic intensity = 0.373128 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 13.530s; elapsed, 9.340s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.640s; elapsed, 9.342s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.530s; elapsed, 9.341s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 15.100s; elapsed, 9.341s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 11.090s; elapsed, 9.339s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 13.860s; elapsed, 9.331s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.840s; elapsed, 9.341s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.380s; elapsed, 9.346s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.640s; elapsed, 9.342s, #calls: 1', 'TOTAL : CPU, 10.640s; elapsed, 9.342s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 15.100s; elapsed, 9.341s, #calls: 1', 'TOTAL : CPU, 15.100s; elapsed, 9.341s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 13.860s; elapsed, 9.331s, #calls: 1', 'TOTAL : CPU, 13.860s; elapsed, 9.331s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.530s; elapsed, 9.341s, #calls: 1', 'TOTAL : CPU, 10.530s; elapsed, 9.341s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 11.090s; elapsed, 9.339s, #calls: 1', 'TOTAL : CPU, 11.090s; elapsed, 9.339s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.840s; elapsed, 9.341s, #calls: 1', 'TOTAL : CPU, 10.840s; elapsed, 9.341s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 13.530s; elapsed, 9.340s, #calls: 1', 'TOTAL : CPU, 13.530s; elapsed, 9.340s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 25.160s; elapsed, 12.626s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.148s, #calls: 1', 'time for EIGEN_LDLT: CPU, 26.160s; elapsed, 25.992s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.380s; elapsed, 9.346s, #calls: 1', 'TOTAL : CPU, 69.940s; elapsed, 48.112s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 2663.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 33.320s; elapsed, 8.380s', 'individual call time for EIGEN_LDLT: CPU, 19.320s; elapsed, 18.813s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.117s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 8 MPI processes', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,171', '# - number of levels = 84', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.216724', '# - matching time = 0.065331', '# - symmetrization time = 0.0429919', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0828478', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 166.866 MB', '# - factor time = 9.05634', '# - factor nonzeros = 20,858,227', '# - factor memory = 166.866 MB', '# - total flops = 5.09765e+10, min = 3.60864e+06, max = 2.56131e+10', '# - flop rate = 5.62882 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.23661e+10', '# --------------------------------------------', '# total = 5.23661e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.61811e-11\trel.res = 1.47953e-15\tbw.error = 4.03122e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.077822', '# - total flops = 6.58429e+07, min = 44454, max = 2.64127e+07', '# - flop rate = 0.846071 GFlop/s', '# - bytes moved = 176.475 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.26768 GByte/s', '# - solve arithmetic intensity = 0.3731 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 19.660s; elapsed, 9.567s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 13.160s; elapsed, 9.567s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 22.860s; elapsed, 9.557s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 28.120s; elapsed, 9.567s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 15.000s; elapsed, 9.566s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 14.010s; elapsed, 9.568sindividual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 13.450s; elapsed, 9.565s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 37.550s; elapsed, 9.573s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 19.660s; elapsed, 9.567s, #calls: 1', 'TOTAL : CPU, 19.660s; elapsed, 9.567s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 28.120s; elapsed, 9.567s, #calls: 1', 'TOTAL : CPU, 28.120s; elapsed, 9.567s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 13.160s; elapsed, 9.567s, #calls: 1', 'TOTAL : CPU, 13.160s; elapsed, 9.567s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 22.860s; elapsed, 9.557s, #calls: 1', 'TOTAL : CPU, 22.860s; elapsed, 9.557s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 15.000s; elapsed, 9.566s, #calls: 1', 'TOTAL : CPU, 15.000s; elapsed, 9.566s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 13.450s; elapsed, 9.565s, #calls: 1', 'TOTAL : CPU, 13.450s; elapsed, 9.565s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 14.010s; elapsed, 9.568s, #calls: 1', 'TOTAL : CPU, 14.010s; elapsed, 9.568s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 33.320s; elapsed, 8.380s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.117s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.320s; elapsed, 18.813s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 37.550s; elapsed, 9.573s, #calls: 1', 'TOTAL : CPU, 90.480s; elapsed, 36.882s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 1331.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 20.290s; elapsed, 20.316s', 'individual call time for EIGEN_LDLT: CPU, 19.290s; elapsed, 19.309s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.184s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,203', '# - number of levels = 86', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.198698', '# - matching time = 0.057863', '# - symmetrization time = 0.0267661', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,203', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0611751', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 166.06 MB', '# - factor time = 8.29785', '# - factor nonzeros = 20,757,549', '# - factor memory = 166.06 MB', '# - total flops = 5.05896e+10, min = 2.88153e+06, max = 2.55391e+10', '# - flop rate = 6.09671 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.1791e+10', '# --------------------------------------------', '# total = 5.1791e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.49816e-11\trel.res = 1.43049e-15\tbw.error = 3.39388e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.051033', '# - total flops = 1.0266e+08, min = 34210, max = 2.62764e+07', '# - flop rate = 2.01164 GFlop/s', '# - bytes moved = 163.666 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.20706 GByte/s', '# - solve arithmetic intensity = 0.627252 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.650s; elapsed, 8.708s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 8.640s; elapsed, 8.710s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.650s; elapsed, 8.707s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 8.690s; elapsed, 8.707s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 8.640s; elapsed, 8.703s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.690s; elapsed, 8.707s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 8.690s; elapsed, 8.707s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 8.670s; elapsed, 8.705s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 8.690s; elapsed, 8.708s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 8.670s; elapsed, 8.706s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 8.680s; elapsed, 8.707s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 8.650s; elapsed, 8.706s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 8.650s; elapsed, 8.702s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 8.660s; elapsed, 8.702s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 8.670s; elapsed, 8.708s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.700s; elapsed, 8.712s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 8.640s; elapsed, 8.710s, #calls: 1', 'TOTAL : CPU, 8.640s; elapsed, 8.710s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 8.690s; elapsed, 8.708s, #calls: 1', 'TOTAL : CPU, 8.690s; elapsed, 8.708s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 8.680s; elapsed, 8.707s, #calls: 1', 'TOTAL : CPU, 8.680s; elapsed, 8.707s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.650s; elapsed, 8.708s, #calls: 1', 'TOTAL : CPU, 8.650s; elapsed, 8.708s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.650s; elapsed, 8.707s, #calls: 1', 'TOTAL : CPU, 8.650s; elapsed, 8.707s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 8.690s; elapsed, 8.707s, #calls: 1', 'TOTAL : CPU, 8.690s; elapsed, 8.707s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 8.670s; elapsed, 8.706s, #calls: 1', 'TOTAL : CPU, 8.670s; elapsed, 8.706s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 8.660s; elapsed, 8.702s, #calls: 1', 'TOTAL : CPU, 8.660s; elapsed, 8.702s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 8.670s; elapsed, 8.705s, #calls: 1', 'TOTAL : CPU, 8.670s; elapsed, 8.705s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 8.650s; elapsed, 8.702s, #calls: 1', 'TOTAL : CPU, 8.650s; elapsed, 8.702s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 8.690s; elapsed, 8.707s, #calls: 1', 'TOTAL : CPU, 8.690s; elapsed, 8.707s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 8.640s; elapsed, 8.703s, #calls: 1', 'TOTAL : CPU, 8.640s; elapsed, 8.703s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.690s; elapsed, 8.707s, #calls: 1', 'TOTAL : CPU, 8.690s; elapsed, 8.707s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 8.650s; elapsed, 8.706s, #calls: 1', 'TOTAL : CPU, 8.650s; elapsed, 8.706s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 8.670s; elapsed, 8.708s, #calls: 1', 'TOTAL : CPU, 8.670s; elapsed, 8.708s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 20.290s; elapsed, 20.316s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.184s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.290s; elapsed, 19.309s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.700s; elapsed, 8.712s, #calls: 1', 'TOTAL : CPU, 48.470s; elapsed, 48.521s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 1331.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 25.720s; elapsed, 12.928s', 'individual call time for EIGEN_LDLT: CPU, 19.080s; elapsed, 18.903s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.177s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,203', '# - number of levels = 86', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.234193', '# - matching time = 0.0656779', '# - symmetrization time = 0.016855', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,203', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0719039', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 166.06 MB', '# - factor time = 9.1743', '# - factor nonzeros = 20,757,549', '# - factor memory = 166.06 MB', '# - total flops = 5.05896e+10, min = 2.88153e+06, max = 2.55391e+10', '# - flop rate = 5.51427 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.1791e+10', '# --------------------------------------------', '# total = 5.1791e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.27482e-11\trel.res = 1.33915e-15\tbw.error = 3.39267e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0532019', '# - total flops = 1.0266e+08, min = 34210, max = 2.62764e+07', '# - flop rate = 1.92963 GFlop/s', '# - bytes moved = 163.668 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.07636 GByte/s', '# - solve arithmetic intensity = 0.627244 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 11.080s; elapsed, 9.634s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 13.730s; elapsed, 9.633s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 12.730s; elapsed, 9.630s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 12.310s; elapsed, 9.635s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 11.550s; elapsed, 9.630s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 12.700s; elapsed, 9.635s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 11.420s; elapsed, 9.634s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 11.210s; elapsed, 9.635s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.150s; elapsed, 9.635s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.690s; elapsed, 9.635s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 10.640s; elapsed, 9.636s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 11.010s; elapsed, 9.634s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.870s; elapsed, 9.634s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.690s; elapsed, 9.636s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 15.570s; elapsed, 9.634s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.970s; elapsed, 9.641s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 10.640s; elapsed, 9.636s, #calls: 1', 'TOTAL : CPU, 10.640s; elapsed, 9.636s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 12.310s; elapsed, 9.635s, #calls: 1', 'TOTAL : CPU, 12.310s; elapsed, 9.635s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 13.730s; elapsed, 9.633s, #calls: 1', 'TOTAL : CPU, 13.730s; elapsed, 9.633s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 11.080s; elapsed, 9.634s, #calls: 1', 'TOTAL : CPU, 11.080s; elapsed, 9.634s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.690s; elapsed, 9.636s, #calls: 1', 'TOTAL : CPU, 10.690s; elapsed, 9.636s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 11.550s; elapsed, 9.630s, #calls: 1', 'TOTAL : CPU, 11.550s; elapsed, 9.630s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.690s; elapsed, 9.635s, #calls: 1', 'TOTAL : CPU, 10.690s; elapsed, 9.635s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.150s; elapsed, 9.635s, #calls: 1', 'TOTAL : CPU, 11.150s; elapsed, 9.635s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 12.730s; elapsed, 9.630s, #calls: 1', 'TOTAL : CPU, 12.730s; elapsed, 9.630s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 15.570s; elapsed, 9.634s, #calls: 1', 'TOTAL : CPU, 15.570s; elapsed, 9.634s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 11.420s; elapsed, 9.634s, #calls: 1', 'TOTAL : CPU, 11.420s; elapsed, 9.634s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.870s; elapsed, 9.634s, #calls: 1', 'TOTAL : CPU, 10.870s; elapsed, 9.634s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 12.700s; elapsed, 9.635s, #calls: 1', 'TOTAL : CPU, 12.700s; elapsed, 9.635s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 11.010s; elapsed, 9.634s, #calls: 1', 'TOTAL : CPU, 11.010s; elapsed, 9.634s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 11.210s; elapsed, 9.635s, #calls: 1', 'TOTAL : CPU, 11.210s; elapsed, 9.635s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 25.720s; elapsed, 12.928s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.177s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.080s; elapsed, 18.903s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.970s; elapsed, 9.641s, #calls: 1', 'TOTAL : CPU, 64.060s; elapsed, 41.649s']
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/A_strum_5k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/b_strum_5k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 18.250s; elapsed, 18.254s', 'individual call time for EIGEN_LDLT: CPU, 18.950s; elapsed, 18.956s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.166s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 18.250s; elapsed, 18.254s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.166s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.950s; elapsed, 18.956s, #calls: 1', 'TOTAL : CPU, 37.370s; elapsed, 37.376s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 20.970s; elapsed, 10.497s', 'individual call time for EIGEN_LDLT: CPU, 18.790s; elapsed, 18.622s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.139s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 20.970s; elapsed, 10.497s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.139s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.790s; elapsed, 18.622s, #calls: 1', 'TOTAL : CPU, 39.990s; elapsed, 29.259s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 30.470s; elapsed, 7.648s', 'individual call time for EIGEN_LDLT: CPU, 21.480s; elapsed, 20.934s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.115s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 30.470s; elapsed, 7.648s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.115s, #calls: 1', 'time for EIGEN_LDLT: CPU, 21.480s; elapsed, 20.934s, #calls: 1', 'TOTAL : CPU, 52.260s; elapsed, 28.697s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 63.560s; elapsed, 7.969s', 'individual call time for EIGEN_LDLT: CPU, 20.020s; elapsed, 18.759s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.370s; elapsed, 0.089s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 63.560s; elapsed, 7.969s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.370s; elapsed, 0.089s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.020s; elapsed, 18.759s, #calls: 1', 'TOTAL : CPU, 83.950s; elapsed, 26.817s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 147.880s; elapsed, 9.274s', 'individual call time for EIGEN_LDLT: CPU, 21.900s; elapsed, 19.223s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.680s; elapsed, 0.091s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 147.880s; elapsed, 9.274s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.680s; elapsed, 0.091s, #calls: 1', 'time for EIGEN_LDLT: CPU, 21.900s; elapsed, 19.223s, #calls: 1', 'TOTAL : CPU, 170.460s; elapsed, 28.587s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 19.430s; elapsed, 19.454s', 'individual call time for EIGEN_LDLT: CPU, 19.160s; elapsed, 19.163s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.170s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,121', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.16427', '# - matching time = 0.0525141', '# - symmetrization time = 0.029557', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,121', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0703499', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 169.261 MB', '# - factor time = 10.7961', '# - factor nonzeros = 21,157,571', '# - factor memory = 169.261 MB', '# - total flops = 5.06936e+10, min = 2.21744e+10, max = 2.85192e+10', '# - flop rate = 4.69553 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.21012e+10', '# --------------------------------------------', '# total = 5.21012e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.85836e-11\trel.res = 1.98671e-15\tbw.error = 5.28691e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0629048', '# - total flops = 4.62998e+07, min = 1.70328e+07, max = 2.9267e+07', '# - flop rate = 0.73603 GFlop/s', '# - bytes moved = 199.391 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.16973 GByte/s', '# - solve arithmetic intensity = 0.232206 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.160s; elapsed, 11.190s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 11.200s; elapsed, 11.207s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.160s; elapsed, 11.190s, #calls: 1', 'TOTAL : CPU, 11.160s; elapsed, 11.190s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 19.430s; elapsed, 19.454s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.170s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.160s; elapsed, 19.163s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 11.200s; elapsed, 11.207s, #calls: 1', 'TOTAL : CPU, 49.960s; elapsed, 49.994s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 22.810s; elapsed, 11.447s', 'individual call time for EIGEN_LDLT: CPU, 18.790s; elapsed, 18.605s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.139s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,121', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.168501', '# - matching time = 0.055546', '# - symmetrization time = 0.0307961', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,121', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.08587', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 169.261 MB', '# - factor time = 7.64489', '# - factor nonzeros = 21,157,571', '# - factor memory = 169.261 MB', '# - total flops = 5.06937e+10, min = 2.21745e+10, max = 2.85192e+10', '# - flop rate = 6.63106 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.21012e+10', '# --------------------------------------------', '# total = 5.21012e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.89278e-11\trel.res = 2.00078e-15\tbw.error = 5.28761e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0652602', '# - total flops = 4.631e+07, min = 1.7043e+07, max = 2.9267e+07', '# - flop rate = 0.709621 GFlop/s', '# - bytes moved = 199.824 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.06196 GByte/s', '# - solve arithmetic intensity = 0.231754 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 15.460s; elapsed, 8.068s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.090s; elapsed, 8.086s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 15.460s; elapsed, 8.068s, #calls: 1', 'TOTAL : CPU, 15.460s; elapsed, 8.068s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 22.810s; elapsed, 11.447s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.139s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.790s; elapsed, 18.605s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 16.090s; elapsed, 8.086s, #calls: 1', 'TOTAL : CPU, 57.920s; elapsed, 38.278s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 31.290s; elapsed, 7.849s', 'individual call time for EIGEN_LDLT: CPU, 20.380s; elapsed, 19.845s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.105s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,121', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.177059', '# - matching time = 0.057514', '# - symmetrization time = 0.031846', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,121', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.085989', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 169.261 MB', '# - factor time = 7.71083', '# - factor nonzeros = 21,157,571', '# - factor memory = 169.261 MB', '# - total flops = 5.06937e+10, min = 2.21745e+10, max = 2.85192e+10', '# - flop rate = 6.57435 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.21012e+10', '# --------------------------------------------', '# total = 5.21012e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.88728e-11\trel.res = 1.99853e-15\tbw.error = 5.28761e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0790169', '# - total flops = 4.631e+07, min = 1.7043e+07, max = 2.9267e+07', '# - flop rate = 0.586077 GFlop/s', '# - bytes moved = 199.843 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.52911 GByte/s', '# - solve arithmetic intensity = 0.231732 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 27.740s; elapsed, 8.161s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 32.400s; elapsed, 8.178s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 27.740s; elapsed, 8.161s, #calls: 1', 'TOTAL : CPU, 27.740s; elapsed, 8.161s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 31.290s; elapsed, 7.849s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.105s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.380s; elapsed, 19.845s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 32.400s; elapsed, 8.178s, #calls: 1', 'TOTAL : CPU, 84.340s; elapsed, 35.978s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 60.420s; elapsed, 7.580s', 'individual call time for EIGEN_LDLT: CPU, 47.880s; elapsed, 46.635s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.111s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,121', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.179703', '# - matching time = 0.0610681', '# - symmetrization time = 0.03175', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,121', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0946641', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 169.261 MB', '# - factor time = 8.19478', '# - factor nonzeros = 21,157,571', '# - factor memory = 169.261 MB', '# - total flops = 5.06937e+10, min = 2.21745e+10, max = 2.85192e+10', '# - flop rate = 6.1861 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.21012e+10', '# --------------------------------------------', '# total = 5.21012e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.88477e-11\trel.res = 1.99751e-15\tbw.error = 5.28761e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.090426', '# - total flops = 4.631e+07, min = 1.7043e+07, max = 2.9267e+07', '# - flop rate = 0.512132 GFlop/s', '# - bytes moved = 199.881 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.21044 GByte/s', '# - solve arithmetic intensity = 0.231688 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 48.280s; elapsed, 8.674s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 68.810s; elapsed, 8.691s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 48.280s; elapsed, 8.674s, #calls: 1', 'TOTAL : CPU, 48.280s; elapsed, 8.674s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 60.420s; elapsed, 7.580s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_LDLT: CPU, 47.880s; elapsed, 46.635s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 68.810s; elapsed, 8.691s, #calls: 1', 'TOTAL : CPU, 177.560s; elapsed, 63.017s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 136.650s; elapsed, 8.573s', 'individual call time for EIGEN_LDLT: CPU, 21.740s; elapsed, 18.950s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.650s; elapsed, 0.088s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,121', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.260934', '# - matching time = 0.069468', '# - symmetrization time = 0.0519772', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,121', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.107315', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 169.261 MB', '# - factor time = 8.24165', '# - factor nonzeros = 21,157,571', '# - factor memory = 169.261 MB', '# - total flops = 5.06943e+10, min = 2.21751e+10, max = 2.85192e+10', '# - flop rate = 6.15099 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.21012e+10', '# --------------------------------------------', '# total = 5.21012e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.88318e-11\trel.res = 1.99685e-15\tbw.error = 5.28757e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0972271', '# - total flops = 4.63137e+07, min = 1.70467e+07, max = 2.9267e+07', '# - flop rate = 0.476346 GFlop/s', '# - bytes moved = 200.018 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.05723 GByte/s', '# - solve arithmetic intensity = 0.231548 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 103.900s; elapsed, 8.850s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 138.910s; elapsed, 8.871s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 103.900s; elapsed, 8.850s, #calls: 1', 'TOTAL : CPU, 103.900s; elapsed, 8.850s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 136.650s; elapsed, 8.573s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.650s; elapsed, 0.088s, #calls: 1', 'time for EIGEN_LDLT: CPU, 21.740s; elapsed, 18.950s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 138.910s; elapsed, 8.871s, #calls: 1', 'TOTAL : CPU, 297.950s; elapsed, 36.483s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 19.140s; elapsed, 19.175s', 'individual call time for EIGEN_LDLT: CPU, 18.890s; elapsed, 18.901s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.167s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 67', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.164316', '# - matching time = 0.0521531', '# - symmetrization time = 0.040143', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.062979', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 170.719 MB', '# - factor time = 7.6457', '# - factor nonzeros = 21,339,853', '# - factor memory = 170.719 MB', '# - total flops = 5.15107e+10, min = 1.48212e+07, max = 2.85626e+10', '# - flop rate = 6.73721 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.2862e+10', '# --------------------------------------------', '# total = 5.2862e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.69603e-11\trel.res = 1.5114e-15\tbw.error = 4.67637e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0821559', '# - total flops = 5.12613e+07, min = 105197, max = 2.92123e+07', '# - flop rate = 0.623952 GFlop/s', '# - bytes moved = 180.23 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.19375 GByte/s', '# - solve arithmetic intensity = 0.284422 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.060s; elapsed, 8.068s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.050s; elapsed, 8.067s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.040s; elapsed, 8.054s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.060s; elapsed, 8.073s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.040s; elapsed, 8.054s, #calls: 1', 'TOTAL : CPU, 8.040s; elapsed, 8.054s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.060s; elapsed, 8.068s, #calls: 1', 'TOTAL : CPU, 8.060s; elapsed, 8.068s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.050s; elapsed, 8.067s, #calls: 1', 'TOTAL : CPU, 8.050s; elapsed, 8.067s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 19.140s; elapsed, 19.175s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.167s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.890s; elapsed, 18.901s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.060s; elapsed, 8.073s, #calls: 1', 'TOTAL : CPU, 46.260s; elapsed, 46.316s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 22.210s; elapsed, 11.126s', 'individual call time for EIGEN_LDLT: CPU, 19.840s; elapsed, 19.676s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.161s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.173896', '# - matching time = 0.058037', '# - symmetrization time = 0.0407491', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0751748', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 170.719 MB', '# - factor time = 8.44203', '# - factor nonzeros = 21,339,853', '# - factor memory = 170.719 MB', '# - total flops = 5.15107e+10, min = 1.48212e+07, max = 2.85626e+10', '# - flop rate = 6.1017 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.2862e+10', '# --------------------------------------------', '# total = 5.2862e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.69771e-11\trel.res = 1.51208e-15\tbw.error = 4.67141e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0735519', '# - total flops = 5.12668e+07, min = 105197, max = 2.92123e+07', '# - flop rate = 0.697016 GFlop/s', '# - bytes moved = 180.446 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.45331 GByte/s', '# - solve arithmetic intensity = 0.284112 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 13.800s; elapsed, 8.872s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.210s; elapsed, 8.883s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 14.880s; elapsed, 8.883s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.510s; elapsed, 8.889s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 13.800s; elapsed, 8.872s, #calls: 1', 'TOTAL : CPU, 13.800s; elapsed, 8.872s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 14.880s; elapsed, 8.883s, #calls: 1', 'TOTAL : CPU, 14.880s; elapsed, 8.883s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.210s; elapsed, 8.883s, #calls: 1', 'TOTAL : CPU, 10.210s; elapsed, 8.883s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 22.210s; elapsed, 11.126s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.161s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.840s; elapsed, 19.676s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.510s; elapsed, 8.889s, #calls: 1', 'TOTAL : CPU, 59.830s; elapsed, 39.852s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.160s; elapsed, 8.075s', 'individual call time for EIGEN_LDLT: CPU, 19.990s; elapsed, 19.470s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.111s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.177271', '# - matching time = 0.0589108', '# - symmetrization time = 0.0420101', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0785232', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 170.719 MB', '# - factor time = 7.71124', '# - factor nonzeros = 21,339,853', '# - factor memory = 170.719 MB', '# - total flops = 5.15107e+10, min = 1.48226e+07, max = 2.85626e+10', '# - flop rate = 6.67996 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.2862e+10', '# --------------------------------------------', '# total = 5.2862e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.69687e-11\trel.res = 1.51174e-15\tbw.error = 4.67141e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.074775', '# - total flops = 5.12668e+07, min = 105197, max = 2.92123e+07', '# - flop rate = 0.685615 GFlop/s', '# - bytes moved = 180.456 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.41332 GByte/s', '# - solve arithmetic intensity = 0.284096 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 20.440s; elapsed, 8.155s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 25.580s; elapsed, 8.169s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 12.080s; elapsed, 8.170s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 32.060s; elapsed, 8.176s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 20.440s; elapsed, 8.155s, #calls: 1', 'TOTAL : CPU, 20.440s; elapsed, 8.155s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 12.080s; elapsed, 8.170s, #calls: 1', 'TOTAL : CPU, 12.080s; elapsed, 8.170s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 25.580s; elapsed, 8.169s, #calls: 1', 'TOTAL : CPU, 25.580s; elapsed, 8.169s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.160s; elapsed, 8.075s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.990s; elapsed, 19.470s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 32.060s; elapsed, 8.176s, #calls: 1', 'TOTAL : CPU, 84.520s; elapsed, 35.832s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 66.870s; elapsed, 8.440s', 'individual call time for EIGEN_LDLT: CPU, 20.330s; elapsed, 19.038s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.420s; elapsed, 0.097s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6CPP -2', 'CPP -2', 'CPP -2', ' = log_2(#threads) + 3', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,131', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.192302', '# - matching time = 0.063755', '# - symmetrization time = 0.05035', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,131', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0919352', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 170.719 MB', '# - factor time = 8.1816', '# - factor nonzeros = 21,339,853', '# - factor memory = 170.719 MB', '# - total flops = 5.15107e+10, min = 1.48228e+07, max = 2.85626e+10', '# - flop rate = 6.29592 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.2862e+10', '# --------------------------------------------', '# total = 5.2862e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.694e-11\trel.res = 1.51057e-15\tbw.error = 4.67141e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.092495', '# - total flops = 5.12668e+07, min = 105197, max = 2.92123e+07', '# - flop rate = 0.554266 GFlop/s', '# - bytes moved = 180.478 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.95122 GByte/s', '# - solve arithmetic intensity = 0.284062 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 37.520s; elapsed, 8.690s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.110s; elapsed, 8.702s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 52.690s; elapsed, 8.703s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 67.910s; elapsed, 8.709s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 37.520s; elapsed, 8.690s, #calls: 1', 'TOTAL : CPU, 37.520s; elapsed, 8.690s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 52.690s; elapsed, 8.703s, #calls: 1', 'TOTAL : CPU, 52.690s; elapsed, 8.703s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.110s; elapsed, 8.702s, #calls: 1', 'TOTAL : CPU, 18.110s; elapsed, 8.702s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 66.870s; elapsed, 8.440s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.420s; elapsed, 0.097s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.330s; elapsed, 19.038s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 67.910s; elapsed, 8.709s, #calls: 1', 'TOTAL : CPU, 155.530s; elapsed, 36.284s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 2413.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 20.060s; elapsed, 20.089s', 'individual call time for EIGEN_LDLT: CPU, 19.100s; elapsed, 19.118s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.176s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,143', '# - number of levels = 66', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.17137', '# - matching time = 0.057636', '# - symmetrization time = 0.025872', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,143', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0622001', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 168.522 MB', '# - factor time = 7.60595', '# - factor nonzeros = 21,065,199', '# - factor memory = 168.522 MB', '# - total flops = 5.03174e+10, min = 1.50222e+07, max = 2.85789e+10', '# - flop rate = 6.61554 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.15536e+10', '# --------------------------------------------', '# total = 5.15536e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.42119e-11\trel.res = 1.80794e-15\tbw.error = 4.5859e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0577991', '# - total flops = 6.23363e+07, min = 92713, max = 2.90586e+07', '# - flop rate = 1.0785 GFlop/s', '# - bytes moved = 166.966 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.88873 GByte/s', '# - solve arithmetic intensity = 0.373348 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 7.950s; elapsed, 7.995s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 7.990s; elapsed, 7.993s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 7.970s; elapsed, 7.995s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 7.940s; elapsed, 7.990s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 7.960s; elapsed, 7.988s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 7.960s; elapsed, 7.994s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 7.970s; elapsed, 7.996s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.000s; elapsed, 8.000s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 7.950s; elapsed, 7.995s, #calls: 1', 'TOTAL : CPU, 7.950s; elapsed, 7.995s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 7.960s; elapsed, 7.988s, #calls: 1', 'TOTAL : CPU, 7.960s; elapsed, 7.988s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 7.990s; elapsed, 7.993s, #calls: 1', 'TOTAL : CPU, 7.990s; elapsed, 7.993s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 7.940s; elapsed, 7.990s, #calls: 1', 'TOTAL : CPU, 7.940s; elapsed, 7.990s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 7.970s; elapsed, 7.995s, #calls: 1', 'TOTAL : CPU, 7.970s; elapsed, 7.995s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 7.970s; elapsed, 7.996s, #calls: 1', 'TOTAL : CPU, 7.970s; elapsed, 7.996s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 7.960s; elapsed, 7.994s, #calls: 1', 'TOTAL : CPU, 7.960s; elapsed, 7.994s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 20.060s; elapsed, 20.089s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.176s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.100s; elapsed, 19.118s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.000s; elapsed, 8.000s, #calls: 1', 'TOTAL : CPU, 47.340s; elapsed, 47.384s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 2413.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 23.360s; elapsed, 11.695s', 'individual call time for EIGEN_LDLT: CPU, 22.940s; elapsed, 22.776s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.148s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,143', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.196844', '# - matching time = 0.063663', '# - symmetrization time = 0.0264111', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,143', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0806272', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 168.522 MB', '# - factor time = 7.695', '# - factor nonzeros = 21,065,199', '# - factor memory = 168.522 MB', '# - total flops = 5.03174e+10, min = 1.50222e+07, max = 2.85789e+10', '# - flop rate = 6.53898 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.15536e+10', '# --------------------------------------------', '# total = 5.15536e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.40836e-11\trel.res = 1.80269e-15\tbw.error = 4.58556e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0623629', '# - total flops = 6.23363e+07, min = 92713, max = 2.90586e+07', '# - flop rate = 0.999574 GFlop/s', '# - bytes moved = 166.973 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.67743 GByte/s', '# - solve arithmetic intensity = 0.373333 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 9.780s; elapsed, 8.141s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.620s; elapsed, 8.141s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.440s; elapsed, 8.141s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.300s; elapsed, 8.141s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 11.860s; elapsed, 8.137s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 13.510s; elapsed, 8.141s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 11.610s; elapsed, 8.133s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 15.960s; elapsed, 8.146s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 11.860s; elapsed, 8.137s, #calls: 1', 'TOTAL : CPU, 11.860s; elapsed, 8.137s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 9.780s; elapsed, 8.141s, #calls: 1', 'TOTAL : CPU, 9.780s; elapsed, 8.141s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.440s; elapsed, 8.141s, #calls: 1', 'TOTAL : CPU, 9.440s; elapsed, 8.141s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.300s; elapsed, 8.141s, #calls: 1', 'TOTAL : CPU, 9.300s; elapsed, 8.141s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 13.510s; elapsed, 8.141s, #calls: 1', 'TOTAL : CPU, 13.510s; elapsed, 8.141s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 11.610s; elapsed, 8.133s, #calls: 1', 'TOTAL : CPU, 11.610s; elapsed, 8.133s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.620s; elapsed, 8.141s, #calls: 1', 'TOTAL : CPU, 9.620s; elapsed, 8.141s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 23.360s; elapsed, 11.695s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.148s, #calls: 1', 'time for EIGEN_LDLT: CPU, 22.940s; elapsed, 22.776s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 15.960s; elapsed, 8.146s, #calls: 1', 'TOTAL : CPU, 62.490s; elapsed, 42.765s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 2413.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 31.040s; elapsed, 7.798s', 'individual call time for EIGEN_LDLT: CPU, 20.530s; elapsed, 20.013s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.111s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,143', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.206015', '# - matching time = 0.065202', '# - symmetrization time = 0.0305159', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,143', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0826271', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 168.522 MB', '# - factor time = 8.08654', '# - factor nonzeros = 21,065,199', '# - factor memory = 168.522 MB', '# - total flops = 5.03174e+10, min = 1.50234e+07, max = 2.85789e+10', '# - flop rate = 6.22237 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.15536e+10', '# --------------------------------------------', '# total = 5.15536e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.40619e-11\trel.res = 1.8018e-15\tbw.error = 4.58422e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.074604', '# - total flops = 6.23363e+07, min = 92713, max = 2.90586e+07', '# - flop rate = 0.835563 GFlop/s', '# - bytes moved = 166.981 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.23824 GByte/s', '# - solve arithmetic intensity = 0.373313 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 21.020s; elapsed, 8.562s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.920s; elapsed, 8.560s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 26.440s; elapsed, 8.568s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 13.760s; elapsed, 8.569s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 13.410s; elapsed, 8.571s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 12.760s; elapsed, 8.570s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 12.230s; elapsed, 8.567s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 33.510s; elapsed, 8.574s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 13.410s; elapsed, 8.571s, #calls: 1', 'TOTAL : CPU, 13.410s; elapsed, 8.571s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 21.020s; elapsed, 8.562s, #calls: 1', 'TOTAL : CPU, 21.020s; elapsed, 8.562s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 17.920s; elapsed, 8.560s, #calls: 1', 'TOTAL : CPU, 17.920s; elapsed, 8.560s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 26.440s; elapsed, 8.568s, #calls: 1', 'TOTAL : CPU, 26.440s; elapsed, 8.568s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 13.760s; elapsed, 8.569s, #calls: 1', 'TOTAL : CPU, 13.760s; elapsed, 8.569s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 12.230s; elapsed, 8.567s, #calls: 1', 'TOTAL : CPU, 12.230s; elapsed, 8.567s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 12.760s; elapsed, 8.570s, #calls: 1', 'TOTAL : CPU, 12.760s; elapsed, 8.570s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 31.040s; elapsed, 7.798s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.530s; elapsed, 20.013s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 33.510s; elapsed, 8.574s, #calls: 1', 'TOTAL : CPU, 85.350s; elapsed, 36.496s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 1206.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 20.160s; elapsed, 20.186s', 'individual call time for EIGEN_LDLT: CPU, 19.820s; elapsed, 19.836s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.178s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,143', '# - number of levels = 65', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.175043', '# - matching time = 0.0574579', '# - symmetrization time = 0.014714', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,143', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.06039', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 168.522 MB', '# - factor time = 8.54775', '# - factor nonzeros = 21,065,199', '# - factor memory = 168.522 MB', '# - total flops = 5.03174e+10, min = 1.38671e+07, max = 2.85337e+10', '# - flop rate = 5.88662 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.13241e+10', '# --------------------------------------------', '# total = 5.13241e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.22003e-11\trel.res = 1.31675e-15\tbw.error = 3.37627e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.054352', '# - total flops = 9.15661e+07, min = 86678, max = 2.87842e+07', '# - flop rate = 1.68469 GFlop/s', '# - bytes moved = 158.412 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.91455 GByte/s', '# - solve arithmetic intensity = 0.578027 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 8.900s; elapsed, 8.924s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.890s; elapsed, 8.923s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 8.850s; elapsed, 8.923s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 8.900s; elapsed, 8.919s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.900s; elapsed, 8.923s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 8.880s; elapsed, 8.925s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 8.850s; elapsed, 8.924s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 8.880s; elapsed, 8.922s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.880s; elapsed, 8.922s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 8.910s; elapsed, 8.922s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 8.920s; elapsed, 8.922s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 8.880s; elapsed, 8.921s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 8.890s; elapsed, 8.927s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 8.870s; elapsed, 8.928s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 8.860s; elapsed, 8.922s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.930s; elapsed, 8.933s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 8.880s; elapsed, 8.925s, #calls: 1', 'TOTAL : CPU, 8.880s; elapsed, 8.925s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 8.900s; elapsed, 8.924s, #calls: 1', 'TOTAL : CPU, 8.900s; elapsed, 8.924s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 8.870s; elapsed, 8.928s, #calls: 1', 'TOTAL : CPU, 8.870s; elapsed, 8.928s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 8.880s; elapsed, 8.921s, #calls: 1', 'TOTAL : CPU, 8.880s; elapsed, 8.921s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 8.900s; elapsed, 8.919s, #calls: 1', 'TOTAL : CPU, 8.900s; elapsed, 8.919s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 8.910s; elapsed, 8.922s, #calls: 1', 'TOTAL : CPU, 8.910s; elapsed, 8.922s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 8.850s; elapsed, 8.924s, #calls: 1', 'TOTAL : CPU, 8.850s; elapsed, 8.924s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.890s; elapsed, 8.923s, #calls: 1', 'TOTAL : CPU, 8.890s; elapsed, 8.923s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.900s; elapsed, 8.923s, #calls: 1', 'TOTAL : CPU, 8.900s; elapsed, 8.923s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 8.850s; elapsed, 8.923s, #calls: 1', 'TOTAL : CPU, 8.850s; elapsed, 8.923s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 8.880s; elapsed, 8.922s, #calls: 1', 'TOTAL : CPU, 8.880s; elapsed, 8.922s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.880s; elapsed, 8.922s, #calls: 1', 'TOTAL : CPU, 8.880s; elapsed, 8.922s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 8.890s; elapsed, 8.927s, #calls: 1', 'TOTAL : CPU, 8.890s; elapsed, 8.927s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 8.860s; elapsed, 8.922s, #calls: 1', 'TOTAL : CPU, 8.860s; elapsed, 8.922s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 8.920s; elapsed, 8.922s, #calls: 1', 'TOTAL : CPU, 8.920s; elapsed, 8.922s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 20.160s; elapsed, 20.186s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.178s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.820s; elapsed, 19.836s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.930s; elapsed, 8.933s, #calls: 1', 'TOTAL : CPU, 49.080s; elapsed, 49.134s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 1206.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 25.040s; elapsed, 12.567s', 'individual call time for EIGEN_LDLT: CPU, 19.530s; elapsed, 19.374s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.161s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 487,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,143', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.194928', '# - matching time = 0.06425', '# - symmetrization time = 0.015614', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,143', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0713201', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 168.522 MB', '# - factor time = 9.45702', '# - factor nonzeros = 21,065,199', '# - factor memory = 168.522 MB', '# - total flops = 5.03174e+10, min = 1.38671e+07, max = 2.85337e+10', '# - flop rate = 5.32064 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.13241e+10', '# --------------------------------------------', '# total = 5.13241e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.21233e-11\trel.res = 1.3136e-15\tbw.error = 3.37627e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0504751', '# - total flops = 9.15661e+07, min = 86678, max = 2.87842e+07', '# - flop rate = 1.81408 GFlop/s', '# - bytes moved = 158.417 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.13852 GByte/s', '# - solve arithmetic intensity = 0.578006 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.450s; elapsed, 9.871s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 16.210s; elapsed, 9.869s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 12.910s; elapsed, 9.867s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 11.290s; elapsed, 9.868s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 10.900s; elapsed, 9.869s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 11.780s; elapsed, 9.864s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 11.290s; elapsed, 9.870s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 11.130s; elapsed, 9.869s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.950s; elapsed, 9.869s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.970s; elapsed, 9.869s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 11.810s; elapsed, 9.869s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 11.490s; elapsed, 9.869s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 11.370s; elapsed, 9.868s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 14.010s; elapsed, 9.867s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 12.690s; elapsed, 9.863s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.390s; elapsed, 9.875s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 12.910s; elapsed, 9.867s, #calls: 1', 'TOTAL : CPU, 12.910s; elapsed, 9.867s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.450s; elapsed, 9.871s, #calls: 1', 'TOTAL : CPU, 11.450s; elapsed, 9.871s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 16.210s; elapsed, 9.869s, #calls: 1', 'TOTAL : CPU, 16.210s; elapsed, 9.869s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 11.810s; elapsed, 9.869s, #calls: 1', 'TOTAL : CPU, 11.810s; elapsed, 9.869s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 11.780s; elapsed, 9.864s, #calls: 1', 'TOTAL : CPU, 11.780s; elapsed, 9.864s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 11.290s; elapsed, 9.868s, #calls: 1', 'TOTAL : CPU, 11.290s; elapsed, 9.868s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.950s; elapsed, 9.869s, #calls: 1', 'TOTAL : CPU, 10.950s; elapsed, 9.869s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 10.900s; elapsed, 9.869s, #calls: 1', 'TOTAL : CPU, 10.900s; elapsed, 9.869s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 11.490s; elapsed, 9.869s, #calls: 1', 'TOTAL : CPU, 11.490s; elapsed, 9.869s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 11.290s; elapsed, 9.870s, #calls: 1', 'TOTAL : CPU, 11.290s; elapsed, 9.870s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 14.010s; elapsed, 9.867s, #calls: 1', 'TOTAL : CPU, 14.010s; elapsed, 9.867s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 11.370s; elapsed, 9.868s, #calls: 1', 'TOTAL : CPU, 11.370s; elapsed, 9.868s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 11.130s; elapsed, 9.869s, #calls: 1', 'TOTAL : CPU, 11.130s; elapsed, 9.869s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.970s; elapsed, 9.869s, #calls: 1', 'TOTAL : CPU, 10.970s; elapsed, 9.869s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 12.690s; elapsed, 9.863s, #calls: 1', 'TOTAL : CPU, 12.690s; elapsed, 9.863s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 25.040s; elapsed, 12.567s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.161s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.530s; elapsed, 19.374s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.390s; elapsed, 9.875s, #calls: 1', 'TOTAL : CPU, 64.230s; elapsed, 41.977s']
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_5k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_5k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 4.270s; elapsed, 4.286s', 'individual call time for EIGEN_LDLT: CPU, 2.500s; elapsed, 2.507s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.051s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 4.270s; elapsed, 4.286s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.051s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.500s; elapsed, 2.507s, #calls: 1', 'TOTAL : CPU, 6.820s; elapsed, 6.844s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.660s; elapsed, 2.850s', 'individual call time for EIGEN_LDLT: CPU, 2.730s; elapsed, 2.544s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.050s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.660s; elapsed, 2.850s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.050s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.730s; elapsed, 2.544s, #calls: 1', 'TOTAL : CPU, 8.470s; elapsed, 5.444s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.750s; elapsed, 1.969s', 'individual call time for EIGEN_LDLT: CPU, 3.020s; elapsed, 2.481s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.043s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.750s; elapsed, 1.969s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.043s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.020s; elapsed, 2.481s, #calls: 1', 'TOTAL : CPU, 10.860s; elapsed, 4.493s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 15.210s; elapsed, 1.935s', 'individual call time for EIGEN_LDLT: CPU, 4.550s; elapsed, 3.262s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.038s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 15.210s; elapsed, 1.935s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.038s, #calls: 1', 'time for EIGEN_LDLT: CPU, 4.550s; elapsed, 3.262s, #calls: 1', 'TOTAL : CPU, 19.910s; elapsed, 5.236s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 34.190s; elapsed, 2.181s', 'individual call time for EIGEN_LDLT: CPU, 6.980s; elapsed, 4.300s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.036s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 34.190s; elapsed, 2.181s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.036s, #calls: 1', 'time for EIGEN_LDLT: CPU, 6.980s; elapsed, 4.300s, #calls: 1', 'TOTAL : CPU, 41.420s; elapsed, 6.517s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 4.450s; elapsed, 4.453s', 'individual call time for EIGEN_LDLT: CPU, 2.540s; elapsed, 2.543s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.053s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,135', '# - number of levels = 77', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.147217', '# - matching time = 0.03107', '# - symmetrization time = 0.014946', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,135', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0406971', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.9326 MB', '# - factor time = 2.9176', '# - factor nonzeros = 6,491,577', '# - factor memory = 51.9326 MB', '# - total flops = 7.45429e+09, min = 3.29924e+09, max = 4.15505e+09', '# - flop rate = 2.55494 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.81378e+09', '# --------------------------------------------', '# total = 7.81378e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.4489e-11\trel.res = 1.47627e-15\tbw.error = 3.21475e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0388868', '# - total flops = 1.51891e+07, min = 7.43097e+06, max = 7.75814e+06', '# - flop rate = 0.390598 GFlop/s', '# - bytes moved = 97.4472 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.50592 GByte/s', '# - solve arithmetic intensity = 0.15587 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.200s; elapsed, 3.206s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.210s; elapsed, 3.216s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.200s; elapsed, 3.206s, #calls: 1', 'TOTAL : CPU, 3.200s; elapsed, 3.206s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 4.450s; elapsed, 4.453s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.053s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.540s; elapsed, 2.543s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.210s; elapsed, 3.216s, #calls: 1', 'TOTAL : CPU, 10.260s; elapsed, 10.264s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.880s; elapsed, 2.962s', 'individual call time for EIGEN_LDLT: CPU, 2.700s; elapsed, 2.515s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.042s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,135', '# - number of levels = 77', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.15005', '# - matching time = 0.030987', '# - symmetrization time = 0.0150139', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,135', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.049484', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.9326 MB', '# - factor time = 1.88924', '# - factor nonzeros = 6,491,577', '# - factor memory = 51.9326 MB', '# - total flops = 7.45434e+09, min = 3.29924e+09, max = 4.1551e+09', '# - flop rate = 3.94568 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.81378e+09', '# --------------------------------------------', '# total = 7.81378e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.44115e-11\trel.res = 1.47295e-15\tbw.error = 3.21352e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0401881', '# - total flops = 1.51917e+07, min = 7.43353e+06, max = 7.75814e+06', '# - flop rate = 0.378014 GFlop/s', '# - bytes moved = 97.5548 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.42746 GByte/s', '# - solve arithmetic intensity = 0.155725 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.250s; elapsed, 2.193s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.400s; elapsed, 2.205s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.250s; elapsed, 2.193s, #calls: 1', 'TOTAL : CPU, 4.250s; elapsed, 2.193s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.880s; elapsed, 2.962s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.042s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.700s; elapsed, 2.515s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.400s; elapsed, 2.205s, #calls: 1', 'TOTAL : CPU, 13.040s; elapsed, 7.725s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.800s; elapsed, 1.979s', 'individual call time for EIGEN_LDLT: CPU, 5.150s; elapsed, 4.594s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.041s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,135', '# - number of levels = 77', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.137206', '# - matching time = 0.030055', '# - symmetrization time = 0.0155549', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,135', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0462148', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.9326 MB', '# - factor time = 1.54288', '# - factor nonzeros = 6,491,577', '# - factor memory = 51.9326 MB', '# - total flops = 7.45434e+09, min = 3.29924e+09, max = 4.1551e+09', '# - flop rate = 4.83145 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.81378e+09', '# --------------------------------------------', '# total = 7.81378e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.43531e-11\trel.res = 1.47045e-15\tbw.error = 3.21352e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0401499', '# - total flops = 1.51917e+07, min = 7.43353e+06, max = 7.75814e+06', '# - flop rate = 0.378374 GFlop/s', '# - bytes moved = 97.5688 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.43011 GByte/s', '# - solve arithmetic intensity = 0.155702 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 7.230s; elapsed, 1.832s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 7.360s; elapsed, 1.842s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 7.230s; elapsed, 1.832s, #calls: 1', 'TOTAL : CPU, 7.230s; elapsed, 1.832s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.800s; elapsed, 1.979s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.041s, #calls: 1', 'time for EIGEN_LDLT: CPU, 5.150s; elapsed, 4.594s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 7.360s; elapsed, 1.842s, #calls: 1', 'TOTAL : CPU, 20.400s; elapsed, 8.457s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 15.920s; elapsed, 2.026s', 'individual call time for EIGEN_LDLT: CPU, 4.570s; elapsed, 3.292s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.038s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,135', '# - number of levels = 77', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.172157', '# - matching time = 0.0414472', '# - symmetrization time = 0.016206', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,135', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0516469', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.9326 MB', '# - factor time = 2.00355', '# - factor nonzeros = 6,491,577', '# - factor memory = 51.9326 MB', '# - total flops = 7.45434e+09, min = 3.29924e+09, max = 4.1551e+09', '# - flop rate = 3.72056 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.81378e+09', '# --------------------------------------------', '# total = 7.81378e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.43303e-11\trel.res = 1.46947e-15\tbw.error = 3.21352e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.048352', '# - total flops = 1.51917e+07, min = 7.43353e+06, max = 7.75814e+06', '# - flop rate = 0.314189 GFlop/s', '# - bytes moved = 97.5975 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.01848 GByte/s', '# - solve arithmetic intensity = 0.155656 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 16.150s; elapsed, 2.354s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.850s; elapsed, 2.359s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 16.150s; elapsed, 2.354s, #calls: 1', 'TOTAL : CPU, 16.150s; elapsed, 2.354s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 15.920s; elapsed, 2.026s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.038s, #calls: 1', 'time for EIGEN_LDLT: CPU, 4.570s; elapsed, 3.292s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.850s; elapsed, 2.359s, #calls: 1', 'TOTAL : CPU, 39.480s; elapsed, 7.714s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 34.280s; elapsed, 2.189s', 'individual call time for EIGEN_LDLT: CPU, 5.260s; elapsed, 2.581s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.045s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,135', '# - number of levels = 77', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.191091', '# - matching time = 0.0466368', '# - symmetrization time = 0.0185668', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,135', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0600119', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.9326 MB', '# - factor time = 1.89173', '# - factor nonzeros = 6,491,577', '# - factor memory = 51.9326 MB', '# - total flops = 7.45449e+09, min = 3.29924e+09, max = 4.15525e+09', '# - flop rate = 3.94056 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.81378e+09', '# --------------------------------------------', '# total = 7.81378e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.41901e-11\trel.res = 1.46347e-15\tbw.error = 3.20616e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0562589', '# - total flops = 1.51917e+07, min = 7.43353e+06, max = 7.75814e+06', '# - flop rate = 0.270031 GFlop/s', '# - bytes moved = 97.6642 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.73598 GByte/s', '# - solve arithmetic intensity = 0.15555 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 32.270s; elapsed, 2.294s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 36.470s; elapsed, 2.304s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 32.270s; elapsed, 2.294s, #calls: 1', 'TOTAL : CPU, 32.270s; elapsed, 2.294s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 34.280s; elapsed, 2.189s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.045s, #calls: 1', 'time for EIGEN_LDLT: CPU, 5.260s; elapsed, 2.581s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 36.470s; elapsed, 2.304s, #calls: 1', 'TOTAL : CPU, 76.320s; elapsed, 7.118s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 4.530s; elapsed, 4.518s', 'individual call time for EIGEN_LDLT: CPU, 2.480s; elapsed, 2.493s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.052s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,141', '# - number of levels = 73', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.145284', '# - matching time = 0.0294719', '# - symmetrization time = 0.0199199', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0341599', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.6883 MB', '# - factor time = 1.77627', '# - factor nonzeros = 6,461,037', '# - factor memory = 51.6883 MB', '# - total flops = 7.39199e+09, min = 916324, max = 3.29543e+09', '# - flop rate = 4.16152 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.73217e+09', '# --------------------------------------------', '# total = 7.73217e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.78316e-11\trel.res = 1.1913e-15\tbw.error = 2.54495e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.040529', '# - total flops = 1.65485e+07, min = 39371, max = 7.71852e+06', '# - flop rate = 0.408313 GFlop/s', '# - bytes moved = 90.5862 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.23509 GByte/s', '# - solve arithmetic intensity = 0.182683 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.040s; elapsed, 2.052s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.050s; elapsed, 2.060s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.060s; elapsed, 2.059s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.060s; elapsed, 2.064s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.050s; elapsed, 2.060s, #calls: 1', 'TOTAL : CPU, 2.050s; elapsed, 2.060s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.040s; elapsed, 2.052s, #calls: 1', 'TOTAL : CPU, 2.040s; elapsed, 2.052s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.060s; elapsed, 2.059s, #calls: 1', 'TOTAL : CPU, 2.060s; elapsed, 2.059s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 4.530s; elapsed, 4.518s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.052s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.480s; elapsed, 2.493s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.060s; elapsed, 2.064s, #calls: 1', 'TOTAL : CPU, 9.120s; elapsed, 9.126s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.860s; elapsed, 2.952s', 'individual call time for EIGEN_LDLT: CPU, 2.780s; elapsed, 2.599s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.047s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,141', '# - number of levels = 73', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.16566', '# - matching time = 0.0348568', '# - symmetrization time = 0.0226281', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.042275', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.6883 MB', '# - factor time = 1.72614', '# - factor nonzeros = 6,461,037', '# - factor memory = 51.6883 MB', '# - total flops = 7.39201e+09, min = 916324, max = 3.29543e+09', '# - flop rate = 4.2824 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.73217e+09', '# --------------------------------------------', '# total = 7.73217e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.70025e-11\trel.res = 1.15582e-15\tbw.error = 2.66061e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.046644', '# - total flops = 1.65485e+07, min = 39371, max = 7.71852e+06', '# - flop rate = 0.354783 GFlop/s', '# - bytes moved = 90.6001 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.94238 GByte/s', '# - solve arithmetic intensity = 0.182654 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.920s; elapsed, 2.051s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.740s; elapsed, 2.060s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.970s; elapsed, 2.060s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.080s; elapsed, 2.065s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.920s; elapsed, 2.051s, #calls: 1', 'TOTAL : CPU, 3.920s; elapsed, 2.051s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.740s; elapsed, 2.060s, #calls: 1', 'TOTAL : CPU, 2.740s; elapsed, 2.060s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.970s; elapsed, 2.060s, #calls: 1', 'TOTAL : CPU, 3.970s; elapsed, 2.060s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.860s; elapsed, 2.952s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.047s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.780s; elapsed, 2.599s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.080s; elapsed, 2.065s, #calls: 1', 'TOTAL : CPU, 12.800s; elapsed, 7.663s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 8.980s; elapsed, 2.276s', 'individual call time for EIGEN_LDLT: CPU, 5.330s; elapsed, 4.791s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.042s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,141', '# - number of levels = 73', '# - nd time = 0.170296', '# - matching time = 0.0358679', '# - symmetrization time = 0.0230162', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0447381', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.6883 MB', '# - factor time = 1.99517', '# - factor nonzeros = 6,461,037', '# - factor memory = 51.6883 MB', '# - total flops = 7.39202e+09, min = 916780, max = 3.29543e+09', '# - flop rate = 3.70496 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.73217e+09', '# --------------------------------------------', '# total = 7.73217e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.69566e-11\trel.res = 1.15385e-15\tbw.error = 2.65871e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.055737', '# - total flops = 1.65485e+07, min = 39371, max = 7.71852e+06', '# - flop rate = 0.296903 GFlop/s', '# - bytes moved = 90.6178 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.62581 GByte/s', '# - solve arithmetic intensity = 0.182619 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.490s; elapsed, 2.347s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.080s; elapsed, 2.346s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 7.210s; elapsed, 2.339s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.280s; elapsed, 2.352s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.080s; elapsed, 2.346s, #calls: 1', 'TOTAL : CPU, 8.080s; elapsed, 2.346s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.490s; elapsed, 2.347s, #calls: 1', 'TOTAL : CPU, 4.490s; elapsed, 2.347s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 7.210s; elapsed, 2.339s, #calls: 1', 'TOTAL : CPU, 7.210s; elapsed, 2.339s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 8.980s; elapsed, 2.276s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.042s, #calls: 1', 'time for EIGEN_LDLT: CPU, 5.330s; elapsed, 4.791s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.280s; elapsed, 2.352s, #calls: 1', 'TOTAL : CPU, 23.680s; elapsed, 9.460s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 14.460s; elapsed, 1.843s', 'individual call time for EIGEN_LDLT: CPU, 4.620s; elapsed, 3.362s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.035s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', 'CPP -2', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,141', '# - number of levels = 73', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.16953', '# - matching time = 0.038388', '# - symmetrization time = 0.0228341', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.052299', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.6883 MB', '# - factor time = 1.86488', '# - factor nonzeros = 6,461,037', '# - factor memory = 51.6883 MB', '# - total flops = 7.39202e+09, min = 916780, max = 3.29543e+09', '# - flop rate = 3.9638 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.73217e+09', '# --------------------------------------------', '# total = 7.73217e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.76568e-11\trel.res = 1.18382e-15\tbw.error = 2.6511e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0514381', '# - total flops = 1.65485e+07, min = 39371, max = 7.71852e+06', '# - flop rate = 0.321717 GFlop/s', '# - bytes moved = 90.6531 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.76237 GByte/s', '# - solve arithmetic intensity = 0.182548 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 7.200s; elapsed, 2.220s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.300s; elapsed, 2.222s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 13.080s; elapsed, 2.215s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.510s; elapsed, 2.228s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.300s; elapsed, 2.222s, #calls: 1', 'TOTAL : CPU, 16.300s; elapsed, 2.222s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 7.200s; elapsed, 2.220s, #calls: 1', 'TOTAL : CPU, 7.200s; elapsed, 2.220s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 13.080s; elapsed, 2.215s, #calls: 1', 'TOTAL : CPU, 13.080s; elapsed, 2.215s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 14.460s; elapsed, 1.843s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.035s, #calls: 1', 'time for EIGEN_LDLT: CPU, 4.620s; elapsed, 3.362s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.510s; elapsed, 2.228s, #calls: 1', 'TOTAL : CPU, 36.710s; elapsed, 7.467s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 2413.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 4.780s; elapsed, 4.781s', 'individual call time for EIGEN_LDLT: CPU, 3.260s; elapsed, 3.263s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.060s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,141', '# - number of levels = 72', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.164098', '# - matching time = 0.031805', '# - symmetrization time = 0.016156', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0366211', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.6883 MB', '# - factor time = 1.73019', '# - factor nonzeros = 6,461,037', '# - factor memory = 51.6883 MB', '# - total flops = 7.39199e+09, min = 730831, max = 3.2947e+09', '# - flop rate = 4.27236 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.70381e+09', '# --------------------------------------------', '# total = 7.70381e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.8199e-11\trel.res = 1.20703e-15\tbw.error = 2.54697e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.042578', '# - total flops = 2.11372e+07, min = 25719, max = 7.69055e+06', '# - flop rate = 0.496436 GFlop/s', '# - bytes moved = 85.1253 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.99928 GByte/s', '# - solve arithmetic intensity = 0.248307 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.020s; elapsed, 2.035s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.000s; elapsed, 2.033s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.990s; elapsed, 2.033s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.990s; elapsed, 2.035s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.020s; elapsed, 2.034s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.020s; elapsed, 2.034s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.000s; elapsed, 2.028s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.040s; elapsed, 2.039s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.020s; elapsed, 2.035s, #calls: 1', 'TOTAL : CPU, 2.020s; elapsed, 2.035s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.020s; elapsed, 2.034s, #calls: 1', 'TOTAL : CPU, 2.020s; elapsed, 2.034s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.990s; elapsed, 2.033s, #calls: 1', 'TOTAL : CPU, 1.990s; elapsed, 2.033s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.000s; elapsed, 2.028s, #calls: 1', 'TOTAL : CPU, 2.000s; elapsed, 2.028s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.990s; elapsed, 2.035s, #calls: 1', 'TOTAL : CPU, 1.990s; elapsed, 2.035s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.000s; elapsed, 2.033s, #calls: 1', 'TOTAL : CPU, 2.000s; elapsed, 2.033s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.020s; elapsed, 2.034s, #calls: 1', 'TOTAL : CPU, 2.020s; elapsed, 2.034s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 4.780s; elapsed, 4.781s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.060s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.260s; elapsed, 3.263s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.040s; elapsed, 2.039s, #calls: 1', 'TOTAL : CPU, 10.140s; elapsed, 10.142s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 2413.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.540s; elapsed, 3.291s', 'individual call time for EIGEN_LDLT: CPU, 2.950s; elapsed, 2.769s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.051s', 'CPP -2', '# Initializing STRUMPACK', 'CPP -2', 'CPP -2', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,141', '# - number of levels = 72', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.167989', '# - matching time = 0.035311', '# - symmetrization time = 0.0158229', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0450821', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.6883 MB', '# - factor time = 1.63997', '# - factor nonzeros = 6,461,037', '# - factor memory = 51.6883 MB', '# - total flops = 7.392e+09, min = 730831, max = 3.2947e+09', '# - flop rate = 4.50739 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.70381e+09', '# --------------------------------------------', '# total = 7.70381e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.81859e-11\trel.res = 1.20647e-15\tbw.error = 2.54697e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0353692', '# - total flops = 2.11372e+07, min = 25719, max = 7.69055e+06', '# - flop rate = 0.597617 GFlop/s', '# - bytes moved = 85.1328 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.40698 GByte/s', '# - solve arithmetic intensity = 0.248285 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.610s; elapsed, 1.955s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 3.500s; elapsed, 1.951s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 3.450s; elapsed, 1.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.580s; elapsed, 1.955s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.790s; elapsed, 1.953s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.870s; elapsed, 1.956s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 3.550s; elapsed, 1.955s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.870s; elapsed, 1.961s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.610s; elapsed, 1.955s, #calls: 1', 'TOTAL : CPU, 2.610s; elapsed, 1.955s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 3.500s; elapsed, 1.951s, #calls: 1', 'TOTAL : CPU, 3.500s; elapsed, 1.951s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.870s; elapsed, 1.956s, #calls: 1', 'TOTAL : CPU, 2.870s; elapsed, 1.956s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.790s; elapsed, 1.953s, #calls: 1', 'TOTAL : CPU, 3.790s; elapsed, 1.953s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 3.450s; elapsed, 1.954s, #calls: 1', 'TOTAL : CPU, 3.450s; elapsed, 1.954s', 'Exiting profiler', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 3.550s; elapsed, 1.955s, #calls: 1', 'TOTAL : CPU, 3.550s; elapsed, 1.955s', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.580s; elapsed, 1.955s, #calls: 1', 'TOTAL : CPU, 2.580s; elapsed, 1.955s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.540s; elapsed, 3.291s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.051s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.950s; elapsed, 2.769s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.870s; elapsed, 1.961s, #calls: 1', 'TOTAL : CPU, 13.440s; elapsed, 8.072s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 2413.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 8.560s; elapsed, 2.174s', 'individual call time for EIGEN_LDLT: CPU, 3.250s; elapsed, 2.713s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.043s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,141', '# - number of levels = 72', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.176374', '# - matching time = 0.036586', '# - symmetrization time = 0.0310369', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0450699', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.6883 MB', '# - factor time = 2.04246', '# - factor nonzeros = 6,461,037', '# - factor memory = 51.6883 MB', '# - total flops = 7.392e+09, min = 731259, max = 3.2947e+09', '# - flop rate = 3.61917 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.70381e+09', '# --------------------------------------------', '# total = 7.70381e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.81788e-11\trel.res = 1.20616e-15\tbw.error = 2.54697e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0479572', '# - total flops = 2.11372e+07, min = 25719, max = 7.69055e+06', '# - flop rate = 0.440752 GFlop/s', '# - bytes moved = 85.1418 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.77537 GByte/s', '# - solve arithmetic intensity = 0.248259 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 6.810s; elapsed, 2.400s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 6.880s; elapsed, 2.393sindividual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.550s; elapsed, 2.400s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 7.170s; elapsed, 2.399s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.400s; elapsed, 2.399s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.220s; elapsed, 2.399s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 5.350s; elapsed, 2.399s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.450s; elapsed, 2.404s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 6.810s; elapsed, 2.400s, #calls: 1', 'TOTAL : CPU, 6.810s; elapsed, 2.400s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.550s; elapsed, 2.400s, #calls: 1', 'TOTAL : CPU, 4.550s; elapsed, 2.400s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.220s; elapsed, 2.399s, #calls: 1', 'TOTAL : CPU, 8.220s; elapsed, 2.399s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 6.880s; elapsed, 2.393s, #calls: 1', 'TOTAL : CPU, 6.880s; elapsed, 2.393s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.400s; elapsed, 2.399s, #calls: 1', 'TOTAL : CPU, 4.400s; elapsed, 2.399s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 7.170s; elapsed, 2.399s, #calls: 1', 'TOTAL : CPU, 7.170s; elapsed, 2.399s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 5.350s; elapsed, 2.399s, #calls: 1', 'TOTAL : CPU, 5.350s; elapsed, 2.399s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 8.560s; elapsed, 2.174s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.043s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.250s; elapsed, 2.713s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.450s; elapsed, 2.404s, #calls: 1', 'TOTAL : CPU, 21.360s; elapsed, 7.335s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 1206.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.140s; elapsed, 5.141s', 'individual call time for EIGEN_LDLT: CPU, 2.760s; elapsed, 2.766s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.070s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,141', '# - number of levels = 71', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.170756', '# - matching time = 0.0361779', '# - symmetrization time = 0.0151711', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0388381', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.6883 MB', '# - factor time = 1.79188', '# - factor nonzeros = 6,461,037', '# - factor memory = 51.6883 MB', '# - total flops = 7.39199e+09, min = 730831, max = 3.28676e+09', '# - flop rate = 4.12527 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.64871e+09', '# --------------------------------------------', '# total = 7.64871e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.4552e-11\trel.res = 1.05092e-15\tbw.error = 2.21425e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.035815', '# - total flops = 2.9672e+07, min = 19684, max = 7.58043e+06', '# - flop rate = 0.82848 GFlop/s', '# - bytes moved = 82.5245 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.30419 GByte/s', '# - solve arithmetic intensity = 0.359554 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.030s; elapsed, 2.103s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.060s; elapsed, 2.101s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.080s; elapsed, 2.101s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.050s; elapsed, 2.101s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 2.070s; elapsed, 2.100s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 2.030s; elapsed, 2.101s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 2.100s; elapsed, 2.101s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.040s; elapsed, 2.102s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.080s; elapsed, 2.103s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.080s; elapsed, 2.102s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 2.040s; elapsed, 2.101s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 2.040s; elapsed, 2.099s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 2.060s; elapsed, 2.096s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 2.090s; elapsed, 2.100s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 2.030s; elapsed, 2.101s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.100s; elapsed, 2.107s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.030s; elapsed, 2.103s, #calls: 1', 'TOTAL : CPU, 2.030s; elapsed, 2.103s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.080s; elapsed, 2.103s, #calls: 1', 'TOTAL : CPU, 2.080s; elapsed, 2.103s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.080s; elapsed, 2.102s, #calls: 1', 'TOTAL : CPU, 2.080s; elapsed, 2.102s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 2.100s; elapsed, 2.101s, #calls: 1', 'TOTAL : CPU, 2.100s; elapsed, 2.101s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.040s; elapsed, 2.102s, #calls: 1', 'TOTAL : CPU, 2.040s; elapsed, 2.102s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 2.070s; elapsed, 2.100s, #calls: 1', 'TOTAL : CPU, 2.070s; elapsed, 2.100s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 2.060s; elapsed, 2.096s, #calls: 1', 'TOTAL : CPU, 2.060s; elapsed, 2.096s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 2.040s; elapsed, 2.101s, #calls: 1', 'TOTAL : CPU, 2.040s; elapsed, 2.101s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.050s; elapsed, 2.101s, #calls: 1', 'TOTAL : CPU, 2.050s; elapsed, 2.101s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 2.040s; elapsed, 2.099s, #calls: 1', 'TOTAL : CPU, 2.040s; elapsed, 2.099s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.060s; elapsed, 2.101s, #calls: 1', 'TOTAL : CPU, 2.060s; elapsed, 2.101s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.080s; elapsed, 2.101s, #calls: 1', 'TOTAL : CPU, 2.080s; elapsed, 2.101s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 2.030s; elapsed, 2.101s, #calls: 1', 'TOTAL : CPU, 2.030s; elapsed, 2.101s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 2.030s; elapsed, 2.101s, #calls: 1', 'TOTAL : CPU, 2.030s; elapsed, 2.101s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 2.090s; elapsed, 2.100s, #calls: 1', 'TOTAL : CPU, 2.090s; elapsed, 2.100s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.140s; elapsed, 5.141s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.070s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.760s; elapsed, 2.766s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.100s; elapsed, 2.107s, #calls: 1', 'TOTAL : CPU, 10.070s; elapsed, 10.083s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '19309 1206.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.260s; elapsed, 3.151s', 'individual call time for EIGEN_LDLT: CPU, 2.740s; elapsed, 2.566s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.052s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 19,309', '# - number of nonzeros = 251,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,141', '# - number of levels = 71', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.172332', '# - matching time = 0.036546', '# - symmetrization time = 0.01494', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0438919', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 51.6883 MB', '# - factor time = 1.77551', '# - factor nonzeros = 6,461,037', '# - factor memory = 51.6883 MB', '# - total flops = 7.39199e+09, min = 730831, max = 3.28676e+09', '# - flop rate = 4.1633 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.64871e+09', '# --------------------------------------------', '# total = 7.64871e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.455e-11\trel.res = 1.05084e-15\tbw.error = 2.21425e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0354691', '# - total flops = 2.9672e+07, min = 19684, max = 7.58043e+06', '# - flop rate = 0.836561 GFlop/s', '# - bytes moved = 82.5284 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.32677 GByte/s', '# - solve arithmetic intensity = 0.359537 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.810s; elapsed, 2.098sindividual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.760s; elapsed, 2.095s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 3.350s; elapsed, 2.095s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.720s; elapsed, 2.095s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 2.970s; elapsed, 2.096s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 3.670s; elapsed, 2.096s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 3.010s; elapsed, 2.097s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 2.990s; elapsed, 2.096s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 3.240s; elapsed, 2.095s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 3.530s; elapsed, 2.093s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.850s; elapsed, 2.095s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 3.610s; elapsed, 2.090s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.720s; elapsed, 2.095s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 3.900s; elapsed, 2.095s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 3.070s; elapsed, 2.095s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.150s; elapsed, 2.101s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.760s; elapsed, 2.095s, #calls: 1', 'TOTAL : CPU, 2.760s; elapsed, 2.095s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 3.010s; elapsed, 2.097s, #calls: 1', 'TOTAL : CPU, 3.010s; elapsed, 2.097s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.720s; elapsed, 2.095s, #calls: 1', 'TOTAL : CPU, 2.720s; elapsed, 2.095s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 3.670s; elapsed, 2.096s, #calls: 1', 'TOTAL : CPU, 3.670s; elapsed, 2.096s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 2.970s; elapsed, 2.096s, #calls: 1', 'TOTAL : CPU, 2.970s; elapsed, 2.096s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 3.240s; elapsed, 2.095s, #calls: 1', 'TOTAL : CPU, 3.240s; elapsed, 2.095s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 3.610s; elapsed, 2.090s, #calls: 1', 'TOTAL : CPU, 3.610s; elapsed, 2.090s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 3.350s; elapsed, 2.095s, #calls: 1', 'TOTAL : CPU, 3.350s; elapsed, 2.095s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.850s; elapsed, 2.095s, #calls: 1', 'TOTAL : CPU, 2.850s; elapsed, 2.095s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 3.530s; elapsed, 2.093s, #calls: 1', 'TOTAL : CPU, 3.530s; elapsed, 2.093s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 3.900s; elapsed, 2.095s, #calls: 1', 'TOTAL : CPU, 3.900s; elapsed, 2.095s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 2.990s; elapsed, 2.096s, #calls: 1', 'TOTAL : CPU, 2.990s; elapsed, 2.096s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 3.070s; elapsed, 2.095s, #calls: 1', 'TOTAL : CPU, 3.070s; elapsed, 2.095s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.720s; elapsed, 2.095s, #calls: 1', 'TOTAL : CPU, 2.720s; elapsed, 2.095s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.810s; elapsed, 2.098s, #calls: 1', 'TOTAL : CPU, 2.810s; elapsed, 2.098s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.260s; elapsed, 3.151s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.052s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.740s; elapsed, 2.566s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.150s; elapsed, 2.101s, #calls: 1', 'TOTAL : CPU, 13.220s; elapsed, 7.870s']
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/A_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/b_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 25309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 24.010s; elapsed, 24.030s', 'individual call time for EIGEN_LDLT: CPU, 18.910s; elapsed, 18.929s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.176s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 24.010s; elapsed, 24.030s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.176s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.910s; elapsed, 18.929s, #calls: 1', 'TOTAL : CPU, 43.100s; elapsed, 43.134s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 25309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 31.440s; elapsed, 15.758s', 'individual call time for EIGEN_LDLT: CPU, 19.000s; elapsed, 18.818s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.151s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 31.440s; elapsed, 15.758s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.151s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.000s; elapsed, 18.818s, #calls: 1', 'TOTAL : CPU, 50.690s; elapsed, 34.727s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 25309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 34.410s; elapsed, 8.681s', 'individual call time for EIGEN_LDLT: CPU, 19.180s; elapsed, 18.653s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.110s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 34.410s; elapsed, 8.681s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.110s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.180s; elapsed, 18.653s, #calls: 1', 'TOTAL : CPU, 53.880s; elapsed, 27.443s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 25309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 69.650s; elapsed, 8.779s', 'individual call time for EIGEN_LDLT: CPU, 19.870s; elapsed, 18.637s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.097s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 69.650s; elapsed, 8.779s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.097s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.870s; elapsed, 18.637s, #calls: 1', 'TOTAL : CPU, 89.990s; elapsed, 27.513s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 25309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 155.190s; elapsed, 9.792s', 'individual call time for EIGEN_LDLT: CPU, 21.680s; elapsed, 19.080s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.820s; elapsed, 0.096s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 155.190s; elapsed, 9.792s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.820s; elapsed, 0.096s, #calls: 1', 'time for EIGEN_LDLT: CPU, 21.680s; elapsed, 19.080s, #calls: 1', 'TOTAL : CPU, 177.690s; elapsed, 28.968s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 12654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 25.210s; elapsed, 25.227s', 'individual call time for EIGEN_LDLT: CPU, 19.520s; elapsed, 19.536s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.186s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,083', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.238683', '# - matching time = 0.0571821', '# - symmetrization time = 0.0360129', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,083', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0735319', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 159.473 MB', '# - factor time = 16.7673', '# - factor nonzeros = 19,934,141', '# - factor memory = 159.473 MB', '# - total flops = 4.94019e+10, min = 1.95383e+10, max = 2.98636e+10', '# - flop rate = 2.94632 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.19065e+10', '# --------------------------------------------', '# total = 5.19065e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.46581e-11\trel.res = 1.82618e-15\tbw.error = 4.40681e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0968361', '# - total flops = 4.46752e+07, min = 2.18291e+07, max = 2.28462e+07', '# - flop rate = 0.461349 GFlop/s', '# - bytes moved = 239.032 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.46842 GByte/s', '# - solve arithmetic intensity = 0.186901 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.300s; elapsed, 17.308s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.320s; elapsed, 17.319s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.300s; elapsed, 17.308s, #calls: 1', 'TOTAL : CPU, 17.300s; elapsed, 17.308s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 25.210s; elapsed, 25.227s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.186s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.520s; elapsed, 19.536s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.320s; elapsed, 17.319s, #calls: 1', 'TOTAL : CPU, 62.230s; elapsed, 62.268s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 12654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.050s; elapsed, 16.085s', 'individual call time for EIGEN_LDLT: CPU, 20.090s; elapsed, 19.921s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.159s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,083', '# - number of levels = 132', '# - nd time = 0.243844', '# - matching time = 0.0580831', '# - symmetrization time = 0.0362482', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,083', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0699229', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 159.473 MB', '# - factor time = 10.2909', '# - factor nonzeros = 19,934,141', '# - factor memory = 159.473 MB', '# - total flops = 4.94021e+10, min = 1.95383e+10, max = 2.98639e+10', '# - flop rate = 4.80056 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.19065e+10', '# --------------------------------------------', '# total = 5.19065e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.38325e-11\trel.res = 1.79242e-15\tbw.error = 4.30412e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.120665', '# - total flops = 4.46938e+07, min = 2.18291e+07, max = 2.28648e+07', '# - flop rate = 0.370396 GFlop/s', '# - bytes moved = 239.769 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.98706 GByte/s', '# - solve arithmetic intensity = 0.186404 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 21.550s; elapsed, 10.863s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.820s; elapsed, 10.876s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 21.550s; elapsed, 10.863s, #calls: 1', 'TOTAL : CPU, 21.550s; elapsed, 10.863s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.050s; elapsed, 16.085s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.159s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.090s; elapsed, 19.921s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.820s; elapsed, 10.876s, #calls: 1', 'TOTAL : CPU, 72.220s; elapsed, 47.040s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 12654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 35.710s; elapsed, 9.015s', 'individual call time for EIGEN_LDLT: CPU, 19.680s; elapsed, 19.132s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.111s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,083', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.229069', '# - matching time = 0.055161', '# - symmetrization time = 0.038403', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,083', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.076633', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 159.473 MB', '# - factor time = 9.54431', '# - factor nonzeros = 19,934,141', '# - factor memory = 159.473 MB', '# - total flops = 4.94022e+10, min = 1.95383e+10, max = 2.98639e+10', '# - flop rate = 5.17608 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.19065e+10', '# --------------------------------------------', '# total = 5.19065e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.38597e-11\trel.res = 1.79353e-15\tbw.error = 4.30623e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.103002', '# - total flops = 4.46938e+07, min = 2.18291e+07, max = 2.28648e+07', '# - flop rate = 0.433912 GFlop/s', '# - bytes moved = 239.788 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.32799 GByte/s', '# - solve arithmetic intensity = 0.186389 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 38.380s; elapsed, 10.091s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 40.120s; elapsed, 10.104s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 38.380s; elapsed, 10.091s, #calls: 1', 'TOTAL : CPU, 38.380s; elapsed, 10.091s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 35.710s; elapsed, 9.015s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.680s; elapsed, 19.132s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 40.120s; elapsed, 10.104s, #calls: 1', 'TOTAL : CPU, 95.810s; elapsed, 38.361s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 12654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 72.110s; elapsed, 9.098s', 'individual call time for EIGEN_LDLT: CPU, 23.350s; elapsed, 22.124s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.116s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using CPP -2', '2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,083', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.255446', '# - matching time = 0.066854', '# - symmetrization time = 0.0424969', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,083', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.084245', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 159.473 MB', '# - factor time = 8.69547', '# - factor nonzeros = 19,934,141', '# - factor memory = 159.473 MB', '# - total flops = 4.94022e+10, min = 1.95383e+10, max = 2.98639e+10', '# - flop rate = 5.68137 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.19065e+10', '# --------------------------------------------', '# total = 5.19065e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.385e-11\trel.res = 1.79314e-15\tbw.error = 4.30623e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.101639', '# - total flops = 4.46938e+07, min = 2.18291e+07, max = 2.28648e+07', '# - flop rate = 0.439732 GFlop/s', '# - bytes moved = 239.826 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.35959 GByte/s', '# - solve arithmetic intensity = 0.186359 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 70.500s; elapsed, 9.292s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 73.350s; elapsed, 9.301s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 70.500s; elapsed, 9.292s, #calls: 1', 'TOTAL : CPU, 70.500s; elapsed, 9.292s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 72.110s; elapsed, 9.098s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.116s, #calls: 1', 'time for EIGEN_LDLT: CPU, 23.350s; elapsed, 22.124s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 73.350s; elapsed, 9.301s, #calls: 1', 'TOTAL : CPU, 169.330s; elapsed, 40.639s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 12654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 162.250s; elapsed, 10.235s', 'individual call time for EIGEN_LDLT: CPU, 21.680s; elapsed, 19.086s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.790s; elapsed, 0.100s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', '# using CPP -2', '2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,083', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.261143', '# - matching time = 0.0749629', '# - symmetrization time = 0.047236', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,083', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0946751', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 159.473 MB', '# - factor time = 9.78961', '# - factor nonzeros = 19,934,141', '# - factor memory = 159.473 MB', '# - total flops = 4.94029e+10, min = 1.95383e+10, max = 2.98646e+10', '# - flop rate = 5.04646 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.19065e+10', '# --------------------------------------------', '# total = 5.19065e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.38061e-11\trel.res = 1.79134e-15\tbw.error = 4.30803e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.105924', '# - total flops = 4.46975e+07, min = 2.18291e+07, max = 2.28684e+07', '# - flop rate = 0.421977 GFlop/s', '# - bytes moved = 239.962 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.26542 GByte/s', '# - solve arithmetic intensity = 0.186269 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 163.470s; elapsed, 10.421s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 164.560s; elapsed, 10.435s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 163.470s; elapsed, 10.421s, #calls: 1', 'TOTAL : CPU, 163.470s; elapsed, 10.421s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 162.250s; elapsed, 10.235s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.790s; elapsed, 0.100s, #calls: 1', 'time for EIGEN_LDLT: CPU, 21.680s; elapsed, 19.086s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 164.560s; elapsed, 10.435s, #calls: 1', 'TOTAL : CPU, 349.280s; elapsed, 39.856s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 6327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 25.190s; elapsed, 25.199s', 'individual call time for EIGEN_LDLT: CPU, 19.290s; elapsed, 19.304s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.187s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,133', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************# - nd time = ', '0.238845', '# - matching time = 0.0584478', '# - symmetrization time = 0.035651', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,133', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0646672', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 159.055 MB', '# - factor time = 9.22978', '# - factor nonzeros = 19,881,835', '# - factor memory = 159.055 MB', '# - total flops = 4.94679e+10, min = 6.57516e+09, max = 1.95818e+10', '# - flop rate = 5.35959 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.19221e+10', '# --------------------------------------------', '# total = 5.19221e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.08337e-11\trel.res = 1.66979e-15\tbw.error = 5.21064e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.062953', '# - total flops = 6.3129e+07, min = 1.23718e+07, max = 2.18423e+07', '# - flop rate = 1.0028 GFlop/s', '# - bytes moved = 191.22 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.03751 GByte/s', '# - solve arithmetic intensity = 0.330137 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.700s; elapsed, 9.719s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.700s; elapsed, 9.715s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.680s; elapsed, 9.701s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.710s; elapsed, 9.721s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.700s; elapsed, 9.715s, #calls: 1', 'TOTAL : CPU, 9.700s; elapsed, 9.715s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.700s; elapsed, 9.719s, #calls: 1', 'TOTAL : CPU, 9.700s; elapsed, 9.719s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.680s; elapsed, 9.701s, #calls: 1', 'TOTAL : CPU, 9.680s; elapsed, 9.701s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 25.190s; elapsed, 25.199s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.187s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.290s; elapsed, 19.304s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.710s; elapsed, 9.721s, #calls: 1', 'TOTAL : CPU, 54.380s; elapsed, 54.411s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 6327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 35.370s; elapsed, 17.780s', 'individual call time for EIGEN_LDLT: CPU, 19.430s; elapsed, 19.261s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.150s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,133', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.245296', '# - matching time = 0.0582621', '# - symmetrization time = 0.040354', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,133', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0785532', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 159.055 MB', '# - factor time = 8.35068', '# - factor nonzeros = 19,881,835', '# - factor memory = 159.055 MB', '# - total flops = 4.94679e+10, min = 6.57516e+09, max = 1.95818e+10', '# - flop rate = 5.92381 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.19221e+10', '# --------------------------------------------', '# total = 5.19221e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.07101e-11\trel.res = 1.66474e-15\tbw.error = 5.08115e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0637131', '# - total flops = 6.3129e+07, min = 1.23718e+07, max = 2.18423e+07', '# - flop rate = 0.990832 GFlop/s', '# - bytes moved = 191.229 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.00141 GByte/s', '# - solve arithmetic intensity = 0.330123 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 13.410s; elapsed, 8.871s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 14.350s; elapsed, 8.855s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.320s; elapsed, 8.866s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.640s; elapsed, 8.874s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 14.350s; elapsed, 8.855s, #calls: 1', 'TOTAL : CPU, 14.350s; elapsed, 8.855s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.320s; elapsed, 8.866s, #calls: 1', 'TOTAL : CPU, 17.320s; elapsed, 8.866s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 13.410s; elapsed, 8.871s, #calls: 1', 'TOTAL : CPU, 13.410s; elapsed, 8.871s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 35.370s; elapsed, 17.780s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.150s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.430s; elapsed, 19.261s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.640s; elapsed, 8.874s, #calls: 1', 'TOTAL : CPU, 72.700s; elapsed, 46.065s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 6327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 39.550s; elapsed, 10.129s', 'individual call time for EIGEN_LDLT: CPU, 21.500s; elapsed, 20.966s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.340s; elapsed, 0.129s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,133', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.274039', '# - matching time = 0.066931', '# - symmetrization time = 0.0401289', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,133', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0795271', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 159.055 MB', '# - factor time = 8.44067', '# - factor nonzeros = 19,881,835', '# - factor memory = 159.055 MB', '# - total flops = 4.94679e+10, min = 6.57516e+09, max = 1.95818e+10', '# - flop rate = 5.86066 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.19221e+10', '# --------------------------------------------', '# total = 5.19221e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.06632e-11\trel.res = 1.66282e-15\tbw.error = 5.08115e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0751741', '# - total flops = 6.3129e+07, min = 1.23718e+07, max = 2.18423e+07', '# - flop rate = 0.83977 GFlop/s', '# - bytes moved = 191.239 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.54395 GByte/s', '# - solve arithmetic intensity = 0.330104 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 21.240s; elapsed, 9.009s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 25.590s; elapsed, 8.991s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 35.190s; elapsed, 9.005s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 34.600s; elapsed, 9.011s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 25.590s; elapsed, 8.991s, #calls: 1', 'TOTAL : CPU, 25.590s; elapsed, 8.991s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 21.240s; elapsed, 9.009s, #calls: 1', 'TOTAL : CPU, 21.240s; elapsed, 9.009s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 35.190s; elapsed, 9.005s, #calls: 1', 'TOTAL : CPU, 35.190s; elapsed, 9.005s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 39.550s; elapsed, 10.129s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.340s; elapsed, 0.129s, #calls: 1', 'time for EIGEN_LDLT: CPU, 21.500s; elapsed, 20.966s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 34.600s; elapsed, 9.011s, #calls: 1', 'TOTAL : CPU, 95.990s; elapsed, 40.235s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 6327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 75.050s; elapsed, 9.462s', 'individual call time for EIGEN_LDLT: CPU, 20.740s; elapsed, 19.500s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.101s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,133', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.257689', '# - matching time = 0.0663118', '# - symmetrization time = 0.045496', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,133', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0819788', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 159.055 MB', '# - factor time = 9.93799', '# - factor nonzeros = 19,881,835', '# - factor memory = 159.055 MB', '# - total flops = 4.94679e+10, min = 6.57516e+09, max = 1.95818e+10', '# - flop rate = 4.97765 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.19221e+10', '# --------------------------------------------', '# total = 5.19221e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.06603e-11\trel.res = 1.6627e-15\tbw.error = 5.08115e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.124165', '# - total flops = 6.3129e+07, min = 1.23718e+07, max = 2.18423e+07', '# - flop rate = 0.508428 GFlop/s', '# - bytes moved = 191.261 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.54038 GByte/s', '# - solve arithmetic intensity = 0.330067 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 41.120s; elapsed, 10.553s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 53.570s; elapsed, 10.534s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 77.140s; elapsed, 10.546s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 83.600s; elapsed, 10.555s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 77.140s; elapsed, 10.546s, #calls: 1', 'TOTAL : CPU, 77.140s; elapsed, 10.546s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 53.570s; elapsed, 10.534s, #calls: 1', 'TOTAL : CPU, 53.570s; elapsed, 10.534s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 41.120s; elapsed, 10.553s, #calls: 1', 'TOTAL : CPU, 41.120s; elapsed, 10.553s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 75.050s; elapsed, 9.462s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.101s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.740s; elapsed, 19.500s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 83.600s; elapsed, 10.555s, #calls: 1', 'TOTAL : CPU, 179.840s; elapsed, 39.618s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 3163.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 29.110s; elapsed, 29.148s', 'individual call time for EIGEN_LDLT: CPU, 20.830s; elapsed, 20.839s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.200s; elapsed, 0.213s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 8 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,073', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.284664', '# - matching time = 0.0658951', '# - symmetrization time = 0.039274', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,073', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0608759', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 161.985 MB', '# - factor time = 10.2263', '# - factor nonzeros = 20,248,165', '# - factor memory = 161.985 MB', '# - total flops = 5.08363e+10, min = 1.44631e+06, max = 1.96465e+10', '# - flop rate = 4.97116 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.31844e+10', '# --------------------------------------------', '# total = 5.31844e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.54542e-11\trel.res = 1.44981e-15\tbw.error = 4.08301e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0758569', '# - total flops = 7.45165e+07, min = 37841, max = 2.17786e+07', '# - flop rate = 0.98233 GFlop/s', '# - bytes moved = 192.484 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.53747 GByte/s', '# - solve arithmetic intensity = 0.38713 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 10.740s; elapsed, 10.781s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 10.760s; elapsed, 10.783s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.770s; elapsed, 10.780s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.780s; elapsed, 10.779s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.760s; elapsed, 10.780s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.740s; elapsed, 10.778s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.750s; elapsed, 10.768s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.780s; elapsed, 10.786s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.780s; elapsed, 10.779s, #calls: 1', 'TOTAL : CPU, 10.780s; elapsed, 10.779s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.750s; elapsed, 10.768s, #calls: 1', 'TOTAL : CPU, 10.750s; elapsed, 10.768s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.740s; elapsed, 10.778s, #calls: 1', 'TOTAL : CPU, 10.740s; elapsed, 10.778s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 10.740s; elapsed, 10.781s, #calls: 1', 'TOTAL : CPU, 10.740s; elapsed, 10.781s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.770s; elapsed, 10.780s, #calls: 1', 'TOTAL : CPU, 10.770s; elapsed, 10.780s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.760s; elapsed, 10.780s, #calls: 1', 'TOTAL : CPU, 10.760s; elapsed, 10.780s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 10.760s; elapsed, 10.783s, #calls: 1', 'TOTAL : CPU, 10.760s; elapsed, 10.783s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 29.110s; elapsed, 29.148s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.200s; elapsed, 0.213s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.830s; elapsed, 20.839s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.780s; elapsed, 10.786s, #calls: 1', 'TOTAL : CPU, 60.920s; elapsed, 60.985s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 3163.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.300s; elapsed, 16.214s', 'individual call time for EIGEN_LDLT: CPU, 20.070s; elapsed, 19.907s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.171s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,073', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.277512', '# - matching time = 0.06619', '# - symmetrization time = 0.035372', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,073', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0705609', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 161.985 MB', '# - factor time = 8.36865', '# - factor nonzeros = 20,248,165', '# - factor memory = 161.985 MB', '# - total flops = 5.08364e+10, min = 1.44631e+06, max = 1.96465e+10', '# - flop rate = 6.07462 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.31844e+10', '# --------------------------------------------', '# total = 5.31844e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.53219e-11\trel.res = 1.4444e-15\tbw.error = 4.08301e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.073396', '# - total flops = 7.45165e+07, min = 37841, max = 2.17786e+07', '# - flop rate = 1.01527 GFlop/s', '# - bytes moved = 192.496 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.62271 GByte/s', '# - solve arithmetic intensity = 0.387107 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 11.030s; elapsed, 8.923s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 12.960s; elapsed, 8.923s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.980s; elapsed, 8.922s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.890s; elapsed, 8.921s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.110s; elapsed, 8.921s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.570s; elapsed, 8.921s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 14.810s; elapsed, 8.911s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.530s; elapsed, 8.927s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 16.890s; elapsed, 8.921s, #calls: 1', 'TOTAL : CPU, 16.890s; elapsed, 8.921s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.980s; elapsed, 8.922s, #calls: 1', 'TOTAL : CPU, 9.980s; elapsed, 8.922s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 11.030s; elapsed, 8.923s, #calls: 1', 'TOTAL : CPU, 11.030s; elapsed, 8.923s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 12.960s; elapsed, 8.923s, #calls: 1', 'TOTAL : CPU, 12.960s; elapsed, 8.923s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.110s; elapsed, 8.921s, #calls: 1', 'TOTAL : CPU, 10.110s; elapsed, 8.921s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.570s; elapsed, 8.921s, #calls: 1', 'TOTAL : CPU, 10.570s; elapsed, 8.921s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 14.810s; elapsed, 8.911s, #calls: 1', 'TOTAL : CPU, 14.810s; elapsed, 8.911s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.300s; elapsed, 16.214s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.171s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.070s; elapsed, 19.907s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.530s; elapsed, 8.927s, #calls: 1', 'TOTAL : CPU, 70.190s; elapsed, 45.218s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 3163.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 36.510s; elapsed, 9.238s', 'individual call time for EIGEN_LDLT: CPU, 21.190s; elapsed, 20.682s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.133s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,073', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.259164', '# - matching time = 0.0665901', '# - symmetrization time = 0.0399342', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,073', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0747271', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 161.985 MB', '# - factor time = 9.64344', '# - factor nonzeros = 20,248,165', '# - factor memory = 161.985 MB', '# - total flops = 5.08364e+10, min = 1.44731e+06, max = 1.96465e+10', '# - flop rate = 5.2716 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.31844e+10', '# --------------------------------------------', '# total = 5.31844e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.52804e-11\trel.res = 1.4427e-15\tbw.error = 4.08455e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0801442', '# - total flops = 7.45165e+07, min = 37841, max = 2.17786e+07', '# - flop rate = 0.929781 GFlop/s', '# - bytes moved = 192.511 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.40206 GByte/s', '# - solve arithmetic intensity = 0.387076 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 16.360s; elapsed, 10.203s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 22.640s; elapsed, 10.204s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 13.800s; elapsed, 10.204s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 15.260s; elapsed, 10.201s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 28.430s; elapsed, 10.191s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 13.360s; elapsed, 10.202s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 38.510s; elapsed, 10.203s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 40.050s; elapsed, 10.209s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 15.260s; elapsed, 10.201s, #calls: 1', 'TOTAL : CPU, 15.260s; elapsed, 10.201s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 16.360s; elapsed, 10.203s, #calls: 1', 'TOTAL : CPU, 16.360s; elapsed, 10.203s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 28.430s; elapsed, 10.191s, #calls: 1', 'TOTAL : CPU, 28.430s; elapsed, 10.191s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 22.640s; elapsed, 10.204s, #calls: 1', 'TOTAL : CPU, 22.640s; elapsed, 10.204s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 13.800s; elapsed, 10.204s, #calls: 1', 'TOTAL : CPU, 13.800s; elapsed, 10.204s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 38.510s; elapsed, 10.203s, #calls: 1', 'TOTAL : CPU, 38.510s; elapsed, 10.203s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 13.360s; elapsed, 10.202s, #calls: 1', 'TOTAL : CPU, 13.360s; elapsed, 10.202s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 36.510s; elapsed, 9.238s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.133s, #calls: 1', 'time for EIGEN_LDLT: CPU, 21.190s; elapsed, 20.682s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 40.050s; elapsed, 10.209s, #calls: 1', 'TOTAL : CPU, 98.130s; elapsed, 40.263s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 1581.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 25.380s; elapsed, 25.401s', 'individual call time for EIGEN_LDLT: CPU, 19.740s; elapsed, 19.749s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.189s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,073', '# - number of levels = 129', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.241346', '# - matching time = 0.0584431', '# - symmetrization time = 0.0200119', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,073', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.061177', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 161.985 MB', '# - factor time = 8.41239', '# - factor nonzeros = 20,248,165', '# - factor memory = 161.985 MB', '# - total flops = 5.08363e+10, min = 1.44631e+06, max = 1.96413e+10', '# - flop rate = 6.04303 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.29833e+10', '# --------------------------------------------', '# total = 5.29833e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.04348e-11\trel.res = 1.24455e-15\tbw.error = 3.45393e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0617669', '# - total flops = 1.17556e+08, min = 29840, max = 2.16901e+07', '# - flop rate = 1.90321 GFlop/s', '# - bytes moved = 184.706 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.99037 GByte/s', '# - solve arithmetic intensity = 0.636448 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 8.860s; elapsed, 8.866s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 8.770s; elapsed, 8.866s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 8.810s; elapsed, 8.874s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 8.790s; elapsed, 8.873s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.800s; elapsed, 8.876s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.850s; elapsed, 8.876s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 8.880s; elapsed, 8.875s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 8.820s; elapsed, 8.874s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 8.840s; elapsed, 8.874s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 8.840s; elapsed, 8.873s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 8.790s; elapsed, 8.875s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 8.790s; elapsed, 8.875s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 8.860s; elapsed, 8.873s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 8.810s; elapsed, 8.871s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.810s; elapsed, 8.875s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.870s; elapsed, 8.881s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 8.840s; elapsed, 8.874s, #calls: 1', 'TOTAL : CPU, 8.840s; elapsed, 8.874s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.850s; elapsed, 8.876s, #calls: 1', 'TOTAL : CPU, 8.850s; elapsed, 8.876s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 8.860s; elapsed, 8.866s, #calls: 1', 'TOTAL : CPU, 8.860s; elapsed, 8.866s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 8.790s; elapsed, 8.875s, #calls: 1', 'TOTAL : CPU, 8.790s; elapsed, 8.875s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 8.860s; elapsed, 8.873s, #calls: 1', 'TOTAL : CPU, 8.860s; elapsed, 8.873s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 8.810s; elapsed, 8.874s, #calls: 1', 'TOTAL : CPU, 8.810s; elapsed, 8.874s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 8.770s; elapsed, 8.866s, #calls: 1', 'TOTAL : CPU, 8.770s; elapsed, 8.866s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.800s; elapsed, 8.876s, #calls: 1', 'TOTAL : CPU, 8.800s; elapsed, 8.876s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 8.790s; elapsed, 8.875s, #calls: 1', 'TOTAL : CPU, 8.790s; elapsed, 8.875s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 8.820s; elapsed, 8.874s, #calls: 1', 'TOTAL : CPU, 8.820s; elapsed, 8.874s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 8.880s; elapsed, 8.875s, #calls: 1', 'TOTAL : CPU, 8.880s; elapsed, 8.875s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 8.840s; elapsed, 8.873s, #calls: 1', 'TOTAL : CPU, 8.840s; elapsed, 8.873s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 8.810s; elapsed, 8.871s, #calls: 1', 'TOTAL : CPU, 8.810s; elapsed, 8.871s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 8.790s; elapsed, 8.873s, #calls: 1', 'TOTAL : CPU, 8.790s; elapsed, 8.873s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.810s; elapsed, 8.875s, #calls: 1', 'TOTAL : CPU, 8.810s; elapsed, 8.875s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 25.380s; elapsed, 25.401s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.189s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.740s; elapsed, 19.749s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.870s; elapsed, 8.881s, #calls: 1', 'TOTAL : CPU, 54.180s; elapsed, 54.220s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 1581.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 35.670s; elapsed, 17.912s', 'individual call time for EIGEN_LDLT: CPU, 20.780s; elapsed, 20.625s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.167s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,073', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.288574', '# - matching time = 0.0691402', '# - symmetrization time = 0.0201359', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,073', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.068644', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 161.985 MB', '# - factor time = 9.27618', '# - factor nonzeros = 20,248,165', '# - factor memory = 161.985 MB', '# - total flops = 5.08364e+10, min = 1.44631e+06, max = 1.96413e+10', '# - flop rate = 5.48031 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.29833e+10', '# --------------------------------------------', '# total = 5.29833e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.03837e-11\trel.res = 1.24247e-15\tbw.error = 3.45393e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0689569', '# - total flops = 1.17556e+08, min = 29840, max = 2.16901e+07', '# - flop rate = 1.70477 GFlop/s', '# - bytes moved = 184.716 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.67872 GByte/s', '# - solve arithmetic intensity = 0.636411 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 12.600s; elapsed, 9.817s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 11.700s; elapsed, 9.817s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.770s; elapsed, 9.816s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 11.910s; elapsed, 9.815s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 13.380s; elapsed, 9.816s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.990s; elapsed, 9.815s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.880s; elapsed, 9.815s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 18.380s; elapsed, 9.814s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 11.690s; elapsed, 9.813s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 11.320s; elapsed, 9.813s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.130s; elapsed, 9.815s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.780s; elapsed, 9.814s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 11.500s; elapsed, 9.815s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 11.270s; elapsed, 9.808s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 15.390s; elapsed, 9.809s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.290s; elapsed, 9.820s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.770s; elapsed, 9.816s, #calls: 1', 'TOTAL : CPU, 10.770s; elapsed, 9.816s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.990s; elapsed, 9.815s, #calls: 1', 'TOTAL : CPU, 10.990s; elapsed, 9.815s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 12.600s; elapsed, 9.817s, #calls: 1', 'TOTAL : CPU, 12.600s; elapsed, 9.817s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 11.700s; elapsed, 9.817s, #calls: 1', 'TOTAL : CPU, 11.700s; elapsed, 9.817s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 15.390s; elapsed, 9.809s, #calls: 1', 'TOTAL : CPU, 15.390s; elapsed, 9.809s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.880s; elapsed, 9.815s, #calls: 1', 'TOTAL : CPU, 10.880s; elapsed, 9.815s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 11.910s; elapsed, 9.815s, #calls: 1', 'TOTAL : CPU, 11.910s; elapsed, 9.815s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 11.690s; elapsed, 9.813s, #calls: 1', 'TOTAL : CPU, 11.690s; elapsed, 9.813s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 18.380s; elapsed, 9.814s, #calls: 1', 'TOTAL : CPU, 18.380s; elapsed, 9.814s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.130s; elapsed, 9.815s, #calls: 1', 'TOTAL : CPU, 11.130s; elapsed, 9.815s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.780s; elapsed, 9.814s, #calls: 1', 'TOTAL : CPU, 10.780s; elapsed, 9.814s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 13.380s; elapsed, 9.816s, #calls: 1', 'TOTAL : CPU, 13.380s; elapsed, 9.816s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 11.320s; elapsed, 9.813s, #calls: 1', 'TOTAL : CPU, 11.320s; elapsed, 9.813s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 11.500s; elapsed, 9.815s, #calls: 1', 'TOTAL : CPU, 11.500s; elapsed, 9.815s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 11.270s; elapsed, 9.808s, #calls: 1', 'TOTAL : CPU, 11.270s; elapsed, 9.808s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 35.670s; elapsed, 17.912s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.167s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.780s; elapsed, 20.625s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.290s; elapsed, 9.820s, #calls: 1', 'TOTAL : CPU, 76.020s; elapsed, 48.523s']
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/A_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/b_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 23.520s; elapsed, 23.537s', 'individual call time for EIGEN_LDLT: CPU, 18.570s; elapsed, 18.586s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.169s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 23.520s; elapsed, 23.537s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.169s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.570s; elapsed, 18.586s, #calls: 1', 'TOTAL : CPU, 42.250s; elapsed, 42.291s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 28.530s; elapsed, 14.297s', 'individual call time for EIGEN_LDLT: CPU, 18.850s; elapsed, 18.674s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.161s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 28.530s; elapsed, 14.297s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.161s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.850s; elapsed, 18.674s, #calls: 1', 'TOTAL : CPU, 47.650s; elapsed, 33.132s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 38.220s; elapsed, 9.606s', 'individual call time for EIGEN_LDLT: CPU, 20.000s; elapsed, 19.481s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.320s; elapsed, 0.116s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 38.220s; elapsed, 9.606s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.320s; elapsed, 0.116s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.000s; elapsed, 19.481s, #calls: 1', 'TOTAL : CPU, 58.540s; elapsed, 29.203s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 77.030s; elapsed, 9.674s', 'individual call time for EIGEN_LDLT: CPU, 22.310s; elapsed, 21.060s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.460s; elapsed, 0.102s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 77.030s; elapsed, 9.674s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.460s; elapsed, 0.102s, #calls: 1', 'time for EIGEN_LDLT: CPU, 22.310s; elapsed, 21.060s, #calls: 1', 'TOTAL : CPU, 99.800s; elapsed, 30.836s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 162.640s; elapsed, 10.223s', 'individual call time for EIGEN_LDLT: CPU, 21.710s; elapsed, 19.069s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.760s; elapsed, 0.102s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 162.640s; elapsed, 10.223s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.760s; elapsed, 0.102s, #calls: 1', 'time for EIGEN_LDLT: CPU, 21.710s; elapsed, 19.069s, #calls: 1', 'TOTAL : CPU, 185.110s; elapsed, 29.394s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 24.610s; elapsed, 24.641s', 'individual call time for EIGEN_LDLT: CPU, 18.790s; elapsed, 18.800s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.175s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,781', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.211995', '# - matching time = 0.0547431', '# - symmetrization time = 0.0329671', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,781', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0706401', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 160.156 MB', '# - factor time = 14.3535', '# - factor nonzeros = 20,019,517', '# - factor memory = 160.156 MB', '# - total flops = 5.00254e+10, min = 2.29344e+10, max = 2.7091e+10', '# - flop rate = 3.48524 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.23886e+10', '# --------------------------------------------', '# total = 5.23886e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.51073e-11\trel.res = 1.84455e-15\tbw.error = 4.16326e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0870469', '# - total flops = 4.47923e+07, min = 2.05301e+07, max = 2.42622e+07', '# - flop rate = 0.514577 GFlop/s', '# - bytes moved = 229.689 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.63869 GByte/s', '# - solve arithmetic intensity = 0.195013 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 14.830s; elapsed, 14.840s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 14.850s; elapsed, 14.855s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 14.830s; elapsed, 14.840s, #calls: 1', 'TOTAL : CPU, 14.830s; elapsed, 14.840s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 24.610s; elapsed, 24.641s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.175s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.790s; elapsed, 18.800s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 14.850s; elapsed, 14.855s, #calls: 1', 'TOTAL : CPU, 58.430s; elapsed, 58.470s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 28.470s; elapsed, 14.268s', 'individual call time for EIGEN_LDLT: CPU, 18.850s; elapsed, 18.675s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.135s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,781', '# - number of levels = 132', '# - nd time = 0.209833', '# - matching time = 0.0536091', '# - symmetrization time = 0.03408', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,781', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.075856', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 160.156 MB', '# - factor time = 9.54952', '# - factor nonzeros = 20,019,517', '# - factor memory = 160.156 MB', '# - total flops = 5.00256e+10, min = 2.29344e+10, max = 2.70912e+10', '# - flop rate = 5.23855 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.23886e+10', '# --------------------------------------------', '# total = 5.23886e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.46879e-11\trel.res = 1.8274e-15\tbw.error = 4.16242e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0695362', '# - total flops = 4.481e+07, min = 2.05478e+07, max = 2.42622e+07', '# - flop rate = 0.644413 GFlop/s', '# - bytes moved = 230.398 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.31335 GByte/s', '# - solve arithmetic intensity = 0.19449 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.250s; elapsed, 10.026s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.980s; elapsed, 10.041s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.250s; elapsed, 10.026s, #calls: 1', 'TOTAL : CPU, 18.250s; elapsed, 10.026s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 28.470s; elapsed, 14.268s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.135s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.850s; elapsed, 18.675s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.980s; elapsed, 10.041s, #calls: 1', 'TOTAL : CPU, 67.520s; elapsed, 43.119s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 38.720s; elapsed, 9.737s', 'individual call time for EIGEN_LDLT: CPU, 19.170s; elapsed, 18.639s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.110s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,781', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.215914', '# - matching time = 0.0550861', '# - symmetrization time = 0.035147', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,781', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0830991', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 160.156 MB', '# - factor time = 9.78612', '# - factor nonzeros = 20,019,517', '# - factor memory = 160.156 MB', '# - total flops = 5.00256e+10, min = 2.29344e+10, max = 2.70912e+10', '# - flop rate = 5.1119 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.23886e+10', '# --------------------------------------------', '# total = 5.23886e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.46826e-11\trel.res = 1.82718e-15\tbw.error = 4.16509e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.111301', '# - total flops = 4.481e+07, min = 2.05478e+07, max = 2.42622e+07', '# - flop rate = 0.402601 GFlop/s', '# - bytes moved = 230.416 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.0702 GByte/s', '# - solve arithmetic intensity = 0.194475 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 37.610s; elapsed, 10.324s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 40.910s; elapsed, 10.339s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 37.610s; elapsed, 10.324s, #calls: 1', 'TOTAL : CPU, 37.610s; elapsed, 10.324s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 38.720s; elapsed, 9.737s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.110s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.170s; elapsed, 18.639s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 40.910s; elapsed, 10.339s, #calls: 1', 'TOTAL : CPU, 99.110s; elapsed, 38.826s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 80.670s; elapsed, 10.148s', 'individual call time for EIGEN_LDLT: CPU, 21.390s; elapsed, 20.161s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.440s; elapsed, 0.107s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,781', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.219375', '# - matching time = 0.063447', '# - symmetrization time = 0.035455', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,781', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0844729', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 160.156 MB', '# - factor time = 10.2043', '# - factor nonzeros = 20,019,517', '# - factor memory = 160.156 MB', '# - total flops = 5.00256e+10, min = 2.29344e+10, max = 2.70912e+10', '# - flop rate = 4.9024 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.23886e+10', '# --------------------------------------------', '# total = 5.23886e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.47005e-11\trel.res = 1.82791e-15\tbw.error = 4.16543e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.090358', '# - total flops = 4.481e+07, min = 2.05478e+07, max = 2.42622e+07', '# - flop rate = 0.495916 GFlop/s', '# - bytes moved = 230.453 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.55044 GByte/s', '# - solve arithmetic intensity = 0.194443 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 69.480s; elapsed, 10.738s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 85.150s; elapsed, 10.751s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 69.480s; elapsed, 10.738s, #calls: 1', 'TOTAL : CPU, 69.480s; elapsed, 10.738s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 80.670s; elapsed, 10.148s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.440s; elapsed, 0.107s, #calls: 1', 'time for EIGEN_LDLT: CPU, 21.390s; elapsed, 20.161s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 85.150s; elapsed, 10.751s, #calls: 1', 'TOTAL : CPU, 187.650s; elapsed, 41.166s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 162.150s; elapsed, 10.211s', 'individual call time for EIGEN_LDLT: CPU, 26.180s; elapsed, 23.539s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.860s; elapsed, 0.113s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,781', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.241594', '# - matching time = 0.0844431', '# - symmetrization time = 0.0435441', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,781', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.103542', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 160.156 MB', '# - factor time = 10.5717', '# - factor nonzeros = 20,019,517', '# - factor memory = 160.156 MB', '# - total flops = 5.00263e+10, min = 2.29344e+10, max = 2.70919e+10', '# - flop rate = 4.73211 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.23886e+10', '# --------------------------------------------', '# total = 5.23886e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.46636e-11\trel.res = 1.82641e-15\tbw.error = 4.16509e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.103272', '# - total flops = 4.48137e+07, min = 2.05515e+07, max = 2.42622e+07', '# - flop rate = 0.433939 GFlop/s', '# - bytes moved = 230.586 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.2328 GByte/s', '# - solve arithmetic intensity = 0.194347 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 149.690s; elapsed, 11.196s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 176.790s; elapsed, 11.202s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 149.690s; elapsed, 11.196s, #calls: 1', 'TOTAL : CPU, 149.690s; elapsed, 11.196s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 162.150s; elapsed, 10.211s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.860s; elapsed, 0.113s, #calls: 1', 'time for EIGEN_LDLT: CPU, 26.180s; elapsed, 23.539s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 176.790s; elapsed, 11.202s, #calls: 1', 'TOTAL : CPU, 365.980s; elapsed, 45.064s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 24.600s; elapsed, 24.614s', 'individual call time for EIGEN_LDLT: CPU, 19.330s; elapsed, 19.340s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.179s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,763', '# - number of levels = 131', '# - nd time = 0.230185', '# - matching time = 0.058701', '# - symmetrization time = 0.022408', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,763', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.065038', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 162.795 MB', '# - factor time = 11.2902', '# - factor nonzeros = 20,349,413', '# - factor memory = 162.795 MB', '# - total flops = 5.13294e+10, min = 2.45595e+06, max = 2.28929e+10', '# - flop rate = 4.54638 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.36486e+10', '# --------------------------------------------', '# total = 5.36486e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.31158e-11\trel.res = 1.76311e-15\tbw.error = 4.50443e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0651221', '# - total flops = 5.20842e+07, min = 54847, max = 2.42155e+07', '# - flop rate = 0.799793 GFlop/s', '# - bytes moved = 202.404 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.10806 GByte/s', '# - solve arithmetic intensity = 0.257329 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 11.720s; elapsed, 11.745s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 11.730s; elapsed, 11.742s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.720s; elapsed, 11.747s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 11.740s; elapsed, 11.755s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 11.720s; elapsed, 11.745s, #calls: 1', 'TOTAL : CPU, 11.720s; elapsed, 11.745s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 11.730s; elapsed, 11.742s, #calls: 1', 'TOTAL : CPU, 11.730s; elapsed, 11.742s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.720s; elapsed, 11.747s, #calls: 1', 'TOTAL : CPU, 11.720s; elapsed, 11.747s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 24.600s; elapsed, 24.614s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.179s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.330s; elapsed, 19.340s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 11.740s; elapsed, 11.755s, #calls: 1', 'TOTAL : CPU, 55.840s; elapsed, 55.886s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 29.550s; elapsed, 14.820s', 'individual call time for EIGEN_LDLT: CPU, 19.370s; elapsed, 19.205s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.131s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,763', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.224222', '# - matching time = 0.0576', '# - symmetrization time = 0.0244329', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,763', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0728419', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 162.795 MB', '# - factor time = 9.77922', '# - factor nonzeros = 20,349,413', '# - factor memory = 162.795 MB', '# - total flops = 5.13295e+10, min = 2.45595e+06, max = 2.28929e+10', '# - flop rate = 5.24883 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.36486e+10', '# --------------------------------------------', '# total = 5.36486e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.1948e-11\trel.res = 1.71536e-15\tbw.error = 4.50638e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0607681', '# - total flops = 5.20895e+07, min = 54847, max = 2.42155e+07', '# - flop rate = 0.857185 GFlop/s', '# - bytes moved = 202.615 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.33423 GByte/s', '# - solve arithmetic intensity = 0.257087 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 15.360s; elapsed, 10.240s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.490s; elapsed, 10.241s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.430s; elapsed, 10.234s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.220s; elapsed, 10.248s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.490s; elapsed, 10.241s, #calls: 1', 'TOTAL : CPU, 11.490s; elapsed, 10.241s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 18.430s; elapsed, 10.234s, #calls: 1', 'TOTAL : CPU, 18.430s; elapsed, 10.234s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 15.360s; elapsed, 10.240s, #calls: 1', 'TOTAL : CPU, 15.360s; elapsed, 10.240s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 29.550s; elapsed, 14.820s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.220s; elapsed, 0.131s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.370s; elapsed, 19.205s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 20.220s; elapsed, 10.248s, #calls: 1', 'TOTAL : CPU, 69.360s; elapsed, 44.404s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 38.520s; elapsed, 9.706s', 'individual call time for EIGEN_LDLT: CPU, 20.060s; elapsed, 19.544s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.116s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,763', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.226176', '# - matching time = 0.059078', '# - symmetrization time = 0.026844', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,763', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0758209', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 162.795 MB', '# - factor time = 9.47062', '# - factor nonzeros = 20,349,413', '# - factor memory = 162.795 MB', '# - total flops = 5.13295e+10, min = 2.45658e+06, max = 2.28929e+10', '# - flop rate = 5.41986 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.36486e+10', '# --------------------------------------------', '# total = 5.36486e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.19742e-11\trel.res = 1.71643e-15\tbw.error = 4.50833e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0787191', '# - total flops = 5.20895e+07, min = 54847, max = 2.42155e+07', '# - flop rate = 0.661714 GFlop/s', '# - bytes moved = 202.628 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.57406 GByte/s', '# - solve arithmetic intensity = 0.25707 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 13.540s; elapsed, 9.966s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 25.170s; elapsed, 9.964s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 34.530s; elapsed, 9.959s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 39.270s; elapsed, 9.973s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 13.540s; elapsed, 9.966s, #calls: 1', 'TOTAL : CPU, 13.540s; elapsed, 9.966s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 25.170s; elapsed, 9.964s, #calls: 1', 'TOTAL : CPU, 25.170s; elapsed, 9.964s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 34.530s; elapsed, 9.959s, #calls: 1', 'TOTAL : CPU, 34.530s; elapsed, 9.959s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 38.520s; elapsed, 9.706s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.116s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.060s; elapsed, 19.544s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 39.270s; elapsed, 9.973s, #calls: 1', 'TOTAL : CPU, 98.150s; elapsed, 39.338s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 80.370s; elapsed, 10.135s', 'individual call time for EIGEN_LDLT: CPU, 22.740s; elapsed, 21.478s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.120s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,763', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.249073', '# - matching time = 0.0699379', '# - symmetrization time = 0.0275559', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,763', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.08253', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 162.795 MB', '# - factor time = 10.3077', '# - factor nonzeros = 20,349,413', '# - factor memory = 162.795 MB', '# - total flops = 5.13295e+10, min = 2.45658e+06, max = 2.28929e+10', '# - flop rate = 4.97972 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.36486e+10', '# --------------------------------------------', '# total = 5.36486e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.19328e-11\trel.res = 1.71474e-15\tbw.error = 4.50833e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0784569', '# - total flops = 5.20895e+07, min = 54847, max = 2.42155e+07', '# - flop rate = 0.663926 GFlop/s', '# - bytes moved = 202.654 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.583 GByte/s', '# - solve arithmetic intensity = 0.257036 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 48.540s; elapsed, 10.840s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 19.250s; elapsed, 10.842s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 69.140s; elapsed, 10.837s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 84.680s; elapsed, 10.847s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 19.250s; elapsed, 10.842s, #calls: 1', 'TOTAL : CPU, 19.250s; elapsed, 10.842s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 48.540s; elapsed, 10.840s, #calls: 1', 'TOTAL : CPU, 48.540s; elapsed, 10.840s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 69.140s; elapsed, 10.837s, #calls: 1', 'TOTAL : CPU, 69.140s; elapsed, 10.837s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 80.370s; elapsed, 10.135s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.120s, #calls: 1', 'time for EIGEN_LDLT: CPU, 22.740s; elapsed, 21.478s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 84.680s; elapsed, 10.847s, #calls: 1', 'TOTAL : CPU, 188.310s; elapsed, 42.580s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 2913.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 24.680s; elapsed, 24.707s', 'individual call time for EIGEN_LDLT: CPU, 18.900s; elapsed, 18.913s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.184s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,761', '# - number of levels = 130', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.213747', '# - matching time = 0.0555999', '# - symmetrization time = 0.0225809', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,761', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0533209', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 163.769 MB', '# - factor time = 9.13801', '# - factor nonzeros = 20,471,129', '# - factor memory = 163.769 MB', '# - total flops = 5.19135e+10, min = 2.03243e+06, max = 2.28029e+10', '# - flop rate = 5.68105 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.41139e+10', '# --------------------------------------------', '# total = 5.41139e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.69221e-11\trel.res = 1.50984e-15\tbw.error = 3.89873e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0585361', '# - total flops = 6.8532e+07, min = 38492, max = 2.40848e+07', '# - flop rate = 1.17077 GFlop/s', '# - bytes moved = 190.4 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.2527 GByte/s', '# - solve arithmetic intensity = 0.359936 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.540s; elapsed, 9.557s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 9.500s; elapsed, 9.559s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 9.520s; elapsed, 9.552s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.550s; elapsed, 9.557s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.500s; elapsed, 9.556s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 9.540s; elapsed, 9.559s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 9.510s; elapsed, 9.550s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.550s; elapsed, 9.562s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 9.540s; elapsed, 9.559s, #calls: 1', 'TOTAL : CPU, 9.540s; elapsed, 9.559s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.550s; elapsed, 9.557s, #calls: 1', 'TOTAL : CPU, 9.550s; elapsed, 9.557s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.540s; elapsed, 9.557s, #calls: 1', 'TOTAL : CPU, 9.540s; elapsed, 9.557s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 9.520s; elapsed, 9.552s, #calls: 1', 'TOTAL : CPU, 9.520s; elapsed, 9.552s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.500s; elapsed, 9.556s, #calls: 1', 'TOTAL : CPU, 9.500s; elapsed, 9.556s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 9.510s; elapsed, 9.550s, #calls: 1', 'TOTAL : CPU, 9.510s; elapsed, 9.550s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 9.500s; elapsed, 9.559s, #calls: 1', 'TOTAL : CPU, 9.500s; elapsed, 9.559s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 24.680s; elapsed, 24.707s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.184s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.900s; elapsed, 18.913s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.550s; elapsed, 9.562s, #calls: 1', 'TOTAL : CPU, 53.320s; elapsed, 53.366s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 2913.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 34.790s; elapsed, 17.481s', 'individual call time for EIGEN_LDLT: CPU, 19.970s; elapsed, 19.797s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.175s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,761', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.253005', '# - matching time = 0.0651538', '# - symmetrization time = 0.026468', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,761', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0689108', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 163.769 MB', '# - factor time = 9.58467', '# - factor nonzeros = 20,471,129', '# - factor memory = 163.769 MB', '# - total flops = 5.19135e+10, min = 2.03243e+06, max = 2.28029e+10', '# - flop rate = 5.41631 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.41139e+10', '# --------------------------------------------', '# total = 5.41139e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.71507e-11\trel.res = 1.51918e-15\tbw.error = 3.89716e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0740201', '# - total flops = 6.8532e+07, min = 38492, max = 2.40848e+07', '# - flop rate = 0.925856 GFlop/s', '# - bytes moved = 190.413 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.57245 GByte/s', '# - solve arithmetic intensity = 0.359911 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 13.760s; elapsed, 10.100s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 11.770s; elapsed, 10.087s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.310s; elapsed, 10.097s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 12.000s; elapsed, 10.099s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 14.890s; elapsed, 10.090s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 11.180s; elapsed, 10.095s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 18.410s; elapsed, 10.097s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.910s; elapsed, 10.102s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 12.000s; elapsed, 10.099s, #calls: 1', 'TOTAL : CPU, 12.000s; elapsed, 10.099s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 13.760s; elapsed, 10.100s, #calls: 1', 'TOTAL : CPU, 13.760s; elapsed, 10.100s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 18.410s; elapsed, 10.097s, #calls: 1', 'TOTAL : CPU, 18.410s; elapsed, 10.097s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 11.180s; elapsed, 10.095s, #calls: 1', 'TOTAL : CPU, 11.180s; elapsed, 10.095s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.310s; elapsed, 10.097s, #calls: 1', 'TOTAL : CPU, 11.310s; elapsed, 10.097s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 11.770s; elapsed, 10.087s, #calls: 1', 'TOTAL : CPU, 11.770s; elapsed, 10.087s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 14.890s; elapsed, 10.090s, #calls: 1', 'TOTAL : CPU, 14.890s; elapsed, 10.090s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 34.790s; elapsed, 17.481s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.175s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.970s; elapsed, 19.797s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.910s; elapsed, 10.102s, #calls: 1', 'TOTAL : CPU, 74.960s; elapsed, 47.556s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 2913.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 40.000s; elapsed, 10.071s', 'individual call time for EIGEN_LDLT: CPU, 19.230s; elapsed, 18.711s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.340s; elapsed, 0.123s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,761', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.244784', '# - matching time = 0.0656981', '# - symmetrization time = 0.0262339', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,761', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0717001', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 163.769 MB', '# - factor time = 9.75133', '# - factor nonzeros = 20,471,129', '# - factor memory = 163.769 MB', '# - total flops = 5.19135e+10, min = 2.03308e+06, max = 2.28029e+10', '# - flop rate = 5.32374 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.41139e+10', '# --------------------------------------------', '# total = 5.41139e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.71433e-11\trel.res = 1.51888e-15\tbw.error = 3.89716e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0730031', '# - total flops = 6.8532e+07, min = 38492, max = 2.40848e+07', '# - flop rate = 0.938755 GFlop/s', '# - bytes moved = 190.43 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.60852 GByte/s', '# - solve arithmetic intensity = 0.35988 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 19.620s; elapsed, 10.258s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 15.100s; elapsed, 10.248s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 24.750s; elapsed, 10.249s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 13.940s; elapsed, 10.255s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 35.700s; elapsed, 10.254s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 15.650s; elapsed, 10.254s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 13.500s; elapsed, 10.254s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 40.350s; elapsed, 10.260s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 15.100s; elapsed, 10.248s, #calls: 1', 'TOTAL : CPU, 15.100s; elapsed, 10.248s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 19.620s; elapsed, 10.258s, #calls: 1', 'TOTAL : CPU, 19.620s; elapsed, 10.258s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 24.750s; elapsed, 10.249s, #calls: 1', 'TOTAL : CPU, 24.750s; elapsed, 10.249s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 35.700s; elapsed, 10.254s, #calls: 1', 'TOTAL : CPU, 35.700s; elapsed, 10.254s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 13.940s; elapsed, 10.255s, #calls: 1', 'TOTAL : CPU, 13.940s; elapsed, 10.255s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 15.650s; elapsed, 10.254s, #calls: 1', 'TOTAL : CPU, 15.650s; elapsed, 10.254s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 13.500s; elapsed, 10.254s, #calls: 1', 'TOTAL : CPU, 13.500s; elapsed, 10.254s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 40.000s; elapsed, 10.071s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.340s; elapsed, 0.123s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.230s; elapsed, 18.711s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 40.350s; elapsed, 10.260s, #calls: 1', 'TOTAL : CPU, 99.920s; elapsed, 39.165s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 1456.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 25.260s; elapsed, 25.264s', 'individual call time for EIGEN_LDLT: CPU, 19.580s; elapsed, 19.589s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.182s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,761', '# - number of levels = 129', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.227476', '# - matching time = 0.0586209', '# - symmetrization time = 0.0169449', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,761', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0587652', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 163.769 MB', '# - factor time = 9.89155', '# - factor nonzeros = 20,471,129', '# - factor memory = 163.769 MB', '# - total flops = 5.19135e+10, min = 2.03243e+06, max = 2.27913e+10', '# - flop rate = 5.24827 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.39135e+10', '# --------------------------------------------', '# total = 5.39135e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.04218e-11\trel.res = 1.24402e-15\tbw.error = 3.17516e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.048768', '# - total flops = 1.09026e+08, min = 31207, max = 2.39472e+07', '# - flop rate = 2.23559 GFlop/s', '# - bytes moved = 181.908 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.73006 GByte/s', '# - solve arithmetic intensity = 0.599346 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 10.300s; elapsed, 10.313s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 10.290s; elapsed, 10.316s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 10.240s; elapsed, 10.319s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 10.270s; elapsed, 10.315s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.250s; elapsed, 10.320s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 10.270s; elapsed, 10.317s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.300s; elapsed, 10.318s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.320s; elapsed, 10.317s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 10.280s; elapsed, 10.317s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 10.310s; elapsed, 10.318s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 10.310s; elapsed, 10.317s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 10.280s; elapsed, 10.311s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.300s; elapsed, 10.318s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.270s; elapsed, 10.318s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 10.250s; elapsed, 10.317s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.320s; elapsed, 10.324s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 10.300s; elapsed, 10.313s, #calls: 1', 'TOTAL : CPU, 10.300s; elapsed, 10.313s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.250s; elapsed, 10.320s, #calls: 1', 'TOTAL : CPU, 10.250s; elapsed, 10.320s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 10.240s; elapsed, 10.319s, #calls: 1', 'TOTAL : CPU, 10.240s; elapsed, 10.319s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.270s; elapsed, 10.318s, #calls: 1', 'TOTAL : CPU, 10.270s; elapsed, 10.318s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 10.310s; elapsed, 10.318s, #calls: 1', 'TOTAL : CPU, 10.310s; elapsed, 10.318s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.300s; elapsed, 10.318s, #calls: 1', 'TOTAL : CPU, 10.300s; elapsed, 10.318s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 10.270s; elapsed, 10.315s, #calls: 1', 'TOTAL : CPU, 10.270s; elapsed, 10.315s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 10.290s; elapsed, 10.316s, #calls: 1', 'TOTAL : CPU, 10.290s; elapsed, 10.316s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.320s; elapsed, 10.317s, #calls: 1', 'TOTAL : CPU, 10.320s; elapsed, 10.317s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 10.310s; elapsed, 10.317s, #calls: 1', 'TOTAL : CPU, 10.310s; elapsed, 10.317s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 10.250s; elapsed, 10.317s, #calls: 1', 'TOTAL : CPU, 10.250s; elapsed, 10.317s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.300s; elapsed, 10.318s, #calls: 1', 'TOTAL : CPU, 10.300s; elapsed, 10.318s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 10.280s; elapsed, 10.311s, #calls: 1', 'TOTAL : CPU, 10.280s; elapsed, 10.311s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 10.280s; elapsed, 10.317s, #calls: 1', 'TOTAL : CPU, 10.280s; elapsed, 10.317s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 10.270s; elapsed, 10.317s, #calls: 1', 'TOTAL : CPU, 10.270s; elapsed, 10.317s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 25.260s; elapsed, 25.264s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.182s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.580s; elapsed, 19.589s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.320s; elapsed, 10.324s, #calls: 1', 'TOTAL : CPU, 55.350s; elapsed, 55.360s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 1456.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 30.620s; elapsed, 15.376s', 'individual call time for EIGEN_LDLT: CPU, 19.500s; elapsed, 19.337s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.157s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 491,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 7,761', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.266438', '# - matching time = 0.0650461', '# - symmetrization time = 0.020874', '# symbolic factorization:', '# - nr of dense Frontal matrices = 7,761', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0764129', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 163.769 MB', '# - factor time = 10.1717', '# - factor nonzeros = 20,471,129', '# - factor memory = 163.769 MB', '# - total flops = 5.19135e+10, min = 2.03243e+06, max = 2.27913e+10', '# - flop rate = 5.10373 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.39135e+10', '# --------------------------------------------', '# total = 5.39135e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.01963e-11\trel.res = 1.2348e-15\tbw.error = 3.17394e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0554342', '# - total flops = 1.09026e+08, min = 31207, max = 2.39472e+07', '# - flop rate = 1.96676 GFlop/s', '# - bytes moved = 181.917 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.28168 GByte/s', '# - solve arithmetic intensity = 0.599314 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 12.580s; elapsed, 10.675s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 11.680s; elapsed, 10.676s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 12.420s; elapsed, 10.676s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 13.910s; elapsed, 10.677s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 14.140s; elapsed, 10.678s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 12.070s; elapsed, 10.677s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 11.930s; elapsed, 10.679s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 12.140s; elapsed, 10.671s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 15.450s; elapsed, 10.671s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 11.860s; elapsed, 10.678s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 11.690s; elapsed, 10.677s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 11.650s; elapsed, 10.677s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 18.970s; elapsed, 10.677s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 12.620s; elapsed, 10.677s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 12.240s; elapsed, 10.675s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 21.050s; elapsed, 10.683s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 12.420s; elapsed, 10.676s, #calls: 1', 'TOTAL : CPU, 12.420s; elapsed, 10.676s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 11.680s; elapsed, 10.676s, #calls: 1', 'TOTAL : CPU, 11.680s; elapsed, 10.676s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 11.930s; elapsed, 10.679s, #calls: 1', 'TOTAL : CPU, 11.930s; elapsed, 10.679s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 13.910s; elapsed, 10.677s, #calls: 1', 'TOTAL : CPU, 13.910s; elapsed, 10.677s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 12.580s; elapsed, 10.675s, #calls: 1', 'TOTAL : CPU, 12.580s; elapsed, 10.675s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 12.140s; elapsed, 10.671s, #calls: 1', 'TOTAL : CPU, 12.140s; elapsed, 10.671s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 18.970s; elapsed, 10.677s, #calls: 1', 'TOTAL : CPU, 18.970s; elapsed, 10.677s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 11.860s; elapsed, 10.678s, #calls: 1', 'TOTAL : CPU, 11.860s; elapsed, 10.678s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 11.650s; elapsed, 10.677s, #calls: 1', 'TOTAL : CPU, 11.650s; elapsed, 10.677s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 12.620s; elapsed, 10.677s, #calls: 1', 'TOTAL : CPU, 12.620s; elapsed, 10.677s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 15.450s; elapsed, 10.671s, #calls: 1', 'TOTAL : CPU, 15.450s; elapsed, 10.671s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 11.690s; elapsed, 10.677s, #calls: 1', 'TOTAL : CPU, 11.690s; elapsed, 10.677s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 12.070s; elapsed, 10.677s, #calls: 1', 'TOTAL : CPU, 12.070s; elapsed, 10.677s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 12.240s; elapsed, 10.675s, #calls: 1', 'TOTAL : CPU, 12.240s; elapsed, 10.675s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 14.140s; elapsed, 10.678s, #calls: 1', 'TOTAL : CPU, 14.140s; elapsed, 10.678s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 30.620s; elapsed, 15.376s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.157s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.500s; elapsed, 19.337s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 21.050s; elapsed, 10.683s, #calls: 1', 'TOTAL : CPU, 71.430s; elapsed, 45.554s']
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.650s; elapsed, 5.657s', 'individual call time for EIGEN_LDLT: CPU, 2.460s; elapsed, 2.462s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.053s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.650s; elapsed, 5.657s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.053s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.460s; elapsed, 2.462s, #calls: 1', 'TOTAL : CPU, 8.170s; elapsed, 8.172s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.560s; elapsed, 3.786s', 'individual call time for EIGEN_LDLT: CPU, 2.690s; elapsed, 2.512s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.046s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.560s; elapsed, 3.786s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.046s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.690s; elapsed, 2.512s, #calls: 1', 'TOTAL : CPU, 10.320s; elapsed, 6.344s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 16.070s; elapsed, 4.020s', 'individual call time for EIGEN_LDLT: CPU, 5.700s; elapsed, 5.158s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.046s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 16.070s; elapsed, 4.020s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.046s, #calls: 1', 'time for EIGEN_LDLT: CPU, 5.700s; elapsed, 5.158s, #calls: 1', 'TOTAL : CPU, 21.870s; elapsed, 9.224s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 18.460s; elapsed, 2.316s', 'individual call time for EIGEN_LDLT: CPU, 6.120s; elapsed, 4.881s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.041s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 18.460s; elapsed, 2.316s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.041s, #calls: 1', 'time for EIGEN_LDLT: CPU, 6.120s; elapsed, 4.881s, #calls: 1', 'TOTAL : CPU, 24.740s; elapsed, 7.238s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 38.490s; elapsed, 2.414s', 'individual call time for EIGEN_LDLT: CPU, 6.120s; elapsed, 3.489s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.038s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 38.490s; elapsed, 2.414s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.038s, #calls: 1', 'time for EIGEN_LDLT: CPU, 6.120s; elapsed, 3.489s, #calls: 1', 'TOTAL : CPU, 44.870s; elapsed, 5.940s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.900s; elapsed, 5.906s', 'individual call time for EIGEN_LDLT: CPU, 2.550s; elapsed, 2.549s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.055s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,139', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.188117', '# - matching time = 0.0319049', '# - symmetrization time = 0.016331', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.043174', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.2791 MB', '# - factor time = 4.04599', '# - factor nonzeros = 6,034,893', '# - factor memory = 48.2791 MB', '# - total flops = 7.11168e+09, min = 2.51306e+09, max = 4.59862e+09', '# - flop rate = 1.75771 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.7315e+09', '# --------------------------------------------', '# total = 7.7315e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.18684e-11\trel.res = 1.36409e-15\tbw.error = 3.68017e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.04597', '# - total flops = 1.45529e+07, min = 6.25352e+06, max = 8.29941e+06', '# - flop rate = 0.316575 GFlop/s', '# - bytes moved = 107.14 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.33066 GByte/s', '# - solve arithmetic intensity = 0.13583 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.380s; elapsed, 4.394s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.410s; elapsed, 4.405s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.380s; elapsed, 4.394s, #calls: 1', 'TOTAL : CPU, 4.380s; elapsed, 4.394s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.900s; elapsed, 5.906s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.055s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.550s; elapsed, 2.549s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.410s; elapsed, 4.405s, #calls: 1', 'TOTAL : CPU, 12.910s; elapsed, 12.915s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.770s; elapsed, 3.890s', 'individual call time for EIGEN_LDLT: CPU, 2.780s; elapsed, 2.605s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.052s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,139', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.186938', '# - matching time = 0.0320699', '# - symmetrization time = 0.0165391', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0417151', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.2791 MB', '# - factor time = 2.17121', '# - factor nonzeros = 6,034,893', '# - factor memory = 48.2791 MB', '# - total flops = 7.11174e+09, min = 2.51306e+09, max = 4.59868e+09', '# - flop rate = 3.27547 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.7315e+09', '# --------------------------------------------', '# total = 7.7315e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.16628e-11\trel.res = 1.3553e-15\tbw.error = 3.67931e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0402181', '# - total flops = 1.45583e+07, min = 6.25352e+06, max = 8.30482e+06', '# - flop rate = 0.361985 GFlop/s', '# - bytes moved = 107.353 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.66928 GByte/s', '# - solve arithmetic intensity = 0.135611 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 5.000s; elapsed, 2.519s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 5.040s; elapsed, 2.529s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 5.000s; elapsed, 2.519s, #calls: 1', 'TOTAL : CPU, 5.000s; elapsed, 2.519s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.770s; elapsed, 3.890s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.052s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.780s; elapsed, 2.605s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 5.040s; elapsed, 2.529s, #calls: 1', 'TOTAL : CPU, 15.670s; elapsed, 9.076s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 8.790s; elapsed, 2.203s', 'individual call time for EIGEN_LDLT: CPU, 4.090s; elapsed, 3.549s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.042s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 2CPP -2', ' MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,139', '# - number of levels = 132', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.173058', '# - matching time = 0.030426', '# - symmetrization time = 0.0179629', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.046849', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.2791 MB', '# - factor time = 2.19404', '# - factor nonzeros = 6,034,893', '# - factor memory = 48.2791 MB', '# - total flops = 7.11174e+09, min = 2.51306e+09, max = 4.59868e+09', '# - flop rate = 3.24139 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.7315e+09', '# --------------------------------------------', '# total = 7.7315e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.16386e-11\trel.res = 1.35426e-15\tbw.error = 3.67931e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.058646', '# - total flops = 1.45583e+07, min = 6.25352e+06, max = 8.30482e+06', '# - flop rate = 0.248241 GFlop/s', '# - bytes moved = 107.363 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.8307 GByte/s', '# - solve arithmetic intensity = 0.135599 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.170s; elapsed, 2.553s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.630s; elapsed, 2.564s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.170s; elapsed, 2.553s, #calls: 1', 'TOTAL : CPU, 10.170s; elapsed, 2.553s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 8.790s; elapsed, 2.203s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.042s, #calls: 1', 'time for EIGEN_LDLT: CPU, 4.090s; elapsed, 3.549s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.630s; elapsed, 2.564s, #calls: 1', 'TOTAL : CPU, 22.610s; elapsed, 8.358s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 20.040s; elapsed, 2.514s', 'individual call time for EIGEN_LDLT: CPU, 3.780s; elapsed, 2.551s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.040s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,139', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.19079', '# - matching time = 0.033514', '# - symmetrization time = 0.01736', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0465379', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.2791 MB', '# - factor time = 2.06418', '# - factor nonzeros = 6,034,893', '# - factor memory = 48.2791 MB', '# - total flops = 7.11175e+09, min = 2.51306e+09, max = 4.59868e+09', '# - flop rate = 3.44531 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.7315e+09', '# --------------------------------------------', '# total = 7.7315e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.167e-11\trel.res = 1.3556e-15\tbw.error = 3.67931e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0531371', '# - total flops = 1.45583e+07, min = 6.25352e+06, max = 8.30482e+06', '# - flop rate = 0.273977 GFlop/s', '# - bytes moved = 107.383 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.02088 GByte/s', '# - solve arithmetic intensity = 0.135573 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 19.290s; elapsed, 2.435s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.530s; elapsed, 2.445s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 19.290s; elapsed, 2.435s, #calls: 1', 'TOTAL : CPU, 19.290s; elapsed, 2.435s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 20.040s; elapsed, 2.514s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.040s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.780s; elapsed, 2.551s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.530s; elapsed, 2.445s, #calls: 1', 'TOTAL : CPU, 42.500s; elapsed, 7.549s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 39.420s; elapsed, 2.479s', 'individual call time for EIGEN_LDLT: CPU, 7.150s; elapsed, 4.514s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.040s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,139', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.216463', '# - matching time = 0.0463619', '# - symmetrization time = 0.0207109', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.059644', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.2791 MB', '# - factor time = 2.54578', '# - factor nonzeros = 6,034,893', '# - factor memory = 48.2791 MB', '# - total flops = 7.11193e+09, min = 2.51306e+09, max = 4.59887e+09', '# - flop rate = 2.79361 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.7315e+09', '# --------------------------------------------', '# total = 7.7315e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.18243e-11\trel.res = 1.36221e-15\tbw.error = 3.74487e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.052789', '# - total flops = 1.45583e+07, min = 6.25352e+06, max = 8.30482e+06', '# - flop rate = 0.275784 GFlop/s', '# - bytes moved = 107.429 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.03506 GByte/s', '# - solve arithmetic intensity = 0.135516 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 39.880s; elapsed, 2.980s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 47.060s; elapsed, 2.990s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 39.880s; elapsed, 2.980s, #calls: 1', 'TOTAL : CPU, 39.880s; elapsed, 2.980s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 39.420s; elapsed, 2.479s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.040s, #calls: 1', 'time for EIGEN_LDLT: CPU, 7.150s; elapsed, 4.514s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 47.060s; elapsed, 2.990s, #calls: 1', 'TOTAL : CPU, 93.910s; elapsed, 10.022s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 5.910s; elapsed, 5.910s', 'individual call time for EIGEN_LDLT: CPU, 2.530s; elapsed, 2.527s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.058s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,139', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.175676', '# - matching time = 0.029875', '# - symmetrization time = 0.0196559', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.038007', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.2791 MB', '# - factor time = 2.32313', '# - factor nonzeros = 6,034,893', '# - factor memory = 48.2791 MB', '# - total flops = 7.11168e+09, min = 8.93654e+08, max = 2.51306e+09', '# - flop rate = 3.06125 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.71747e+09', '# --------------------------------------------', '# total = 7.71747e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.72392e-11\trel.res = 1.16595e-15\tbw.error = 2.55499e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0380909', '# - total flops = 1.9286e+07, min = 3.82026e+06, max = 6.22438e+06', '# - flop rate = 0.506314 GFlop/s', '# - bytes moved = 94.649 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.48482 GByte/s', '# - solve arithmetic intensity = 0.203763 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.630s; elapsed, 2.644s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.620s; elapsed, 2.642s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.630s; elapsed, 2.635s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.650s; elapsed, 2.649s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.630s; elapsed, 2.644s, #calls: 1', 'TOTAL : CPU, 2.630s; elapsed, 2.644s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.630s; elapsed, 2.635s, #calls: 1', 'TOTAL : CPU, 2.630s; elapsed, 2.635s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.620s; elapsed, 2.642s, #calls: 1', 'TOTAL : CPU, 2.620s; elapsed, 2.642s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 5.910s; elapsed, 5.910s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.058s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.530s; elapsed, 2.527s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.650s; elapsed, 2.649s, #calls: 1', 'TOTAL : CPU, 11.140s; elapsed, 11.144s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.630s; elapsed, 3.815s', 'individual call time for EIGEN_LDLT: CPU, 2.760s; elapsed, 2.578s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.055s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,139', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.191717', '# - matching time = 0.033134', '# - symmetrization time = 0.0183499', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0401731', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.2791 MB', '# - factor time = 2.22883', '# - factor nonzeros = 6,034,893', '# - factor memory = 48.2791 MB', '# - total flops = 7.11169e+09, min = 8.93654e+08, max = 2.51306e+09', '# - flop rate = 3.19077 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.71747e+09', '# --------------------------------------------', '# total = 7.71747e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.71856e-11\trel.res = 1.16365e-15\tbw.error = 2.55499e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0364931', '# - total flops = 1.9286e+07, min = 3.82026e+06, max = 6.22438e+06', '# - flop rate = 0.528483 GFlop/s', '# - bytes moved = 94.654 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.59375 GByte/s', '# - solve arithmetic intensity = 0.203752 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.120s; elapsed, 2.573s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.580s; elapsed, 2.565s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 5.100s; elapsed, 2.572s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.920s; elapsed, 2.577s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.120s; elapsed, 2.573s, #calls: 1', 'TOTAL : CPU, 4.120s; elapsed, 2.573s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 5.100s; elapsed, 2.572s, #calls: 1', 'TOTAL : CPU, 5.100s; elapsed, 2.572s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.580s; elapsed, 2.565s, #calls: 1', 'TOTAL : CPU, 4.580s; elapsed, 2.565s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.630s; elapsed, 3.815s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.055s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.760s; elapsed, 2.578s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.920s; elapsed, 2.577s, #calls: 1', 'TOTAL : CPU, 15.390s; elapsed, 9.025s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 10.400s; elapsed, 2.622s', 'individual call time for EIGEN_LDLT: CPU, 3.230s; elapsed, 2.703s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.045s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,139', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.214863', '# - matching time = 0.03704', '# - symmetrization time = 0.019645', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0441861', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.2791 MB', '# - factor time = 2.73808', '# - factor nonzeros = 6,034,893', '# - factor memory = 48.2791 MB', '# - total flops = 7.11169e+09, min = 8.93654e+08, max = 2.51306e+09', '# - flop rate = 2.59732 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.71747e+09', '# --------------------------------------------', '# total = 7.71747e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.71718e-11\trel.res = 1.16306e-15\tbw.error = 2.55499e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.043875', '# - total flops = 1.9286e+07, min = 3.82026e+06, max = 6.22438e+06', '# - flop rate = 0.439566 GFlop/s', '# - bytes moved = 94.6611 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.15752 GByte/s', '# - solve arithmetic intensity = 0.203737 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 7.730s; elapsed, 3.122s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.620s; elapsed, 3.113s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.350s; elapsed, 3.120s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 12.360s; elapsed, 3.124s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.620s; elapsed, 3.113s, #calls: 1', 'TOTAL : CPU, 9.620s; elapsed, 3.113s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.350s; elapsed, 3.120s, #calls: 1', 'TOTAL : CPU, 10.350s; elapsed, 3.120s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 7.730s; elapsed, 3.122s, #calls: 1', 'TOTAL : CPU, 7.730s; elapsed, 3.122s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 10.400s; elapsed, 2.622s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.045s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.230s; elapsed, 2.703s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 12.360s; elapsed, 3.124s, #calls: 1', 'TOTAL : CPU, 26.100s; elapsed, 8.495s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 17.900s; elapsed, 2.246s', 'individual call time for EIGEN_LDLT: CPU, 3.870s; elapsed, 2.644s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.040s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,139', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.205027', '# - matching time = 0.0371468', '# - symmetrization time = 0.026475', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,139', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0488162', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.2791 MB', '# - factor time = 2.43207', '# - factor nonzeros = 6,034,893', '# - factor memory = 48.2791 MB', '# - total flops = 7.11169e+09, min = 8.93654e+08, max = 2.51306e+09', '# - flop rate = 2.92413 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.71747e+09', '# --------------------------------------------', '# total = 7.71747e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.71495e-11\trel.res = 1.1621e-15\tbw.error = 2.55499e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.045558', '# - total flops = 1.9286e+07, min = 3.82026e+06, max = 6.22438e+06', '# - flop rate = 0.423328 GFlop/s', '# - bytes moved = 94.6754 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.07813 GByte/s', '# - solve arithmetic intensity = 0.203706 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 14.030s; elapsed, 2.823s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.910s; elapsed, 2.815s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 20.810s; elapsed, 2.820s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.280s; elapsed, 2.826s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 16.910s; elapsed, 2.815s, #calls: 1', 'TOTAL : CPU, 16.910s; elapsed, 2.815s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 20.810s; elapsed, 2.820s, #calls: 1', 'TOTAL : CPU, 20.810s; elapsed, 2.820s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 14.030s; elapsed, 2.823s, #calls: 1', 'TOTAL : CPU, 14.030s; elapsed, 2.823s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 17.900s; elapsed, 2.246s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.040s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.870s; elapsed, 2.644s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 22.280s; elapsed, 2.826s, #calls: 1', 'TOTAL : CPU, 44.200s; elapsed, 7.756s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 2913.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.020s; elapsed, 6.034s', 'individual call time for EIGEN_LDLT: CPU, 2.530s; elapsed, 2.518s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.057s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,141', '# - number of levels = 130', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.194121', '# - matching time = 0.0320652', '# - symmetrization time = 0.022748', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0348408', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.8475 MB', '# - factor time = 1.91464', '# - factor nonzeros = 6,230,935', '# - factor memory = 49.8475 MB', '# - total flops = 7.56445e+09, min = 227903, max = 2.51361e+09', '# - flop rate = 3.95085 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.14808e+09', '# --------------------------------------------', '# total = 8.14808e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.58238e-11\trel.res = 1.5334e-15\tbw.error = 2.79918e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0406072', '# - total flops = 2.27636e+07, min = 23029, max = 6.19462e+06', '# - flop rate = 0.560579 GFlop/s', '# - bytes moved = 95.0808 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.34148 GByte/s', '# - solve arithmetic intensity = 0.239413 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.250s; elapsed, 2.257s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.250s; elapsed, 2.258s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.230s; elapsed, 2.256s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.230s; elapsed, 2.256s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.260s; elapsed, 2.256s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.240s; elapsed, 2.257s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.220s; elapsed, 2.248s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.270s; elapsed, 2.262s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.250s; elapsed, 2.258s, #calls: 1', 'TOTAL : CPU, 2.250s; elapsed, 2.258s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.230s; elapsed, 2.256s, #calls: 1', 'TOTAL : CPU, 2.230s; elapsed, 2.256s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.230s; elapsed, 2.256s, #calls: 1', 'TOTAL : CPU, 2.230s; elapsed, 2.256s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.240s; elapsed, 2.257s, #calls: 1', 'TOTAL : CPU, 2.240s; elapsed, 2.257s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.260s; elapsed, 2.256s, #calls: 1', 'TOTAL : CPU, 2.260s; elapsed, 2.256s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.250s; elapsed, 2.257s, #calls: 1', 'TOTAL : CPU, 2.250s; elapsed, 2.257s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.220s; elapsed, 2.248s, #calls: 1', 'TOTAL : CPU, 2.220s; elapsed, 2.248s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.020s; elapsed, 6.034s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.057s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.530s; elapsed, 2.518s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.270s; elapsed, 2.262s, #calls: 1', 'TOTAL : CPU, 10.870s; elapsed, 10.870s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 2913.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 8.140s; elapsed, 4.081s', 'individual call time for EIGEN_LDLT: CPU, 2.840s; elapsed, 2.663s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.060s', 'CPP -2', '# Initializing STRUMPACKCPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,141', '# - number of levels = 130', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.211214', '# - matching time = 0.03666', '# - symmetrization time = 0.0227749', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0421131', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.8475 MB', '# - factor time = 2.24081', '# - factor nonzeros = 6,230,935', '# - factor memory = 49.8475 MB', '# - total flops = 7.56446e+09, min = 227903, max = 2.51361e+09', '# - flop rate = 3.37577 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.14808e+09', '# --------------------------------------------', '# total = 8.14808e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.57235e-11\trel.res = 1.52911e-15\tbw.error = 2.80249e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.041775', '# - total flops = 2.27636e+07, min = 23029, max = 6.19462e+06', '# - flop rate = 0.544909 GFlop/s', '# - bytes moved = 95.089 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.27622 GByte/s', '# - solve arithmetic intensity = 0.239392 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 3.750s; elapsed, 2.620s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 4.180s; elapsed, 2.622s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.290s; elapsed, 2.620s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.240s; elapsed, 2.621s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 5.100s; elapsed, 2.622s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 3.500s; elapsed, 2.620s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 4.490s; elapsed, 2.610s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 5.140s; elapsed, 2.625s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 3.750s; elapsed, 2.620s, #calls: 1', 'TOTAL : CPU, 3.750s; elapsed, 2.620s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.240s; elapsed, 2.621s, #calls: 1', 'TOTAL : CPU, 3.240s; elapsed, 2.621s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 3.500s; elapsed, 2.620s, #calls: 1', 'TOTAL : CPU, 3.500s; elapsed, 2.620s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 5.100s; elapsed, 2.622s, #calls: 1', 'TOTAL : CPU, 5.100s; elapsed, 2.622s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 4.180s; elapsed, 2.622s, #calls: 1', 'TOTAL : CPU, 4.180s; elapsed, 2.622s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.290s; elapsed, 2.620s, #calls: 1', 'TOTAL : CPU, 3.290s; elapsed, 2.620s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 4.490s; elapsed, 2.610s, #calls: 1', 'TOTAL : CPU, 4.490s; elapsed, 2.610s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 8.140s; elapsed, 4.081s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.060s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.840s; elapsed, 2.663s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 5.140s; elapsed, 2.625s, #calls: 1', 'TOTAL : CPU, 16.200s; elapsed, 9.429s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 2913.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 9.250s; elapsed, 2.335s', 'individual call time for EIGEN_LDLT: CPU, 5.280s; elapsed, 4.752s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.055s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 8 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,141', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.215547', '# - matching time = 0.0371702', '# - symmetrization time = 0.02367', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,141', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.044831', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 49.8475 MB', '# - factor time = 2.0909', '# - factor nonzeros = 6,230,935', '# - factor memory = 49.8475 MB', '# - total flops = 7.56446e+09, min = 228737, max = 2.51361e+09', '# - flop rate = 3.6178 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 8.14808e+09', '# --------------------------------------------', '# total = 8.14808e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.5714e-11\trel.res = 1.5287e-15\tbw.error = 2.80816e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0498211', '# - total flops = 2.27636e+07, min = 23029, max = 6.19462e+06', '# - flop rate = 0.456906 GFlop/s', '# - bytes moved = 95.0992 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.90881 GByte/s', '# - solve arithmetic intensity = 0.239366 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 6.960s; elapsed, 2.489s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 5.660s; elapsed, 2.490s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.560s; elapsed, 2.489s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.440s; elapsed, 2.491s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.570s; elapsed, 2.490s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 4.810s; elapsed, 2.488s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 8.260s; elapsed, 2.479s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.740s; elapsed, 2.494s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.440s; elapsed, 2.491s, #calls: 1', 'TOTAL : CPU, 4.440s; elapsed, 2.491s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.570s; elapsed, 2.490s, #calls: 1', 'TOTAL : CPU, 9.570s; elapsed, 2.490s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 5.660s; elapsed, 2.490s, #calls: 1', 'TOTAL : CPU, 5.660s; elapsed, 2.490s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 4.810s; elapsed, 2.488s, #calls: 1', 'TOTAL : CPU, 4.810s; elapsed, 2.488s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.560s; elapsed, 2.489s, #calls: 1', 'TOTAL : CPU, 4.560s; elapsed, 2.489s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 8.260s; elapsed, 2.479s, #calls: 1', 'TOTAL : CPU, 8.260s; elapsed, 2.479s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 6.960s; elapsed, 2.489s, #calls: 1', 'TOTAL : CPU, 6.960s; elapsed, 2.489s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 9.250s; elapsed, 2.335s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.055s, #calls: 1', 'time for EIGEN_LDLT: CPU, 5.280s; elapsed, 4.752s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.740s; elapsed, 2.494s, #calls: 1', 'TOTAL : CPU, 24.390s; elapsed, 9.635s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 1456.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.330s; elapsed, 6.328s', 'individual call time for EIGEN_LDLT: CPU, 2.640s; elapsed, 2.647s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.061s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,135', '# - number of levels = 129', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.206976', '# - matching time = 0.0364771', '# - symmetrization time = 0.0163221', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,135', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0358081', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.9364 MB', '# - factor time = 1.98697', '# - factor nonzeros = 6,117,051', '# - factor memory = 48.9364 MB', '# - total flops = 7.3195e+09, min = 381751, max = 2.51086e+09', '# - flop rate = 3.68375 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.85078e+09', '# --------------------------------------------', '# total = 7.85078e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.63666e-11\trel.res = 1.12859e-15\tbw.error = 2.13979e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0364561', '# - total flops = 3.59797e+07, min = 18684, max = 6.128e+06', '# - flop rate = 0.986932 GFlop/s', '# - bytes moved = 90.6956 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.4878 GByte/s', '# - solve arithmetic intensity = 0.396708 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 2.240s; elapsed, 2.331s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 2.290s; elapsed, 2.332s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.300s; elapsed, 2.333s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 2.270s; elapsed, 2.333s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.250s; elapsed, 2.331s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.230s; elapsed, 2.331s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.310s; elapsed, 2.330s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 2.320s; elapsed, 2.331s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.230s; elapsed, 2.332s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.280s; elapsed, 2.331s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.250s; elapsed, 2.331s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 2.290s; elapsed, 2.331s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 2.310s; elapsed, 2.328s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 2.300s; elapsed, 2.327s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 2.280s; elapsed, 2.332s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.340s; elapsed, 2.337s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 2.300s; elapsed, 2.327s, #calls: 1', 'TOTAL : CPU, 2.300s; elapsed, 2.327s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 2.270s; elapsed, 2.333s, #calls: 1', 'TOTAL : CPU, 2.270s; elapsed, 2.333s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.300s; elapsed, 2.333s, #calls: 1', 'TOTAL : CPU, 2.300s; elapsed, 2.333s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 2.290s; elapsed, 2.331s, #calls: 1', 'TOTAL : CPU, 2.290s; elapsed, 2.331s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.250s; elapsed, 2.331s, #calls: 1', 'TOTAL : CPU, 2.250s; elapsed, 2.331s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.230s; elapsed, 2.332s, #calls: 1', 'TOTAL : CPU, 2.230s; elapsed, 2.332s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 2.310s; elapsed, 2.328s, #calls: 1', 'TOTAL : CPU, 2.310s; elapsed, 2.328s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 2.290s; elapsed, 2.332s, #calls: 1', 'TOTAL : CPU, 2.290s; elapsed, 2.332s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.310s; elapsed, 2.330s, #calls: 1', 'TOTAL : CPU, 2.310s; elapsed, 2.330s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 2.280s; elapsed, 2.331s, #calls: 1', 'TOTAL : CPU, 2.280s; elapsed, 2.331s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.250s; elapsed, 2.331s, #calls: 1', 'TOTAL : CPU, 2.250s; elapsed, 2.331s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 2.240s; elapsed, 2.331s, #calls: 1', 'TOTAL : CPU, 2.240s; elapsed, 2.331s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 2.320s; elapsed, 2.331s, #calls: 1', 'TOTAL : CPU, 2.320s; elapsed, 2.331s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.230s; elapsed, 2.331s, #calls: 1', 'TOTAL : CPU, 2.230s; elapsed, 2.331s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 2.280s; elapsed, 2.332s, #calls: 1', 'TOTAL : CPU, 2.280s; elapsed, 2.332s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.330s; elapsed, 6.328s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.061s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.640s; elapsed, 2.647s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.340s; elapsed, 2.337s, #calls: 1', 'TOTAL : CPU, 11.370s; elapsed, 11.372s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '23309 1456.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 8.540s; elapsed, 4.296s', 'individual call time for EIGEN_LDLT: CPU, 3.370s; elapsed, 3.202s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.063s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 23,309', '# - number of nonzeros = 255,323', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,135', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.222168', '# - matching time = 0.039571', '# - symmetrization time = 0.016113', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,135', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0399868', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 48.9364 MB', '# - factor time = 2.32251', '# - factor nonzeros = 6,117,051', '# - factor memory = 48.9364 MB', '# - total flops = 7.3195e+09, min = 381751, max = 2.51086e+09', '# - flop rate = 3.15155 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 7.85078e+09', '# --------------------------------------------', '# total = 7.85078e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 23362.3\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.6397e-11\trel.res = 1.1299e-15\tbw.error = 2.13979e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0388858', '# - total flops = 3.59797e+07, min = 18684, max = 6.128e+06', '# - flop rate = 0.925265 GFlop/s', '# - bytes moved = 90.7028 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.33254 GByte/s', '# - solve arithmetic intensity = 0.396677 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.350s; elapsed, 2.703s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 4.330s; elapsed, 2.703sindividual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 4.320s; elapsed, 2.703s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.370s; elapsed, 2.702s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 3.640s; elapsed, 2.702s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 3.570s; elapsed, 2.702s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 4.380s; elapsed, 2.703s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.420s; elapsed, 2.703s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 3.820s; elapsed, 2.702s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 3.680s; elapsed, 2.702s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 4.650s; elapsed, 2.699s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 3.860s; elapsed, 2.696s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 3.330s; elapsed, 2.702s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 5.080s; elapsed, 2.702s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 3.590s; elapsed, 2.703s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 5.320s; elapsed, 2.709s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 3.370s; elapsed, 2.702s, #calls: 1', 'TOTAL : CPU, 3.370s; elapsed, 2.702s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 4.330s; elapsed, 2.703s, #calls: 1', 'TOTAL : CPU, 4.330s; elapsed, 2.703s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 4.380s; elapsed, 2.703s, #calls: 1', 'TOTAL : CPU, 4.380s; elapsed, 2.703s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 4.320s; elapsed, 2.703s, #calls: 1', 'TOTAL : CPU, 4.320s; elapsed, 2.703s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 4.650s; elapsed, 2.699s, #calls: 1', 'TOTAL : CPU, 4.650s; elapsed, 2.699s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.420s; elapsed, 2.703s, #calls: 1', 'TOTAL : CPU, 3.420s; elapsed, 2.703s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 3.570s; elapsed, 2.702s, #calls: 1', 'TOTAL : CPU, 3.570s; elapsed, 2.702s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 3.820s; elapsed, 2.702s, #calls: 1', 'TOTAL : CPU, 3.820s; elapsed, 2.702s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 3.860s; elapsed, 2.696s, #calls: 1', 'TOTAL : CPU, 3.860s; elapsed, 2.696s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 3.330s; elapsed, 2.702s, #calls: 1', 'TOTAL : CPU, 3.330s; elapsed, 2.702s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 3.350s; elapsed, 2.703s, #calls: 1', 'TOTAL : CPU, 3.350s; elapsed, 2.703s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 5.080s; elapsed, 2.702s, #calls: 1', 'TOTAL : CPU, 5.080s; elapsed, 2.702s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 3.640s; elapsed, 2.702s, #calls: 1', 'TOTAL : CPU, 3.640s; elapsed, 2.702s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 3.590s; elapsed, 2.703s, #calls: 1', 'TOTAL : CPU, 3.590s; elapsed, 2.703s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 3.680s; elapsed, 2.702s, #calls: 1', 'TOTAL : CPU, 3.680s; elapsed, 2.702s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 8.540s; elapsed, 4.296s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.063s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.370s; elapsed, 3.202s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 5.320s; elapsed, 2.709s, #calls: 1', 'TOTAL : CPU, 17.330s; elapsed, 10.270s']
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_5k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_5k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 19.040s; elapsed, 19.077s', 'individual call time for EIGEN_LDLT: CPU, 18.520s; elapsed, 18.534s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.160s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 19.040s; elapsed, 19.077s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.160s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.520s; elapsed, 18.534s, #calls: 1', 'TOTAL : CPU, 37.720s; elapsed, 37.771s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 22.900s; elapsed, 11.477s', 'individual call time for EIGEN_LDLT: CPU, 19.060s; elapsed, 18.896s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.142s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 22.900s; elapsed, 11.477s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.142s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.060s; elapsed, 18.896s, #calls: 1', 'TOTAL : CPU, 42.200s; elapsed, 30.515s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 33.760s; elapsed, 8.502s', 'individual call time for EIGEN_LDLT: CPU, 19.080s; elapsed, 18.557s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.099s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 33.760s; elapsed, 8.502s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.099s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.080s; elapsed, 18.557s, #calls: 1', 'TOTAL : CPU, 53.100s; elapsed, 27.158s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 67.410s; elapsed, 8.462s', 'individual call time for EIGEN_LDLT: CPU, 20.160s; elapsed, 18.906s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.088s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 67.410s; elapsed, 8.462s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.380s; elapsed, 0.088s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.160s; elapsed, 18.906s, #calls: 1', 'TOTAL : CPU, 87.950s; elapsed, 27.456s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 146.440s; elapsed, 9.201s', 'individual call time for EIGEN_LDLT: CPU, 21.270s; elapsed, 18.605s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.700s; elapsed, 0.100s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 146.440s; elapsed, 9.201s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.700s; elapsed, 0.100s, #calls: 1', 'time for EIGEN_LDLT: CPU, 21.270s; elapsed, 18.605s, #calls: 1', 'TOTAL : CPU, 168.410s; elapsed, 27.906s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 18.910s; elapsed, 18.908s', 'individual call time for EIGEN_LDLT: CPU, 18.530s; elapsed, 18.548s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.166s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,179', '# - number of levels = 85', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.187565', '# - matching time = 0.0523288', '# - symmetrization time = 0.0320089', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,179', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.073024', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 166.554 MB', '# - factor time = 13.4334', '# - factor nonzeros = 20,819,237', '# - factor memory = 166.554 MB', '# - total flops = 5.08732e+10, min = 2.52921e+10, max = 2.55811e+10', '# - flop rate = 3.78706 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.24078e+10', '# --------------------------------------------', '# total = 5.24078e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.98285e-11\trel.res = 2.03761e-15\tbw.error = 5.23477e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0781949', '# - total flops = 4.59385e+07, min = 1.93991e+07, max = 2.65394e+07', '# - flop rate = 0.587488 GFlop/s', '# - bytes moved = 213.256 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.72724 GByte/s', '# - solve arithmetic intensity = 0.215415 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 13.860s; elapsed, 13.878s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 13.880s; elapsed, 13.894s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 13.860s; elapsed, 13.878s, #calls: 1', 'TOTAL : CPU, 13.860s; elapsed, 13.878s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 18.910s; elapsed, 18.908s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.166s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.530s; elapsed, 18.548s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 13.880s; elapsed, 13.894s, #calls: 1', 'TOTAL : CPU, 51.480s; elapsed, 51.517s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 23.450s; elapsed, 11.745s', 'individual call time for EIGEN_LDLT: CPU, 19.750s; elapsed, 19.574s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.171s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,179', '# - number of levels = 85', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.18818', '# - matching time = 0.0529528', '# - symmetrization time = 0.0327299', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,179', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.078342', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 166.554 MB', '# - factor time = 8.82858', '# - factor nonzeros = 20,819,237', '# - factor memory = 166.554 MB', '# - total flops = 5.08734e+10, min = 2.52923e+10, max = 2.55811e+10', '# - flop rate = 5.76235 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.24078e+10', '# --------------------------------------------', '# total = 5.24078e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.74997e-11\trel.res = 1.94238e-15\tbw.error = 5.22417e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0688488', '# - total flops = 4.59555e+07, min = 1.94161e+07, max = 2.65394e+07', '# - flop rate = 0.667484 GFlop/s', '# - bytes moved = 213.935 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.10732 GByte/s', '# - solve arithmetic intensity = 0.21481 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 16.360s; elapsed, 9.277s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.480s; elapsed, 9.292s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 16.360s; elapsed, 9.277s, #calls: 1', 'TOTAL : CPU, 16.360s; elapsed, 9.277s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 23.450s; elapsed, 11.745s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.171s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.750s; elapsed, 19.574s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.480s; elapsed, 9.292s, #calls: 1', 'TOTAL : CPU, 61.960s; elapsed, 40.781s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.540s; elapsed, 8.162s', 'individual call time for EIGEN_LDLT: CPU, 19.530s; elapsed, 18.991s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.112s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,179', '# - number of levels = 85', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.186334', '# - matching time = 0.0547621', '# - symmetrization time = 0.032799', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,179', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.080322', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 166.554 MB', '# - factor time = 7.95718', '# - factor nonzeros = 20,819,237', '# - factor memory = 166.554 MB', '# - total flops = 5.08734e+10, min = 2.52923e+10, max = 2.55811e+10', '# - flop rate = 6.39339 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.24078e+10', '# --------------------------------------------', '# total = 5.24078e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.748e-11\trel.res = 1.94158e-15\tbw.error = 5.22417e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.082998', '# - total flops = 4.59555e+07, min = 1.94161e+07, max = 2.65394e+07', '# - flop rate = 0.553694 GFlop/s', '# - bytes moved = 213.953 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.5778 GByte/s', '# - solve arithmetic intensity = 0.214793 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 25.670s; elapsed, 8.422s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 33.540s; elapsed, 8.439s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 25.670s; elapsed, 8.422s, #calls: 1', 'TOTAL : CPU, 25.670s; elapsed, 8.422s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.540s; elapsed, 8.162s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.290s; elapsed, 0.112s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.530s; elapsed, 18.991s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 33.540s; elapsed, 8.439s, #calls: 1', 'TOTAL : CPU, 85.900s; elapsed, 35.704s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 65.340s; elapsed, 8.207s', 'individual call time for EIGEN_LDLT: CPU, 25.560s; elapsed, 24.291s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.460s; elapsed, 0.118s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,179', '# - number of levels = 85', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.20071', '# - matching time = 0.058666', '# - symmetrization time = 0.0335951', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,179', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0907202', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 166.554 MB', '# - factor time = 8.73868', '# - factor nonzeros = 20,819,237', '# - factor memory = 166.554 MB', '# - total flops = 5.08734e+10, min = 2.52923e+10, max = 2.55811e+10', '# - flop rate = 5.82164 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.24078e+10', '# --------------------------------------------', '# total = 5.24078e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.74504e-11\trel.res = 1.94037e-15\tbw.error = 5.22288e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0792758', '# - total flops = 4.59555e+07, min = 1.94161e+07, max = 2.65394e+07', '# - flop rate = 0.579691 GFlop/s', '# - bytes moved = 213.988 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.69928 GByte/s', '# - solve arithmetic intensity = 0.214758 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 53.730s; elapsed, 9.229s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 73.270s; elapsed, 9.244s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 53.730s; elapsed, 9.229s, #calls: 1', 'TOTAL : CPU, 53.730s; elapsed, 9.229s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 65.340s; elapsed, 8.207s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.460s; elapsed, 0.118s, #calls: 1', 'time for EIGEN_LDLT: CPU, 25.560s; elapsed, 24.291s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 73.270s; elapsed, 9.244s, #calls: 1', 'TOTAL : CPU, 164.630s; elapsed, 41.860s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 139.420s; elapsed, 8.749s', 'individual call time for EIGEN_LDLT: CPU, 21.470s; elapsed, 18.756s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.750s; elapsed, 0.097s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,179', '# - number of levels = 85', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.213284', '# - matching time = 0.066201', '# - symmetrization time = 0.040648', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,179', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.113967', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 166.554 MB', '# - factor time = 8.97821', '# - factor nonzeros = 20,819,237', '# - factor memory = 166.554 MB', '# - total flops = 5.08741e+10, min = 2.5293e+10, max = 2.55811e+10', '# - flop rate = 5.66639 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.24078e+10', '# --------------------------------------------', '# total = 5.24078e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.74347e-11\trel.res = 1.93972e-15\tbw.error = 5.22513e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.107326', '# - total flops = 4.59592e+07, min = 1.94198e+07, max = 2.65394e+07', '# - flop rate = 0.428221 GFlop/s', '# - bytes moved = 214.118 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.99502 GByte/s', '# - solve arithmetic intensity = 0.214644 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 115.400s; elapsed, 9.553s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 151.020s; elapsed, 9.567s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 115.400s; elapsed, 9.553s, #calls: 1', 'TOTAL : CPU, 115.400s; elapsed, 9.553s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 139.420s; elapsed, 8.749s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.750s; elapsed, 0.097s, #calls: 1', 'time for EIGEN_LDLT: CPU, 21.470s; elapsed, 18.756s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 151.020s; elapsed, 9.567s, #calls: 1', 'TOTAL : CPU, 312.660s; elapsed, 37.169s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 20.170s; elapsed, 20.177s', 'individual call time for EIGEN_LDLT: CPU, 19.430s; elapsed, 19.442s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.182s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,197', '# - number of levels = 84', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.2054', '# - matching time = 0.0591681', '# - symmetrization time = 0.0349131', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.064656', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 167.054 MB', '# - factor time = 8.61561', '# - factor nonzeros = 20,881,753', '# - factor memory = 167.054 MB', '# - total flops = 5.11312e+10, min = 4.5447e+06, max = 2.57031e+10', '# - flop rate = 5.93471 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.26108e+10', '# --------------------------------------------', '# total = 5.26108e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.81723e-11\trel.res = 1.56096e-15\tbw.error = 4.2043e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.084022', '# - total flops = 5.19342e+07, min = 62299, max = 2.6526e+07', '# - flop rate = 0.618102 GFlop/s', '# - bytes moved = 187.257 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.22867 GByte/s', '# - solve arithmetic intensity = 0.277341 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.060s; elapsed, 9.073s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.080s; elapsed, 9.082s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.070s; elapsed, 9.080s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.080s; elapsed, 9.088s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.070s; elapsed, 9.080s, #calls: 1', 'TOTAL : CPU, 9.070s; elapsed, 9.080s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.080s; elapsed, 9.082s, #calls: 1', 'TOTAL : CPU, 9.080s; elapsed, 9.082s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.060s; elapsed, 9.073s, #calls: 1', 'TOTAL : CPU, 9.060s; elapsed, 9.073s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 20.170s; elapsed, 20.177s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.182s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.430s; elapsed, 19.442s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.080s; elapsed, 9.088s, #calls: 1', 'TOTAL : CPU, 48.860s; elapsed, 48.889s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 24.900s; elapsed, 12.485s', 'individual call time for EIGEN_LDLT: CPU, 20.040s; elapsed, 19.868s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.137s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,197', '# - number of levels = 84', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.226589', '# - matching time = 0.063375', '# - symmetrization time = 0.039387', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0691011', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 167.054 MB', '# - factor time = 8.18956', '# - factor nonzeros = 20,881,753', '# - factor memory = 167.054 MB', '# - total flops = 5.11312e+10, min = 4.5447e+06, max = 2.57031e+10', '# - flop rate = 6.24347 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.26108e+10', '# --------------------------------------------', '# total = 5.26108e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.79151e-11\trel.res = 1.55044e-15\tbw.error = 4.15674e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.075217', '# - total flops = 5.19403e+07, min = 62299, max = 2.6526e+07', '# - flop rate = 0.690539 GFlop/s', '# - bytes moved = 187.497 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.49275 GByte/s', '# - solve arithmetic intensity = 0.277019 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 14.070s; elapsed, 8.677s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 14.890s; elapsed, 8.686s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.000s; elapsed, 8.686s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.090s; elapsed, 8.693s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 14.070s; elapsed, 8.677s, #calls: 1', 'TOTAL : CPU, 14.070s; elapsed, 8.677s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 14.890s; elapsed, 8.686s, #calls: 1', 'TOTAL : CPU, 14.890s; elapsed, 8.686s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.000s; elapsed, 8.686s, #calls: 1', 'TOTAL : CPU, 10.000s; elapsed, 8.686s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 24.900s; elapsed, 12.485s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.137s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.040s; elapsed, 19.868s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.090s; elapsed, 8.693s, #calls: 1', 'TOTAL : CPU, 62.260s; elapsed, 41.183s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.120s; elapsed, 8.142s', 'individual call time for EIGEN_LDLT: CPU, 19.420s; elapsed, 18.886s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.101s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,197', '# - number of levels = 84', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.199303', '# - matching time = 0.059103', '# - symmetrization time = 0.040134', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.076941', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 167.054 MB', '# - factor time = 8.28534', '# - factor nonzeros = 20,881,753', '# - factor memory = 167.054 MB', '# - total flops = 5.11312e+10, min = 4.54523e+06, max = 2.57031e+10', '# - flop rate = 6.17129 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.26108e+10', '# --------------------------------------------', '# total = 5.26108e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.7877e-11\trel.res = 1.54889e-15\tbw.error = 4.15674e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0909219', '# - total flops = 5.19403e+07, min = 62299, max = 2.6526e+07', '# - flop rate = 0.571263 GFlop/s', '# - bytes moved = 187.511 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.06234 GByte/s', '# - solve arithmetic intensity = 0.276998 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 27.040s; elapsed, 8.776s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 21.560s; elapsed, 8.768s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 12.470s; elapsed, 8.777s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 34.520s; elapsed, 8.783s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 27.040s; elapsed, 8.776s, #calls: 1', 'TOTAL : CPU, 27.040s; elapsed, 8.776s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 21.560s; elapsed, 8.768s, #calls: 1', 'TOTAL : CPU, 21.560s; elapsed, 8.768s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 12.470s; elapsed, 8.777s, #calls: 1', 'TOTAL : CPU, 12.470s; elapsed, 8.777s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.120s; elapsed, 8.142s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.101s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.420s; elapsed, 18.886s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 34.520s; elapsed, 8.783s, #calls: 1', 'TOTAL : CPU, 86.310s; elapsed, 35.913s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 69.850s; elapsed, 8.772s', 'individual call time for EIGEN_LDLT: CPU, 19.880s; elapsed, 18.611s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.390s; elapsed, 0.099s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,197', '# - number of levels = 84', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.217153', '# - matching time = 0.064775', '# - symmetrization time = 0.0398321', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,197', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0847619', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 167.054 MB', '# - factor time = 8.73642', '# - factor nonzeros = 20,881,753', '# - factor memory = 167.054 MB', '# - total flops = 5.11312e+10, min = 4.54523e+06, max = 2.57031e+10', '# - flop rate = 5.85265 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.26108e+10', '# --------------------------------------------', '# total = 5.26108e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.7875e-11\trel.res = 1.54881e-15\tbw.error = 4.15674e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0904601', '# - total flops = 5.19403e+07, min = 62299, max = 2.6526e+07', '# - flop rate = 0.574179 GFlop/s', '# - bytes moved = 187.54 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.07318 GByte/s', '# - solve arithmetic intensity = 0.276956 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 38.930s; elapsed, 9.251s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 53.080s; elapsed, 9.256s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.930s; elapsed, 9.258s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 72.380s; elapsed, 9.265s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 17.930s; elapsed, 9.258s, #calls: 1', 'TOTAL : CPU, 17.930s; elapsed, 9.258s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 38.930s; elapsed, 9.251s, #calls: 1', 'TOTAL : CPU, 38.930s; elapsed, 9.251s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 53.080s; elapsed, 9.256s, #calls: 1', 'TOTAL : CPU, 53.080s; elapsed, 9.256s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 69.850s; elapsed, 8.772s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.390s; elapsed, 0.099s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.880s; elapsed, 18.611s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 72.380s; elapsed, 9.265s, #calls: 1', 'TOTAL : CPU, 162.500s; elapsed, 36.746s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 2663.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 21.280s; elapsed, 21.293s', 'individual call time for EIGEN_LDLT: CPU, 19.830s; elapsed, 19.842s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.194s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,171', '# - number of levels = 84', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.2255', '# - matching time = 0.0620909', '# - symmetrization time = 0.0320089', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0701921', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 166.866 MB', '# - factor time = 10.5222', '# - factor nonzeros = 20,858,227', '# - factor memory = 166.866 MB', '# - total flops = 5.09765e+10, min = 3.60814e+06, max = 2.56131e+10', '# - flop rate = 4.84464 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.23661e+10', '# --------------------------------------------', '# total = 5.23661e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.69923e-11\trel.res = 1.51271e-15\tbw.error = 4.03287e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.070765', '# - total flops = 6.58429e+07, min = 44454, max = 2.64127e+07', '# - flop rate = 0.930444 GFlop/s', '# - bytes moved = 176.451 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.49348 GByte/s', '# - solve arithmetic intensity = 0.373151 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 10.980s; elapsed, 11.001s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.990s; elapsed, 11.001s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.970s; elapsed, 11.000s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.990s; elapsed, 11.001s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.980s; elapsed, 10.999s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 10.960s; elapsed, 10.992s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.960s; elapsed, 11.002s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.990s; elapsed, 11.007s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.990s; elapsed, 11.001s, #calls: 1', 'TOTAL : CPU, 10.990s; elapsed, 11.001s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 10.980s; elapsed, 11.001s, #calls: 1', 'TOTAL : CPU, 10.980s; elapsed, 11.001s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.990s; elapsed, 11.001s, #calls: 1', 'TOTAL : CPU, 10.990s; elapsed, 11.001s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 10.960s; elapsed, 10.992s, #calls: 1', 'TOTAL : CPU, 10.960s; elapsed, 10.992s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.970s; elapsed, 11.000s, #calls: 1', 'TOTAL : CPU, 10.970s; elapsed, 11.000s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.980s; elapsed, 10.999s, #calls: 1', 'TOTAL : CPU, 10.980s; elapsed, 10.999s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.960s; elapsed, 11.002s, #calls: 1', 'TOTAL : CPU, 10.960s; elapsed, 11.002s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 21.280s; elapsed, 21.293s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.194s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.830s; elapsed, 19.842s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.990s; elapsed, 11.007s, #calls: 1', 'TOTAL : CPU, 52.290s; elapsed, 52.337s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 2663.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 26.290s; elapsed, 13.190s', 'individual call time for EIGEN_LDLT: CPU, 21.250s; elapsed, 21.068s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.154s', 'CPP -2', '# Initializing STRUMPACK', 'CPP -2', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,171', '# - number of levels = 84', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.236726', '# - matching time = 0.066009', '# - symmetrization time = 0.032829', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0779581', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 166.866 MB', '# - factor time = 8.36512', '# - factor nonzeros = 20,858,227', '# - factor memory = 166.866 MB', '# - total flops = 5.09765e+10, min = 3.60814e+06, max = 2.56131e+10', '# - flop rate = 6.09393 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.23661e+10', '# --------------------------------------------', '# total = 5.23661e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.61403e-11\trel.res = 1.47787e-15\tbw.error = 4.03122e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0671351', '# - total flops = 6.58429e+07, min = 44454, max = 2.64127e+07', '# - flop rate = 0.980753 GFlop/s', '# - bytes moved = 176.462 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.62846 GByte/s', '# - solve arithmetic intensity = 0.373128 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.000s; elapsed, 8.871s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 13.020s; elapsed, 8.860s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 12.650s; elapsed, 8.870s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.300s; elapsed, 8.872s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.120s; elapsed, 8.870s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 14.770s; elapsed, 8.872s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.590s; elapsed, 8.869s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.430s; elapsed, 8.877s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 13.020s; elapsed, 8.860s, #calls: 1', 'TOTAL : CPU, 13.020s; elapsed, 8.860s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.300s; elapsed, 8.872s, #calls: 1', 'TOTAL : CPU, 10.300s; elapsed, 8.872s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.000s; elapsed, 8.871s, #calls: 1', 'TOTAL : CPU, 10.000s; elapsed, 8.871s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 12.650s; elapsed, 8.870s, #calls: 1', 'TOTAL : CPU, 12.650s; elapsed, 8.870s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.590s; elapsed, 8.869s, #calls: 1', 'TOTAL : CPU, 10.590s; elapsed, 8.869s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 14.770s; elapsed, 8.872s, #calls: 1', 'TOTAL : CPU, 14.770s; elapsed, 8.872s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.120s; elapsed, 8.870s, #calls: 1', 'TOTAL : CPU, 10.120s; elapsed, 8.870s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 26.290s; elapsed, 13.190s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.154s, #calls: 1', 'time for EIGEN_LDLT: CPU, 21.250s; elapsed, 21.068s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.430s; elapsed, 8.877s, #calls: 1', 'TOTAL : CPU, 65.220s; elapsed, 43.288s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 2663.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 34.860s; elapsed, 8.783s', 'individual call time for EIGEN_LDLT: CPU, 20.060s; elapsed, 19.550s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.113s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,171', '# - number of levels = 84', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.229972', '# - matching time = 0.0658889', '# - symmetrization time = 0.0320542', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,171', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0864508', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 166.866 MB', '# - factor time = 8.67558', '# - factor nonzeros = 20,858,227', '# - factor memory = 166.866 MB', '# - total flops = 5.09765e+10, min = 3.60864e+06, max = 2.56131e+10', '# - flop rate = 5.87586 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.23661e+10', '# --------------------------------------------', '# total = 5.23661e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.61811e-11\trel.res = 1.47953e-15\tbw.error = 4.03122e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.085665', '# - total flops = 6.58429e+07, min = 44454, max = 2.64127e+07', '# - flop rate = 0.768609 GFlop/s', '# - bytes moved = 176.475 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.06007 GByte/s', '# - solve arithmetic intensity = 0.3731 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 20.100s; elapsed, 9.198s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 13.610s; elapsed, 9.200s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 12.710s; elapsed, 9.200s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 14.360s; elapsed, 9.198s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 13.100s; elapsed, 9.199s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 27.420s; elapsed, 9.199s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 21.930s; elapsed, 9.190s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 36.050s; elapsed, 9.205s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 27.420s; elapsed, 9.199s, #calls: 1', 'TOTAL : CPU, 27.420s; elapsed, 9.199s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 20.100s; elapsed, 9.198s, #calls: 1', 'TOTAL : CPU, 20.100s; elapsed, 9.198s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 13.610s; elapsed, 9.200s, #calls: 1', 'TOTAL : CPU, 13.610s; elapsed, 9.200s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 13.100s; elapsed, 9.199s, #calls: 1', 'TOTAL : CPU, 13.100s; elapsed, 9.199s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 12.710s; elapsed, 9.200s, #calls: 1', 'TOTAL : CPU, 12.710s; elapsed, 9.200s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 21.930s; elapsed, 9.190s, #calls: 1', 'TOTAL : CPU, 21.930s; elapsed, 9.190s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 14.360s; elapsed, 9.198s, #calls: 1', 'TOTAL : CPU, 14.360s; elapsed, 9.198s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 34.860s; elapsed, 8.783s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.113s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.060s; elapsed, 19.550s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 36.050s; elapsed, 9.205s, #calls: 1', 'TOTAL : CPU, 91.270s; elapsed, 37.651s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 1331.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 21.240s; elapsed, 21.235s', 'individual call time for EIGEN_LDLT: CPU, 18.930s; elapsed, 18.940s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.200s; elapsed, 0.191s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 16 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,203', '# - number of levels = 86', '# - nd time = 0.200942', '# - matching time = 0.058223', '# - symmetrization time = 0.0168569', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,203', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.06142', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 166.06 MB', '# - factor time = 8.34858', '# - factor nonzeros = 20,757,549', '# - factor memory = 166.06 MB', '# - total flops = 5.05896e+10, min = 2.88153e+06, max = 2.55391e+10', '# - flop rate = 6.05967 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.1791e+10', '# --------------------------------------------', '# total = 5.1791e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.49816e-11\trel.res = 1.43049e-15\tbw.error = 3.39388e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0545862', '# - total flops = 1.0266e+08, min = 34210, max = 2.62764e+07', '# - flop rate = 1.88069 GFlop/s', '# - bytes moved = 163.666 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.99831 GByte/s', '# - solve arithmetic intensity = 0.627252 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 8.720s; elapsed, 8.753s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 8.700s; elapsed, 8.754s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 8.740s; elapsed, 8.753s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 8.690s; elapsed, 8.749s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.710s; elapsed, 8.754s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.720s; elapsed, 8.754s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 8.700s; elapsed, 8.752s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 8.720s; elapsed, 8.750s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.730s; elapsed, 8.753s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 8.690s; elapsed, 8.753s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 8.730s; elapsed, 8.753s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 8.690s; elapsed, 8.753s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 8.740s; elapsed, 8.753s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 8.730s; elapsed, 8.749s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 8.710s; elapsed, 8.751s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.750s; elapsed, 8.759s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 8.690s; elapsed, 8.753s, #calls: 1', 'TOTAL : CPU, 8.690s; elapsed, 8.753s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 8.730s; elapsed, 8.749s, #calls: 1', 'TOTAL : CPU, 8.730s; elapsed, 8.749s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 8.720s; elapsed, 8.753s, #calls: 1', 'TOTAL : CPU, 8.720s; elapsed, 8.753s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 8.740s; elapsed, 8.753s, #calls: 1', 'TOTAL : CPU, 8.740s; elapsed, 8.753s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.710s; elapsed, 8.754s, #calls: 1', 'TOTAL : CPU, 8.710s; elapsed, 8.754s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 8.720s; elapsed, 8.750s, #calls: 1', 'TOTAL : CPU, 8.720s; elapsed, 8.750s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 8.700s; elapsed, 8.752s, #calls: 1', 'TOTAL : CPU, 8.700s; elapsed, 8.752s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 8.690s; elapsed, 8.749s, #calls: 1', 'TOTAL : CPU, 8.690s; elapsed, 8.749s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 8.740s; elapsed, 8.753s, #calls: 1', 'TOTAL : CPU, 8.740s; elapsed, 8.753s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.720s; elapsed, 8.754s, #calls: 1', 'TOTAL : CPU, 8.720s; elapsed, 8.754s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 8.690s; elapsed, 8.753s, #calls: 1', 'TOTAL : CPU, 8.690s; elapsed, 8.753s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 8.700s; elapsed, 8.754s, #calls: 1', 'TOTAL : CPU, 8.700s; elapsed, 8.754s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.730s; elapsed, 8.753s, #calls: 1', 'TOTAL : CPU, 8.730s; elapsed, 8.753s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 8.730s; elapsed, 8.753s, #calls: 1', 'TOTAL : CPU, 8.730s; elapsed, 8.753s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 8.710s; elapsed, 8.751s, #calls: 1', 'TOTAL : CPU, 8.710s; elapsed, 8.751s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 21.240s; elapsed, 21.235s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.200s; elapsed, 0.191s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.930s; elapsed, 18.940s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.750s; elapsed, 8.759s, #calls: 1', 'TOTAL : CPU, 49.120s; elapsed, 49.125s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '21309 1331.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 26.980s; elapsed, 13.531s', 'individual call time for EIGEN_LDLT: CPU, 20.860s; elapsed, 20.676s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.151s', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 21,309', '# - number of nonzeros = 489,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 6,203', '# - number of levels = 86', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.# - nd time = ', '# ******************************************************************', '0.233069', '# - matching time = 0.065376', '# - symmetrization time = 0.0224218', '# symbolic factorization:', '# - nr of dense Frontal matrices = 6,203', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0747702', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 166.06 MB', '# - factor time = 9.29119', '# - factor nonzeros = 20,757,549', '# - factor memory = 166.06 MB', '# - total flops = 5.05896e+10, min = 2.88153e+06, max = 2.55391e+10', '# - flop rate = 5.4449 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.1791e+10', '# --------------------------------------------', '# total = 5.1791e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.27482e-11\trel.res = 1.33915e-15\tbw.error = 3.39267e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.054415', '# - total flops = 1.0266e+08, min = 34210, max = 2.62764e+07', '# - flop rate = 1.88661 GFlop/s', '# - bytes moved = 163.668 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.00778 GByte/s', '# - solve arithmetic intensity = 0.627244 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 12.560s; elapsed, 9.762s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 12.470s; elapsed, 9.758s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.820s; elapsed, 9.761s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 15.790s; elapsed, 9.761s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 11.230s; elapsed, 9.761s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 14.350s; elapsed, 9.757s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 11.090s; elapsed, 9.761s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.880s; elapsed, 9.760s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 11.390s; elapsed, 9.761s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 11.700s; elapsed, 9.757s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.240s; elapsed, 9.761s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.820s; elapsed, 9.761s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 10.750s; elapsed, 9.760s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 11.710s; elapsed, 9.761s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 12.820s; elapsed, 9.756s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.190s; elapsed, 9.767s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 11.230s; elapsed, 9.761s, #calls: 1', 'TOTAL : CPU, 11.230s; elapsed, 9.761s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 12.560s; elapsed, 9.762s, #calls: 1', 'TOTAL : CPU, 12.560s; elapsed, 9.762s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 15.790s; elapsed, 9.761s, #calls: 1', 'TOTAL : CPU, 15.790s; elapsed, 9.761s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 14.350s; elapsed, 9.757s, #calls: 1', 'TOTAL : CPU, 14.350s; elapsed, 9.757s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.820s; elapsed, 9.761s, #calls: 1', 'TOTAL : CPU, 10.820s; elapsed, 9.761s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 11.090s; elapsed, 9.761s, #calls: 1', 'TOTAL : CPU, 11.090s; elapsed, 9.761s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 12.820s; elapsed, 9.756s, #calls: 1', 'TOTAL : CPU, 12.820s; elapsed, 9.756s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 11.700s; elapsed, 9.757s, #calls: 1', 'TOTAL : CPU, 11.700s; elapsed, 9.757s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 11.240s; elapsed, 9.761s, #calls: 1', 'TOTAL : CPU, 11.240s; elapsed, 9.761s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 12.470s; elapsed, 9.758s, #calls: 1', 'TOTAL : CPU, 12.470s; elapsed, 9.758s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.820s; elapsed, 9.761s, #calls: 1', 'TOTAL : CPU, 10.820s; elapsed, 9.761s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.880s; elapsed, 9.760s, #calls: 1', 'TOTAL : CPU, 10.880s; elapsed, 9.760s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 10.750s; elapsed, 9.760s, #calls: 1', 'TOTAL : CPU, 10.750s; elapsed, 9.760s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 11.710s; elapsed, 9.761s, #calls: 1', 'TOTAL : CPU, 11.710s; elapsed, 9.761s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 11.390s; elapsed, 9.761s, #calls: 1', 'TOTAL : CPU, 11.390s; elapsed, 9.761s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 26.980s; elapsed, 13.531s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.151s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.860s; elapsed, 20.676s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 19.190s; elapsed, 9.767s, #calls: 1', 'TOTAL : CPU, 67.270s; elapsed, 44.125s']
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 25309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 23.970s; elapsed, 23.990s', 'individual call time for EIGEN_LDLT: CPU, 19.030s; elapsed, 19.046s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.179s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 23.970s; elapsed, 23.990s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.179s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.030s; elapsed, 19.046s, #calls: 1', 'TOTAL : CPU, 43.190s; elapsed, 43.215s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 25309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 31.780s; elapsed, 15.953s', 'individual call time for EIGEN_LDLT: CPU, 18.900s; elapsed, 18.725s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.156s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 31.780s; elapsed, 15.953s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.156s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.900s; elapsed, 18.725s, #calls: 1', 'TOTAL : CPU, 50.940s; elapsed, 34.835s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 25309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 61.940s; elapsed, 15.538s', 'individual call time for EIGEN_LDLT: CPU, 19.450s; elapsed, 18.927s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.320s; elapsed, 0.120s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 61.940s; elapsed, 15.538s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.320s; elapsed, 0.120s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.450s; elapsed, 18.927s, #calls: 1', 'TOTAL : CPU, 81.710s; elapsed, 34.584s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 25309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 77.480s; elapsed, 9.758s', 'individual call time for EIGEN_LDLT: CPU, 20.410s; elapsed, 19.143s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.460s; elapsed, 0.101s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 77.480s; elapsed, 9.758s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.460s; elapsed, 0.101s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.410s; elapsed, 19.143s, #calls: 1', 'TOTAL : CPU, 98.350s; elapsed, 29.002s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 25309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 146.890s; elapsed, 9.263s', 'individual call time for EIGEN_LDLT: CPU, 21.710s; elapsed, 19.080s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.780s; elapsed, 0.099s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 146.890s; elapsed, 9.263s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.780s; elapsed, 0.099s, #calls: 1', 'time for EIGEN_LDLT: CPU, 21.710s; elapsed, 19.080s, #calls: 1', 'TOTAL : CPU, 169.380s; elapsed, 28.442s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 12654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 23.890s; elapsed, 23.909s', 'individual call time for EIGEN_LDLT: CPU, 18.880s; elapsed, 18.892s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.177s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,083', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.222376', '# - matching time = 0.0531359', '# - symmetrization time = 0.0355611', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,083', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0732548', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 159.473 MB', '# - factor time = 17.688', '# - factor nonzeros = 19,934,141', '# - factor memory = 159.473 MB', '# - total flops = 4.94019e+10, min = 1.95383e+10, max = 2.98636e+10', '# - flop rate = 2.79295 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.19065e+10', '# --------------------------------------------', '# total = 5.19065e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.46581e-11\trel.res = 1.82618e-15\tbw.error = 4.40681e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.101741', '# - total flops = 4.46752e+07, min = 2.18291e+07, max = 2.28462e+07', '# - flop rate = 0.439108 GFlop/s', '# - bytes moved = 239.032 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.34942 GByte/s', '# - solve arithmetic intensity = 0.186901 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.190s; elapsed, 18.210s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.220s; elapsed, 18.225s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.190s; elapsed, 18.210s, #calls: 1', 'TOTAL : CPU, 18.190s; elapsed, 18.210s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 23.890s; elapsed, 23.909s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.177s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.880s; elapsed, 18.892s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.220s; elapsed, 18.225s, #calls: 1', 'TOTAL : CPU, 61.170s; elapsed, 61.203s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 12654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 31.360s; elapsed, 15.726s', 'individual call time for EIGEN_LDLT: CPU, 18.990s; elapsed, 18.814s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.154s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,083', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.23295', '# - matching time = 0.0542719', '# - symmetrization time = 0.0345559', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,083', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0671329', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 159.473 MB', '# - factor time = 8.82552', '# - factor nonzeros = 19,934,141', '# - factor memory = 159.473 MB', '# - total flops = 4.94021e+10, min = 1.95383e+10, max = 2.98639e+10', '# - flop rate = 5.59765 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.19065e+10', '# --------------------------------------------', '# total = 5.19065e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.38325e-11\trel.res = 1.79242e-15\tbw.error = 4.30412e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0806', '# - total flops = 4.46938e+07, min = 2.18291e+07, max = 2.28648e+07', '# - flop rate = 0.554514 GFlop/s', '# - bytes moved = 239.769 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.9748 GByte/s', '# - solve arithmetic intensity = 0.186404 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.550s; elapsed, 9.337s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.470s; elapsed, 9.350s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.550s; elapsed, 9.337s, #calls: 1', 'TOTAL : CPU, 18.550s; elapsed, 9.337s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 31.360s; elapsed, 15.726s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.154s, #calls: 1', 'time for EIGEN_LDLT: CPU, 18.990s; elapsed, 18.814s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.470s; elapsed, 9.350s, #calls: 1', 'TOTAL : CPU, 69.070s; elapsed, 44.044s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 12654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 62.970s; elapsed, 15.807s', 'individual call time for EIGEN_LDLT: CPU, 20.170s; elapsed, 19.669s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.320s; elapsed, 0.118s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,083', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.248474', '# - matching time = 0.05983', '# - symmetrization time = 0.0376351', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,083', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.079432', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 159.473 MB', '# - factor time = 10.5434', '# - factor nonzeros = 19,934,141', '# - factor memory = 159.473 MB', '# - total flops = 4.94022e+10, min = 1.95383e+10, max = 2.98639e+10', '# - flop rate = 4.68558 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.19065e+10', '# --------------------------------------------', '# total = 5.19065e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.38597e-11\trel.res = 1.79353e-15\tbw.error = 4.30623e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0991852', '# - total flops = 4.46938e+07, min = 2.18291e+07, max = 2.28648e+07', '# - flop rate = 0.45061 GFlop/s', '# - bytes moved = 239.788 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.41758 GByte/s', '# - solve arithmetic intensity = 0.186389 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 41.690s; elapsed, 11.110s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 44.080s; elapsed, 11.124s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 41.690s; elapsed, 11.110s, #calls: 1', 'TOTAL : CPU, 41.690s; elapsed, 11.110s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 62.970s; elapsed, 15.807s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.320s; elapsed, 0.118s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.170s; elapsed, 19.669s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 44.080s; elapsed, 11.124s, #calls: 1', 'TOTAL : CPU, 127.540s; elapsed, 46.717s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 12654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 68.730s; elapsed, 8.681s', 'individual call time for EIGEN_LDLT: CPU, 49.810s; elapsed, 48.583s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.480s; elapsed, 0.115s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,083', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.240058', '# - matching time = 0.0661011', '# - symmetrization time = 0.0434439', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,083', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.084269', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 159.473 MB', '# - factor time = 8.73535', '# - factor nonzeros = 19,934,141', '# - factor memory = 159.473 MB', '# - total flops = 4.94022e+10, min = 1.95383e+10, max = 2.98639e+10', '# - flop rate = 5.65543 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.19065e+10', '# --------------------------------------------', '# total = 5.19065e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.385e-11\trel.res = 1.79314e-15\tbw.error = 4.30623e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.109228', '# - total flops = 4.46938e+07, min = 2.18291e+07, max = 2.28648e+07', '# - flop rate = 0.409179 GFlop/s', '# - bytes moved = 239.826 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.19565 GByte/s', '# - solve arithmetic intensity = 0.186359 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 70.200s; elapsed, 9.330s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 73.710s; elapsed, 9.342s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 70.200s; elapsed, 9.330s, #calls: 1', 'TOTAL : CPU, 70.200s; elapsed, 9.330s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 68.730s; elapsed, 8.681s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.480s; elapsed, 0.115s, #calls: 1', 'time for EIGEN_LDLT: CPU, 49.810s; elapsed, 48.583s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 73.710s; elapsed, 9.342s, #calls: 1', 'TOTAL : CPU, 192.730s; elapsed, 66.720s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 12654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 165.280s; elapsed, 10.398s', 'individual call time for EIGEN_LDLT: CPU, 21.670s; elapsed, 19.019s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.780s; elapsed, 0.101s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,083', '# - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.250484', '# - matching time = 0.0724931', '# - symmetrization time = 0.053319', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,083', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0961151', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 159.473 MB', '# - factor time = 9.74868', '# - factor nonzeros = 19,934,141', '# - factor memory = 159.473 MB', '# - total flops = 4.94029e+10, min = 1.95383e+10, max = 2.98646e+10', '# - flop rate = 5.06765 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.19065e+10', '# --------------------------------------------', '# total = 5.19065e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.38061e-11\trel.res = 1.79134e-15\tbw.error = 4.30803e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.102045', '# - total flops = 4.46975e+07, min = 2.18291e+07, max = 2.28684e+07', '# - flop rate = 0.438017 GFlop/s', '# - bytes moved = 239.962 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.35153 GByte/s', '# - solve arithmetic intensity = 0.186269 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 154.450s; elapsed, 10.375s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 163.830s; elapsed, 10.392s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 154.450s; elapsed, 10.375s, #calls: 1', 'TOTAL : CPU, 154.450s; elapsed, 10.375s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 165.280s; elapsed, 10.398s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.780s; elapsed, 0.101s, #calls: 1', 'time for EIGEN_LDLT: CPU, 21.670s; elapsed, 19.019s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 163.830s; elapsed, 10.392s, #calls: 1', 'TOTAL : CPU, 351.560s; elapsed, 39.909s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 6327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 26.580s; elapsed, 26.594s', 'individual call time for EIGEN_LDLT: CPU, 19.860s; elapsed, 19.883s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.203s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,133', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.265369', '# - matching time = 0.063823', '# - symmetrization time = 0.0398281', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,133', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.067894', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 159.055 MB', '# - factor time = 9.29889', '# - factor nonzeros = 19,881,835', '# - factor memory = 159.055 MB', '# - total flops = 4.94679e+10, min = 6.57516e+09, max = 1.95818e+10', '# - flop rate = 5.31976 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.19221e+10', '# --------------------------------------------', '# total = 5.19221e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.08337e-11\trel.res = 1.66979e-15\tbw.error = 5.21064e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0752912', '# - total flops = 6.3129e+07, min = 1.23718e+07, max = 2.18423e+07', '# - flop rate = 0.838464 GFlop/s', '# - bytes moved = 191.22 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.53975 GByte/s', '# - solve arithmetic intensity = 0.330137 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.810s; elapsed, 9.838s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.820s; elapsed, 9.834s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.810s; elapsed, 9.821s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.840s; elapsed, 9.839s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.820s; elapsed, 9.834s, #calls: 1', 'TOTAL : CPU, 9.820s; elapsed, 9.834s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.810s; elapsed, 9.838s, #calls: 1', 'TOTAL : CPU, 9.810s; elapsed, 9.838s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.810s; elapsed, 9.821s, #calls: 1', 'TOTAL : CPU, 9.810s; elapsed, 9.821s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 26.580s; elapsed, 26.594s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.203s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.860s; elapsed, 19.883s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.840s; elapsed, 9.839s, #calls: 1', 'TOTAL : CPU, 56.490s; elapsed, 56.519s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 6327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.770s; elapsed, 16.448s', 'individual call time for EIGEN_LDLT: CPU, 20.820s; elapsed, 20.650s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.173s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 4 MPI processesCPP -2', 'CPP -2', 'CPP -2', '', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,133', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.237405', '# - matching time = 0.0585759', '# - symmetrization time = 0.034457', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,133', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0735641', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 159.055 MB', '# - factor time = 8.81283', '# - factor nonzeros = 19,881,835', '# - factor memory = 159.055 MB', '# - total flops = 4.94679e+10, min = 6.57516e+09, max = 1.95818e+10', '# - flop rate = 5.61317 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.19221e+10', '# --------------------------------------------', '# total = 5.19221e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.07101e-11\trel.res = 1.66474e-15\tbw.error = 5.08115e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0912609', '# - total flops = 6.3129e+07, min = 1.23718e+07, max = 2.18423e+07', '# - flop rate = 0.691741 GFlop/s', '# - bytes moved = 191.229 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.09541 GByte/s', '# - solve arithmetic intensity = 0.330123 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 14.260s; elapsed, 9.341s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.450s; elapsed, 9.336s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 15.060s; elapsed, 9.326s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.570s; elapsed, 9.342s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 15.060s; elapsed, 9.326s, #calls: 1', 'TOTAL : CPU, 15.060s; elapsed, 9.326s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 18.450s; elapsed, 9.336s, #calls: 1', 'TOTAL : CPU, 18.450s; elapsed, 9.336s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 14.260s; elapsed, 9.341s, #calls: 1', 'TOTAL : CPU, 14.260s; elapsed, 9.341s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.770s; elapsed, 16.448s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.280s; elapsed, 0.173s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.820s; elapsed, 20.650s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.570s; elapsed, 9.342s, #calls: 1', 'TOTAL : CPU, 72.440s; elapsed, 46.613s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 6327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 64.430s; elapsed, 16.198s', 'individual call time for EIGEN_LDLT: CPU, 20.320s; elapsed, 19.806s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.320s; elapsed, 0.127s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,133', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.270316', '# - matching time = 0.064435', '# - symmetrization time = 0.0405841', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,133', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.077702', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 159.055 MB', '# - factor time = 8.83615', '# - factor nonzeros = 19,881,835', '# - factor memory = 159.055 MB', '# - total flops = 4.94679e+10, min = 6.57516e+09, max = 1.95818e+10', '# - flop rate = 5.59835 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.19221e+10', '# --------------------------------------------', '# total = 5.19221e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.06632e-11\trel.res = 1.66282e-15\tbw.error = 5.08115e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0880868', '# - total flops = 6.3129e+07, min = 1.23718e+07, max = 2.18423e+07', '# - flop rate = 0.716667 GFlop/s', '# - bytes moved = 191.239 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.17103 GByte/s', '# - solve arithmetic intensity = 0.330104 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 21.200s; elapsed, 9.411s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 26.460s; elapsed, 9.396s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 35.970s; elapsed, 9.407s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 37.220s; elapsed, 9.413s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 21.200s; elapsed, 9.411s, #calls: 1', 'TOTAL : CPU, 21.200s; elapsed, 9.411s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 26.460s; elapsed, 9.396s, #calls: 1', 'TOTAL : CPU, 26.460s; elapsed, 9.396s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 35.970s; elapsed, 9.407s, #calls: 1', 'TOTAL : CPU, 35.970s; elapsed, 9.407s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 64.430s; elapsed, 16.198s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.320s; elapsed, 0.127s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.320s; elapsed, 19.806s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 37.220s; elapsed, 9.413s, #calls: 1', 'TOTAL : CPU, 122.290s; elapsed, 45.543s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 6327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 68.240s; elapsed, 8.650s', 'individual call time for EIGEN_LDLT: CPU, 20.810s; elapsed, 19.594s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.440s; elapsed, 0.103s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,133', '# - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.260974', '# - matching time = 0.0658579', '# - symmetrization time = 0.0447209', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,133', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0869229', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 159.055 MB', '# - factor time = 9.10753', '# - factor nonzeros = 19,881,835', '# - factor memory = 159.055 MB', '# - total flops = 4.94679e+10, min = 6.57516e+09, max = 1.95818e+10', '# - flop rate = 5.43154 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.19221e+10', '# --------------------------------------------', '# total = 5.19221e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 4.06603e-11\trel.res = 1.6627e-15\tbw.error = 5.08115e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.104622', '# - total flops = 6.3129e+07, min = 1.23718e+07, max = 2.18423e+07', '# - flop rate = 0.6034 GFlop/s', '# - bytes moved = 191.261 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.82811 GByte/s', '# - solve arithmetic intensity = 0.330067 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 37.070s; elapsed, 9.710s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 72.520s; elapsed, 9.705s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 50.400s; elapsed, 9.692s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 76.740s; elapsed, 9.712s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 72.520s; elapsed, 9.705s, #calls: 1', 'TOTAL : CPU, 72.520s; elapsed, 9.705s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 37.070s; elapsed, 9.710s, #calls: 1', 'TOTAL : CPU, 37.070s; elapsed, 9.710s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 50.400s; elapsed, 9.692s, #calls: 1', 'TOTAL : CPU, 50.400s; elapsed, 9.692s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 68.240s; elapsed, 8.650s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.440s; elapsed, 0.103s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.810s; elapsed, 19.594s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 76.740s; elapsed, 9.712s, #calls: 1', 'TOTAL : CPU, 166.230s; elapsed, 38.059s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 3163.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 25.600s; elapsed, 25.640s', 'individual call time for EIGEN_LDLT: CPU, 19.830s; elapsed, 19.853s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.178s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,073', '# - number of levels = 130', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.238811', '# - matching time = 0.057126', '# - symmetrization time = 0.0320768', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,073', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0566759', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 161.985 MB', '# - factor time = 9.02491', '# - factor nonzeros = 20,248,165', '# - factor memory = 161.985 MB', '# - total flops = 5.08363e+10, min = 1.44631e+06, max = 1.96465e+10', '# - flop rate = 5.63289 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.31844e+10', '# --------------------------------------------', '# total = 5.31844e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.54542e-11\trel.res = 1.44981e-15\tbw.error = 4.08301e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.071017', '# - total flops = 7.45165e+07, min = 37841, max = 2.17786e+07', '# - flop rate = 1.04928 GFlop/s', '# - bytes moved = 192.484 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.7104 GByte/s', '# - solve arithmetic intensity = 0.38713 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 9.460s; elapsed, 9.506s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 9.500s; elapsed, 9.506s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.480s; elapsed, 9.504s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 9.450s; elapsed, 9.494s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.480s; elapsed, 9.505s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 9.470s; elapsed, 9.503s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.480s; elapsed, 9.503s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.480s; elapsed, 9.510s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 9.480s; elapsed, 9.505s, #calls: 1', 'TOTAL : CPU, 9.480s; elapsed, 9.505s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 9.450s; elapsed, 9.494s, #calls: 1', 'TOTAL : CPU, 9.450s; elapsed, 9.494s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 9.460s; elapsed, 9.506s, #calls: 1', 'TOTAL : CPU, 9.460s; elapsed, 9.506s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.480s; elapsed, 9.504s, #calls: 1', 'TOTAL : CPU, 9.480s; elapsed, 9.504s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 9.470s; elapsed, 9.503s, #calls: 1', 'TOTAL : CPU, 9.470s; elapsed, 9.503s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 9.500s; elapsed, 9.506s, #calls: 1', 'TOTAL : CPU, 9.500s; elapsed, 9.506s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 9.480s; elapsed, 9.503s, #calls: 1', 'TOTAL : CPU, 9.480s; elapsed, 9.503s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 25.600s; elapsed, 25.640s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.178s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.830s; elapsed, 19.853s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.480s; elapsed, 9.510s, #calls: 1', 'TOTAL : CPU, 55.090s; elapsed, 55.181s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 3163.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 32.030s; elapsed, 16.088s', 'individual call time for EIGEN_LDLT: CPU, 19.960s; elapsed, 19.785s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.149s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,073', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.24015', '# - matching time = 0.0598629', '# - symmetrization time = 0.0347121', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,073', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0642371', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 161.985 MB', '# - factor time = 8.59414', '# - factor nonzeros = 20,248,165', '# - factor memory = 161.985 MB', '# - total flops = 5.08364e+10, min = 1.44631e+06, max = 1.96465e+10', '# - flop rate = 5.91524 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.31844e+10', '# --------------------------------------------', '# total = 5.31844e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.53219e-11\trel.res = 1.4444e-15\tbw.error = 4.08301e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0809221', '# - total flops = 7.45165e+07, min = 37841, max = 2.17786e+07', '# - flop rate = 0.920842 GFlop/s', '# - bytes moved = 192.496 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.37878 GByte/s', '# - solve arithmetic intensity = 0.387107 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 12.840s; elapsed, 9.106s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 11.250s; elapsed, 9.106s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.760s; elapsed, 9.102s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.290s; elapsed, 9.103s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.330s; elapsed, 9.105s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 14.550s; elapsed, 9.092s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.190s; elapsed, 9.103s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.940s; elapsed, 9.109s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 12.840s; elapsed, 9.106s, #calls: 1', 'TOTAL : CPU, 12.840s; elapsed, 9.106s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 17.330s; elapsed, 9.105s, #calls: 1', 'TOTAL : CPU, 17.330s; elapsed, 9.105s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.760s; elapsed, 9.102s, #calls: 1', 'TOTAL : CPU, 10.760s; elapsed, 9.102s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 14.550s; elapsed, 9.092s, #calls: 1', 'TOTAL : CPU, 14.550s; elapsed, 9.092s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.290s; elapsed, 9.103s, #calls: 1', 'TOTAL : CPU, 10.290s; elapsed, 9.103s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.190s; elapsed, 9.103s, #calls: 1', 'TOTAL : CPU, 10.190s; elapsed, 9.103s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 11.250s; elapsed, 9.106s, #calls: 1', 'TOTAL : CPU, 11.250s; elapsed, 9.106s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 32.030s; elapsed, 16.088s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.149s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.960s; elapsed, 19.785s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 17.940s; elapsed, 9.109s, #calls: 1', 'TOTAL : CPU, 70.170s; elapsed, 45.131s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 3163.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 40.140s; elapsed, 10.137s', 'individual call time for EIGEN_LDLT: CPU, 20.780s; elapsed, 20.257s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.320s; elapsed, 0.126s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,073', '# - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.267111', '# - matching time = 0.065315', '# - symmetrization time = 0.0528588', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,073', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0712948', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 161.985 MB', '# - factor time = 11.199', '# - factor nonzeros = 20,248,165', '# - factor memory = 161.985 MB', '# - total flops = 5.08364e+10, min = 1.44731e+06, max = 1.96465e+10', '# - flop rate = 4.53936 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.31844e+10', '# --------------------------------------------', '# total = 5.31844e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.52804e-11\trel.res = 1.4427e-15\tbw.error = 4.08455e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.079931', '# - total flops = 7.45165e+07, min = 37841, max = 2.17786e+07', '# - flop rate = 0.93226 GFlop/s', '# - bytes moved = 192.511 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.40847 GByte/s', '# - solve arithmetic intensity = 0.387076 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 18.060s; elapsed, 11.771s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 22.980s; elapsed, 11.770s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 14.970s; elapsed, 11.768s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 38.290s; elapsed, 11.768s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 15.400s; elapsed, 11.769s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 16.810s; elapsed, 11.767s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 29.140s; elapsed, 11.757s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 46.300s; elapsed, 11.774s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 18.060s; elapsed, 11.771s, #calls: 1', 'TOTAL : CPU, 18.060s; elapsed, 11.771s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 16.810s; elapsed, 11.767s, #calls: 1', 'TOTAL : CPU, 16.810s; elapsed, 11.767s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 15.400s; elapsed, 11.769s, #calls: 1', 'TOTAL : CPU, 15.400s; elapsed, 11.769s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 22.980s; elapsed, 11.770s, #calls: 1', 'TOTAL : CPU, 22.980s; elapsed, 11.770s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 29.140s; elapsed, 11.757s, #calls: 1', 'TOTAL : CPU, 29.140s; elapsed, 11.757s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 38.290s; elapsed, 11.768s, #calls: 1', 'TOTAL : CPU, 38.290s; elapsed, 11.768s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 14.970s; elapsed, 11.768s, #calls: 1', 'TOTAL : CPU, 14.970s; elapsed, 11.768s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 40.140s; elapsed, 10.137s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.320s; elapsed, 0.126s, #calls: 1', 'time for EIGEN_LDLT: CPU, 20.780s; elapsed, 20.257s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 46.300s; elapsed, 11.774s, #calls: 1', 'TOTAL : CPU, 107.540s; elapsed, 42.294s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 1581.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 26.580s; elapsed, 26.600s', 'individual call time for EIGEN_LDLT: CPU, 19.710s; elapsed, 19.732s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.208s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 16 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,073', '# - number of levels = 129', '# - nd time = 0.239725', '# - matching time = 0.058579', '# - symmetrization time = 0.0199678', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,073', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.056901', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 161.985 MB', '# - factor time = 8.3989', '# - factor nonzeros = 20,248,165', '# - factor memory = 161.985 MB', '# - total flops = 5.08363e+10, min = 1.44631e+06, max = 1.96413e+10', '# - flop rate = 6.05274 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.29833e+10', '# --------------------------------------------', '# total = 5.29833e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.04348e-11\trel.res = 1.24455e-15\tbw.error = 3.45393e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.046207', '# - total flops = 1.17556e+08, min = 29840, max = 2.16901e+07', '# - flop rate = 2.54411 GFlop/s', '# - bytes moved = 184.706 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.99736 GByte/s', '# - solve arithmetic intensity = 0.636448 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 8.820s; elapsed, 8.840s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 8.790s; elapsed, 8.838s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.770s; elapsed, 8.837s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.810s; elapsed, 8.836s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 8.780s; elapsed, 8.830s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 8.770s; elapsed, 8.836s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 8.790s; elapsed, 8.837s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.780s; elapsed, 8.837s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 8.800s; elapsed, 8.836s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 8.810s; elapsed, 8.837s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 8.750s; elapsed, 8.835s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 8.820s; elapsed, 8.837s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 8.810s; elapsed, 8.835s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 8.830s; elapsed, 8.830s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 8.820s; elapsed, 8.836s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.830s; elapsed, 8.842s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 8.820s; elapsed, 8.840s, #calls: 1', 'TOTAL : CPU, 8.820s; elapsed, 8.840s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 8.790s; elapsed, 8.837s, #calls: 1', 'TOTAL : CPU, 8.790s; elapsed, 8.837s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 8.810s; elapsed, 8.835s, #calls: 1', 'TOTAL : CPU, 8.810s; elapsed, 8.835s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 8.810s; elapsed, 8.836s, #calls: 1', 'TOTAL : CPU, 8.810s; elapsed, 8.836s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 8.830s; elapsed, 8.830s, #calls: 1', 'TOTAL : CPU, 8.830s; elapsed, 8.830s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 8.790s; elapsed, 8.838s, #calls: 1', 'TOTAL : CPU, 8.790s; elapsed, 8.838s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 8.780s; elapsed, 8.830s, #calls: 1', 'TOTAL : CPU, 8.780s; elapsed, 8.830s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 8.780s; elapsed, 8.837s, #calls: 1', 'TOTAL : CPU, 8.780s; elapsed, 8.837s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 8.810s; elapsed, 8.837s, #calls: 1', 'TOTAL : CPU, 8.810s; elapsed, 8.837s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 8.800s; elapsed, 8.836s, #calls: 1', 'TOTAL : CPU, 8.800s; elapsed, 8.836s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 8.820s; elapsed, 8.837s, #calls: 1', 'TOTAL : CPU, 8.820s; elapsed, 8.837s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 8.770s; elapsed, 8.836s, #calls: 1', 'TOTAL : CPU, 8.770s; elapsed, 8.836s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.770s; elapsed, 8.837s, #calls: 1', 'TOTAL : CPU, 8.770s; elapsed, 8.837s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 8.750s; elapsed, 8.835s, #calls: 1', 'TOTAL : CPU, 8.750s; elapsed, 8.835s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 8.820s; elapsed, 8.836s, #calls: 1', 'TOTAL : CPU, 8.820s; elapsed, 8.836s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 26.580s; elapsed, 26.600s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.208s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.710s; elapsed, 19.732s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.830s; elapsed, 8.842s, #calls: 1', 'TOTAL : CPU, 55.330s; elapsed, 55.383s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '25309 1581.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 33.610s; elapsed, 16.873s', 'individual call time for EIGEN_LDLT: CPU, 19.150s; elapsed, 18.994s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.155s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 25,309', '# - number of nonzeros = 493,337', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 8,073', '# - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.265644', '# - matching time = 0.066669', '# - symmetrization time = 0.020716', '# symbolic factorization:', '# - nr of dense Frontal matrices = 8,073', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.075568', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 161.985 MB', '# - factor time = 8.93279', '# - factor nonzeros = 20,248,165', '# - factor memory = 161.985 MB', '# - total flops = 5.08364e+10, min = 1.44631e+06, max = 1.96413e+10', '# - flop rate = 5.69098 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 5.29833e+10', '# --------------------------------------------', '# total = 5.29833e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 24454.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 3.03837e-11\trel.res = 1.24247e-15\tbw.error = 3.45393e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0543129', '# - total flops = 1.17556e+08, min = 29840, max = 2.16901e+07', '# - flop rate = 2.16441 GFlop/s', '# - bytes moved = 184.716 MB, min = 0 MB, max = 0 MB', '# - byte rate = 3.40096 GByte/s', '# - solve arithmetic intensity = 0.636411 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.360s; elapsed, 9.440s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.510s; elapsed, 9.439s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 11.300s; elapsed, 9.442s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 10.910s; elapsed, 9.441s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 13.120s; elapsed, 9.441s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 12.050s; elapsed, 9.441s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.610s; elapsed, 9.439s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.360s; elapsed, 9.440s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 18.030s; elapsed, 9.440s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 11.540s; elapsed, 9.441s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 11.240s; elapsed, 9.442s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 10.840s; elapsed, 9.435s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 14.960s; elapsed, 9.434s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.820s; elapsed, 9.441s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 11.070s; elapsed, 9.441s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.510s; elapsed, 9.447s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 10.510s; elapsed, 9.439s, #calls: 1', 'TOTAL : CPU, 10.510s; elapsed, 9.439s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 10.910s; elapsed, 9.441s, #calls: 1', 'TOTAL : CPU, 10.910s; elapsed, 9.441s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 10.360s; elapsed, 9.440s, #calls: 1', 'TOTAL : CPU, 10.360s; elapsed, 9.440s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 11.300s; elapsed, 9.442s, #calls: 1', 'TOTAL : CPU, 11.300s; elapsed, 9.442s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 18.030s; elapsed, 9.440s, #calls: 1', 'TOTAL : CPU, 18.030s; elapsed, 9.440s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 10.840s; elapsed, 9.435s, #calls: 1', 'TOTAL : CPU, 10.840s; elapsed, 9.435s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 13.120s; elapsed, 9.441s, #calls: 1', 'TOTAL : CPU, 13.120s; elapsed, 9.441s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 14.960s; elapsed, 9.434s, #calls: 1', 'TOTAL : CPU, 14.960s; elapsed, 9.434s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 11.540s; elapsed, 9.441s, #calls: 1', 'TOTAL : CPU, 11.540s; elapsed, 9.441s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 10.610s; elapsed, 9.439s, #calls: 1', 'TOTAL : CPU, 10.610s; elapsed, 9.439s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 12.050s; elapsed, 9.441s, #calls: 1', 'TOTAL : CPU, 12.050s; elapsed, 9.441s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 10.820s; elapsed, 9.441s, #calls: 1', 'TOTAL : CPU, 10.820s; elapsed, 9.441s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 11.070s; elapsed, 9.441s, #calls: 1', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 11.240s; elapsed, 9.442s, #calls: 1', 'TOTAL : CPU, 11.240s; elapsed, 9.442s', 'TOTAL : CPU, 11.070s; elapsed, 9.441s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 10.360s; elapsed, 9.440s, #calls: 1', 'TOTAL : CPU, 10.360s; elapsed, 9.440s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 33.610s; elapsed, 16.873s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.155s, #calls: 1', 'time for EIGEN_LDLT: CPU, 19.150s; elapsed, 18.994s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 18.510s; elapsed, 9.447s, #calls: 1', 'TOTAL : CPU, 71.530s; elapsed, 45.469s']
Data Set Size:=32k
Skipping 32k
import json, cPickle
with open("OMP_SOLVER_TIME_5k_MPI.json", "w") as OMP_SOLVER_TIME_FILE:
OMP_SOLVER_TIME_FILE.write(json.dumps(str_out))
with open("OMP_SOLVER_TIME_5k_MPI.pickle", "w") as OMP_SOLVER_TIME_FILE:
OMP_SOLVER_TIME_FILE.write(cPickle.dumps(str_out))
import cPickle
str_out=cPickle.load(open('OMP_SOLVER_TIME_5k_MPI.pickle', 'rb'))
str_out_sm={}
for k in str_out.keys():
str_out_sm.update({k.replace('levmar.parameter_flags=','').replace('_strum_5k_omp1_params','_'):str_out[k]})
import numpy as np
u_dat = {}
threads_list = [1,2,4,8,16]
mpi_proc_list = [1,2,4,8,16]
uniq_ref = set([k.split('_')[1] for k in str_out_sm.keys()])
df_list=[]
for u in uniq_ref:
same_t = lambda: None #Functions are objects, so legit
same_t.strum_scotch_a = np.empty(shape=(5,5))
same_t.strum_mpi_ptscotch = np.empty(shape=(5,5))
same_t.eig_ldlt = np.empty(shape=(5,5))
same_t.eig_bicgstab = np.empty(shape=(5,5))
same_t.strum_scotch_a[:] = np.nan
same_t.strum_mpi_ptscotch[:] = np.nan
same_t.eig_ldlt[:] = np.nan
same_t.eig_bicgstab[:] = np.nan
for m_idx,m in enumerate(mpi_proc_list):
for t_idx,t in enumerate(threads_list):
if m*t <=32:
t_STRUMPACK_OMP = [ s for s in str_out_sm["mpi%domp%d_%s"%(m,t,u)] if 'STRUMPACK_OMP' in s]
t_STRUMPACK_MPI_DIST_RANK = [ s for s in str_out_sm["mpi%domp%d_%s"%(m,t,u)] if 'STRUMPACK_MPI_DIST_RANK' in s]
t_EIGEN_LDLT = [ s for s in str_out_sm["mpi%domp%d_%s"%(m,t,u)] if 'EIGEN_LDLT' in s]
t_EIGEN_BICGSTAB = [ s for s in str_out_sm["mpi%domp%d_%s"%(m,t,u)] if 'EIGEN_BICGSTAB' in s]
if m != 1:
same_t.strum_mpi_ptscotch[m_idx,t_idx] = t_STRUMPACK_MPI_DIST_RANK[-1].rsplit(',')[-2].strip()[0:-1]
same_t.strum_scotch_a[m_idx,t_idx] = t_STRUMPACK_OMP[-1].rsplit(',')[-2].strip()[0:-1]
same_t.eig_bicgstab[m_idx,t_idx] = t_EIGEN_BICGSTAB[-1].rsplit(',')[-2].strip()[0:-1]
same_t.eig_ldlt[m_idx,t_idx] = t_EIGEN_LDLT[-1].rsplit(',')[-2].strip()[0:-1]
u_dat.update({u:same_t})
str_E_LDLT = "EIG_LDLT_%s"%u
str_E_BICGSTAB = "EIG_BICGSTAB_%s"%u
str_S_SA = "STR_SA_%s"%u
str_S_MPI = "STR_MPI_%s"%u
df_dict = { str_E_LDLT:same_t.eig_ldlt,
str_E_BICGSTAB:same_t.eig_bicgstab,
str_S_SA:same_t.strum_scotch_a,
str_S_MPI:same_t.strum_mpi_ptscotch
}
df_dict
With the data correctly parsed, we can plot the performance of each solver with $t\left(\textrm{MPI}, \textrm{OMP}\right)$. Note that the MPI solver does not operate with a single thread, and so will not have data in this row. Subsequently, the relative performance of the OpenMP STRUMPACK backend to the MPI backend is shown. While it would be best compared for the same upp er limit of threads*processes, it can be sufficient to see scalability across the board to identify MPI-enabled sweet-spots. Additionally, as no MPI benefit exists for the other solvers, the launching of processes and overheads can slow performance overal. Thus, it is best to examine the single process data for the non MPI-enabled data.
STRUMPACK MPI
# define the figure size and grid layout properties
figsize = (15, 12)
cols = 4
gs = gridspec.GridSpec(cols, cols)
gs.update(hspace=0.6)
fig1 = plt.figure(num=1, figsize=figsize)
fig1.suptitle('5k strum_mpi_ptscotch', size=16)
ax = []
for i, u in enumerate(uniq_ref):
row = (i // cols)
col = i % cols
ax.append(fig1.add_subplot(gs[row, col]))
cset = ax[-1].imshow(u_dat[u].strum_mpi_ptscotch)
ax[-1].set_xlabel('OMP_NUM_THREADS')
ax[-1].set_ylabel('MPI PROCS')
plt.colorbar(cset)
ax[-1].set_xticks([l for l in xrange(0,5)])
ax[-1].set_xticklabels([str(2**l) for l in xrange(0,5)])
ax[-1].set_yticks([l for l in xrange(0,5)])
ax[-1].set_yticklabels([str(2**l) for l in xrange(0,5)])
 |
STRUMPACK OpenMP
# define the figure size and grid layout properties
figsize = (15, 12)
cols = 4
gs = gridspec.GridSpec(cols, cols)
gs.update(hspace=0.6)
fig1 = plt.figure(num=1, figsize=figsize)
fig1.suptitle('5k strum_scotch_a', size=16)
ax = []
cbars= []
for i, u in enumerate(uniq_ref):
data = u_dat[u].strum_scotch_a
row = (i // cols)
col = i % cols
ax.append(fig1.add_subplot(gs[row, col]))
cset = ax[-1].imshow(data)
ax[-1].set_title(u)
ax[-1].set_xlabel('OMP_NUM_THREADS')
ax[-1].set_ylabel('MPI PROCS')
plt.colorbar(cset)
ax[-1].set_xticks([l for l in xrange(0,5)])
ax[-1].set_xticklabels([str(2**l) for l in xrange(0,5)])
ax[-1].set_yticks([l for l in xrange(0,5)])
ax[-1].set_yticklabels([str(2**l) for l in xrange(0,5)])
plt.show()
 |
EIGEN LDLT
# define the figure size and grid layout properties
figsize = (15, 12)
cols = 4
gs = gridspec.GridSpec(cols, cols)
gs.update(hspace=0.6)
fig1 = plt.figure(num=1, figsize=figsize)
fig1.suptitle('5k eig_ldlt', size=16)
ax = []
cbars= []
for i, u in enumerate(uniq_ref):
data = u_dat[u].eig_ldlt
row = (i // cols)
col = i % cols
ax.append(fig1.add_subplot(gs[row, col]))
cset = ax[-1].imshow(data)
ax[-1].set_title(u)
ax[-1].set_xlabel('OMP_NUM_THREADS')
ax[-1].set_ylabel('MPI PROCS')
plt.colorbar(cset)
ax[-1].set_xticks([l for l in xrange(0,5)])
ax[-1].set_xticklabels([str(2**l) for l in xrange(0,5)])
ax[-1].set_yticks([l for l in xrange(0,5)])
ax[-1].set_yticklabels([str(2**l) for l in xrange(0,5)])
 |
EIGEN BICGSTAB
# define the figure size and grid layout properties
figsize = (15, 12)
cols = 4
gs = gridspec.GridSpec(cols, cols)
gs.update(hspace=0.6)
fig1 = plt.figure(num=1, figsize=figsize)
fig1.suptitle('5k eig_bicgstab', size=16)
ax = []
cbars= []
for i, u in enumerate(uniq_ref):
data = u_dat[u].eig_bicgstab
row = (i // cols)
col = i % cols
ax.append(fig1.add_subplot(gs[row, col]))
cset = ax[-1].imshow(data)
ax[-1].set_title(u)
ax[-1].set_xlabel('OMP_NUM_THREADS')
ax[-1].set_ylabel('MPI PROCS')
plt.colorbar(cset)
ax[-1].set_xticks([l for l in xrange(0,5)])
ax[-1].set_xticklabels([str(2**l) for l in xrange(0,5)])
ax[-1].set_yticks([l for l in xrange(0,5)])
ax[-1].set_yticklabels([str(2**l) for l in xrange(0,5)])
 |
# define the figure size and grid layout properties
figsize = (15, 12)
cols = 4
gs = gridspec.GridSpec(cols, cols)
gs.update(hspace=0.6)
fig1 = plt.figure(num=1, figsize=figsize)
fig1.suptitle('5k strum_mpi_ptscotch / strum_scotch_a', size=16)
ax = []
cbars= []
for i, u in enumerate(uniq_ref):
data = np.divide(u_dat[u].strum_mpi_ptscotch , u_dat[u].strum_scotch_a)
row = (i // cols)
col = i % cols
ax.append(fig1.add_subplot(gs[row, col]))
cset = ax[-1].imshow(data,cmap='bwr', vmin=0.25, vmax=1.75)
ax[-1].set_title(u)
ax[-1].set_xlabel('OMP_NUM_THREADS')
ax[-1].set_ylabel('MPI PROCS')
plt.colorbar(cset)
ax[-1].set_xticks([l for l in xrange(0,5)])
ax[-1].set_xticklabels([str(2**l) for l in xrange(0,5)])
ax[-1].set_yticks([l for l in xrange(0,5)])
ax[-1].set_yticklabels([str(2**l) for l in xrange(0,5)])
 |
