STRUMPACK vs EIGEN performance
The goal of this notebook is to allow the documentation of STRUMPACK vs EIGEN performance to be maintained in a single accessible location. The environment within which this notebook is run follows the standard cctbx conda build instructions available here. For this instance, we are using the STRUMPACK-enabled build of cctbx located at ExaFEL:cctbx_project(str_merge). STRUMPACK is currently built using the installation script STRUMPACK_installer_shared.sh, and if the installation takes place within the same directory as moddules and build, the cctbx build process can make use of it as a backend. After the STRUMPACK solver finishes, enter the build directory, run libtbx.refresh and make. The STRUMPACK-supported modules should now build and link with the new backend.
The solution of the below systems assume that we have some A and b data already, and we can provide these in CSV file format to the solver (A is in “row col value” format). We can begin by importing all of the required modules. We make use of Numpy’s ability to parse CSV files into numeric format, and SciPy’s sparse storage format to format the matrices into the required CSR sparse storage format for solving.
%matplotlib inline
%config InlineBackend.figure_format = 'retina'
import matplotlib
import matplotlib.pyplot as plt
font = {'family' : 'serif',
#'weight' : 'bold',
'size' : 12}
matplotlib.rc('font', **font)
matplotlib.rcParams['figure.figsize'] = (16,10)
matplotlib.rcParams['figure.dpi']= 150
import matplotlib.gridspec as gridspec
from __future__ import division
from cctbx.array_family import flex
from libtbx.test_utils import approx_equal
from libtbx.development.timers import Profiler
import sys
import numpy as np
import scipy.sparse as sps
%env BOOST_ADAPTBX_FPE_DEFAULT=1
%env BOOST_ADAPTBX_SIGNALS_DEFAULT=1
env: BOOST_ADAPTBX_FPE_DEFAULT=1
env: BOOST_ADAPTBX_SIGNALS_DEFAULT=1
We load the A and b data set for solving from the locations specified below. Ideally, we can loop over and load all different data sets and process each individually. For a working example, we begin by processing a single data set, comparing Eigen and Strumpack for varying levels of shared parallelism.
A_path="/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv"
A_mat = np.loadtxt(A_path,dtype={'names':('rows','cols','vals'),'formats':('i8','i8','f8')})
b_path="/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv"
b_vec = np.loadtxt(b_path)
We are solving system of normal equations, and so we can safely assume that we have a square matrix. This ensures the number of rows and colums will be identical, which can be read from the length of the numpy array b_vec.
n_rows = len(b_vec)
n_cols = n_rows
We next wish to create a SciPy CSR sparse matrix type using the data from A_mat.
A_sp = sps.csr_matrix((A_mat['vals'],(A_mat['rows'],A_mat['cols'])))
With this, we can now create the appropriate flex data types for solving the linear system. While the operations performed upto and including this point involve many copy steps, it can be stated that the time to solve the system is much greater than the time to perform internal copies, and so we can ignore the times required for these steps for any realistic datasets.
For the solvers we require the row index pointers, the column indices, and sparse matrix values. However, it is import to first ensure that our A matrix is symmetric, and not the upper or lower triangle only. While Eigen’s Cholesky deocmposition can use the upper or lower triangle, other algorithms and solvers tend to expect a full matrix. We can create this as follows.
tu=sps.triu(A_sp)
tl=sps.tril(A_sp)
sd=sps.diags(A_sp.diagonal())
if tu.nnz == sd.getnnz() or tl.nnz == sd.getnnz():
A_sp = A_sp + A_sp.transpose() - sd
plt.spy(A_sp, marker='.',markersize=1.0)
plt.show()
 |
The resulting data sets can be read directly and converted to flex types as follows.
A_indptr = flex.int(A_sp.indptr)
A_indices = flex.int(A_sp.indices)
A_values = flex.double(A_sp.data)
b = flex.double(b_vec)
With the data in the required format, we can now create a solver object and solve the system. We also include a built-in profiler object to time this solver step for both the STRUMPACK and EIGEN backends.
Next, we load the OpenMP-enabled STRUMPACK solver module and the Eigen based solver. These are contained within “modules/cctbx_project/scitbx/examples/bevington/strumpack_solver_ext.cpp” and where a Boost.Python wrapper for both an EIGEN and STRUMPACK Ax=b solver object is provided.
A non-notebook script exists to test this functionality at “modules/cctbx_project/scitbx/examples/bevington/strumpack_eigen_solver.py”, which can be called as:
OMP_NUM_THREADS=x libtbx.python strumpack_eigen_solver.py A_mat.csv b_vec.csv
where A_mat.csv and b_vec.csv are the CSV files with the linear system to be solved, and x specifies the number of OpenMP threads to spawn. Additionally, the MPI-enabled STRUMPACK backend is provided by “modules/cctbx_project/scitbx/examples/bevington/strumpack_solver_ext_mpi_dist.cpp”, with the Python script to test the result located at “modules/cctbx_project/scitbx/examples/bevington/strumpack_eigen_solver_mpi_dist.py”. This is called as
OMP_NUM_THREADS=x mpirun -n y libtbx.python strumpack_eigen_solver.py A_mat.csv b_vec.csv
As an example, the previously loaded can be provided to the solvers as follows:
import boost.python
ext_omp = boost.python.import_ext("scitbx_examples_strumpack_solver_ext")
ext_mpi = boost.python.import_ext("scitbx_examples_strumpack_mpi_dist_solver_ext")
es = ext_omp.eigen_solver
ss = ext_omp.strumpack_solver
ssmd = ext_mpi.strumpack_mpi_dist_solver
def run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b):
P = Profiler("STRUMPACK_SCOTCH")
res_strum_sc = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.scotch, ext_omp.auto)
del P
P = Profiler("STRUMPACK_METIS")
res_strum_me = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.metis, ext_omp.auto)
del P
P = Profiler("EIGEN_LDLT")
res_eig_ldlt = es(1, n_rows, n_cols, A_indptr, A_indices, A_values, b)
del P
P = Profiler("EIGEN_BICGSTAB")
res_eig_bicgstab = es(2, n_rows, n_cols, A_indptr, A_indices, A_values, b)
del P
return res_strum_sc, res_strum_me, res_eig_ldlt, res_eig_bicgstab
r_s_sc, r_s_me, r_e_ldlt, r_e_bicgstab = run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b)
for i in xrange(len(r_s_sc.x)):
assert( approx_equal(r_s_sc.x[i], r_s_me.x[i]) )
assert( approx_equal(r_s_sc.x[i], r_e_ldlt.x[i]) )
assert( approx_equal(r_s_sc.x[i], r_e_bicgstab.x[i]) )
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.
try: mod = __import__(name)
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.
try: mod = __import__(name)
individual call time for STRUMPACK_SCOTCH: CPU, 21.410s; elapsed, 0.475s
individual call time for STRUMPACK_METIS: CPU, 146.220s; elapsed, 2.334s
individual call time for EIGEN_LDLT: CPU, 4.400s; elapsed, 0.069s
individual call time for EIGEN_BICGSTAB: CPU, 1.840s; elapsed, 0.030s
While we can use the above method to process the data, we must respecify the OMP_NUM_THREADS variable to observe different levels of scalability. %env OMP_NUM_THREADS is sufficient to test this for the first run, however for an undetermined reason the notebook must be started to allow the value to be changed. There, we can run the above tasks from a !<command> cell, specifying the thread count here instead.
OMP_SOLVER='''
from __future__ import division
from cctbx.array_family import flex
from libtbx.test_utils import approx_equal
from libtbx.development.timers import Profiler
import sys
import numpy as np
import scipy.sparse as sps
import matplotlib
import matplotlib.pyplot as plt
import matplotlib
import matplotlib.pyplot as plt
matplotlib.rcParams['figure.figsize'] = (16,10)
A_path=sys.argv[1]
A_mat = np.loadtxt(A_path,dtype={'names':('rows','cols','vals'),'formats':('i8','i8','f8')})
b_path=sys.argv[2]
b_vec = np.loadtxt(b_path)
n_rows = len(b_vec)
n_cols = n_rows
A_sp = sps.csr_matrix((A_mat['vals'],(A_mat['rows'],A_mat['cols'])))
#Check for triangular matrix. If so, A_sp := A+A^T - diag(A)
tu=sps.triu(A_sp)
tl=sps.tril(A_sp)
sd=sps.diags(A_sp.diagonal())
A_spS = A_sp
if tu.nnz == sd.getnnz() or tl.nnz == sd.getnnz():
A_spS = A_sp + A_sp.transpose() - sd
A_indptr = flex.int(A_sp.indptr)
A_indices = flex.int(A_sp.indices)
A_values = flex.double(A_sp.data)
b = flex.double(b_vec)
#import time
#timing_dict = {"strum":0, "eigen":0}
import boost.python
ext_omp = boost.python.import_ext("scitbx_examples_strumpack_solver_ext")
ext_mpi = boost.python.import_ext("scitbx_examples_strumpack_mpi_dist_solver_ext")
es = ext_omp.eigen_solver
ss = ext_omp.strumpack_solver
def run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS):
P = Profiler("EIGEN_LLT_CHOL")
res_eig_llt_chol = es(0, n_rows, n_cols, A_indptr, A_indices, A_values, b)
del P
P = Profiler("EIGEN_LDLT_CHOL")
res_eig_ldlt_chol = es(1, n_rows, n_cols, A_indptr, A_indices, A_values, b)
del P
A_indptr = flex.int(A_spS.indptr)
A_indices = flex.int(A_spS.indices)
A_values = flex.double(A_spS.data)
P = Profiler("EIGEN_BICGSTAB")
res_eig_bicgstab = es(2, n_rows, n_cols, A_indptr, A_indices, A_values, b)
del P
P = Profiler("STRUMPACK_SCOTCH_AUTO")
res_strum_sc_a = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.scotch, ext_omp.auto)
del P
P = Profiler("STRUMPACK_METIS_AUTO")
res_strum_mt_a = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.metis, ext_omp.auto)
del P
P = Profiler("STRUMPACK_SCOTCH_BICGSTAB")
res_strum_sc_bi = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.scotch, ext_omp.bicgstab)
del P
P = Profiler("STRUMPACK_METIS_BICGSTAB")
res_strum_mt_bi = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.metis, ext_omp.bicgstab)
del P
P = Profiler("STRUMPACK_SCOTCH_PRECBICGSTAB")
res_strum_sc_pr = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.scotch, ext_omp.prec_bicgstab)
del P
P = Profiler("STRUMPACK_METIS_PRECBICGSTAB")
res_strum_mt_pr = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.metis, ext_omp.prec_bicgstab)
del P
for i in xrange(len(res_strum_sc.x)):
assert( approx_equal(res_eig_llt_chol.x[i], res_eig_ldlt_chol.x[i]) )
assert( approx_equal(res_strum_sc_a.x[i], res_eig_bicgstab.x[i]) )
assert( approx_equal(res_strum_sc_a.x[i], res_eig_ldlt_chol.x[i]) )
assert( approx_equal(res_strum_mt_a.x[i], res_strum_sc_a.x[i]) )
assert( approx_equal(res_strum_mt_a.x[i], res_strum_sc_bi.x[i]) )
assert( approx_equal(res_strum_mt_a.x[i], res_strum_mt_bi.x[i]) )
assert( approx_equal(res_strum_mt_a.x[i], res_strum_sc_pr.x[i]) )
assert( approx_equal(res_strum_mt_a.x[i], res_strum_mt_pr.x[i]) )
run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)
'''
OMP_SOLVER = OMP_SOLVER
OMP_SOLVER_FILE = open("OMP_SOLVER.py", "w")
OMP_SOLVER_FILE.write(OMP_SOLVER)
OMP_SOLVER_FILE.close()
DATAPATH="/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/"
A_LIST = !find {DATAPATH} -iname "A*.csv"
B_LIST = [ii.replace('/A_','/b_') for ii in A_LIST]
We now record the indices of the data with a specific number of images. We can use these indices to later submit jobs with a given number of frames, and hence resulting matrix size.
list_idx={}
for imgs in ['1k','5k','10k','32k']:
list_idx.update({imgs:[i for i, j in enumerate(A_LIST) if imgs in j]})
list_idx
{'10k': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14],
'1k': [15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29],
'32k': [30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44],
'5k': [45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59]}
With the file writtent to disk, we now create a loop in mixed Python/Bash to call the solver, then extract the standard output from the profiler to examine the OpenMP scalability of STRUMPACK. The profiler code wraps each solver, and does not take into account the load-times for the data.
str_out={}
import os
threads_list = [16]
for imgs_size in list_idx:
print "Data Set Size:=%s"%imgs_size
#Subselect the smallest data size for now
if imgs_size != "1k":
print "Skipping %s"%imgs_size
continue
for imgs_idx in list_idx[imgs_size]:
A_path = A_LIST[imgs_idx]; b_path = B_LIST[imgs_idx]
dat_name = A_path.split('/')[-1][2:-4]
print "Data Set Name:=%s"%(dat_name)
#Ensure the A and b data are matched correctly
assert(os.path.dirname(A_path) == os.path.dirname(b_path))
print {A_path}, {b_path}
for threads in threads_list:
print "OMP_NUM_THREADS:=%d"%threads
val = !OMP_NUM_THREADS={threads} libtbx.python ./OMP_SOLVER.py {A_path} {b_path}
print val
for s in val:
if "assert" in s:
raise Exception("Solutions not equal! Halting")
key = 'omp' + str(threads) + '_' + dat_name
str_out.update({key:val})
—CLICK FOR OUTPUT—
Data Set Size:=10k
Skipping 10k
Data Set Size:=1k
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.810s; elapsed, 0.815s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.870s; elapsed, 0.863s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.025s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 4.280s; elapsed, 0.271s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 31.080s; elapsed, 1.960s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.930s; elapsed, 0.121s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.880s; elapsed, 0.117s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.920s; elapsed, 0.182s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 33.550s; elapsed, 2.101s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 33.550s; elapsed, 2.101s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.810s; elapsed, 0.815s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.930s; elapsed, 0.121s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.025s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 4.280s; elapsed, 0.271s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.920s; elapsed, 0.182s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.880s; elapsed, 0.117s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.870s; elapsed, 0.863s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 31.080s; elapsed, 1.960s, #calls: 1', 'TOTAL : CPU, 77.570s; elapsed, 6.455s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.220s; elapsed, 0.232s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.238s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.570s; elapsed, 0.049s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 7.060s; elapsed, 0.447s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 28.970s; elapsed, 1.831s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.720s; elapsed, 0.169s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.830s; elapsed, 0.177s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.750s; elapsed, 0.422s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 31.770s; elapsed, 2.001s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 31.770s; elapsed, 2.001s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.220s; elapsed, 0.232s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.720s; elapsed, 0.169s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.570s; elapsed, 0.049s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 7.060s; elapsed, 0.447s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.750s; elapsed, 0.422s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.830s; elapsed, 0.177s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.238s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 28.970s; elapsed, 1.831s, #calls: 1', 'TOTAL : CPU, 81.130s; elapsed, 5.566s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Deff-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.240s; elapsed, 0.237s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.230s; elapsed, 0.232s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.024s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.400s; elapsed, 0.215s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 28.710s; elapsed, 1.812s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.750s; elapsed, 0.109s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.740s; elapsed, 0.109s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.070s; elapsed, 0.192s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.190s; elapsed, 1.893s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.190s; elapsed, 1.893s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.240s; elapsed, 0.237s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.750s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.024s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.400s; elapsed, 0.215s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.070s; elapsed, 0.192s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.740s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.230s; elapsed, 0.232s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 28.710s; elapsed, 1.812s, #calls: 1', 'TOTAL : CPU, 69.570s; elapsed, 4.823s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.070s; elapsed, 0.064s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.050s; elapsed, 0.052s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.025s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.350s; elapsed, 0.212s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 28.510s; elapsed, 1.799s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.670s; elapsed, 0.105s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.760s; elapsed, 0.110s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.310s; elapsed, 0.207s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 26.660s; elapsed, 1.673s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 26.660s; elapsed, 1.673s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.070s; elapsed, 0.064s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.670s; elapsed, 0.105s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.230s; elapsed, 0.025s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.350s; elapsed, 0.212s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.310s; elapsed, 0.207s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.760s; elapsed, 0.110s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.050s; elapsed, 0.052s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 28.510s; elapsed, 1.799s, #calls: 1', 'TOTAL : CPU, 65.610s; elapsed, 4.247s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 1.990s; elapsed, 1.995s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 2.110s; elapsed, 2.120s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.048s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 6.590s; elapsed, 0.417s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 34.870s; elapsed, 2.191s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.440s; elapsed, 0.153s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.620s; elapsed, 0.163s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.920s; elapsed, 0.433s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 38.160s; elapsed, 2.391s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 38.160s; elapsed, 2.391s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 1.990s; elapsed, 1.995s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.440s; elapsed, 0.153s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.048s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 6.590s; elapsed, 0.417s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.920s; elapsed, 0.433s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.620s; elapsed, 0.163s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 2.110s; elapsed, 2.120s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 34.870s; elapsed, 2.191s, #calls: 1', 'TOTAL : CPU, 96.220s; elapsed, 9.909s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 2.040s; elapsed, 2.044s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 2.090s; elapsed, 2.091s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.024s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.470s; elapsed, 0.221s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 35.250s; elapsed, 2.219s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.910s; elapsed, 0.118s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.840s; elapsed, 0.115s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.990s; elapsed, 0.188s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 38.560s; elapsed, 2.412s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 38.560s; elapsed, 2.412s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 2.040s; elapsed, 2.044s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.910s; elapsed, 0.118s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.024s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.470s; elapsed, 0.221s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.990s; elapsed, 0.188s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.840s; elapsed, 0.115s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 2.090s; elapsed, 2.091s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 35.250s; elapsed, 2.219s, #calls: 1', 'TOTAL : CPU, 88.400s; elapsed, 9.434s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.830s; elapsed, 0.819s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.860s; elapsed, 0.858s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.030s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.520s; elapsed, 0.223s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 30.270s; elapsed, 1.907s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.000s; elapsed, 0.125s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.070s; elapsed, 0.130s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.260s; elapsed, 0.204s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.920s; elapsed, 1.944s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.920s; elapsed, 1.944s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.830s; elapsed, 0.819s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.000s; elapsed, 0.125s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.030s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.520s; elapsed, 0.223s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.260s; elapsed, 0.204s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.070s; elapsed, 0.130s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.860s; elapsed, 0.858s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 30.270s; elapsed, 1.907s, #calls: 1', 'TOTAL : CPU, 74.030s; elapsed, 6.238s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.850s; elapsed, 0.843s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.860s; elapsed, 0.864s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.050s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 7.220s; elapsed, 0.459s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 31.850s; elapsed, 2.009s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.370s; elapsed, 0.148s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.580s; elapsed, 0.162s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.480s; elapsed, 0.406s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 34.160s; elapsed, 2.146s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 34.160s; elapsed, 2.146s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.850s; elapsed, 0.843s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.370s; elapsed, 0.148s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.050s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 7.220s; elapsed, 0.459s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.480s; elapsed, 0.406s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.580s; elapsed, 0.162s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.860s; elapsed, 0.864s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 31.850s; elapsed, 2.009s, #calls: 1', 'TOTAL : CPU, 86.900s; elapsed, 7.087s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.220s; elapsed, 0.231s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.236s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.046s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 7.200s; elapsed, 0.455s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 28.890s; elapsed, 1.823s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.610s; elapsed, 0.163s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.930s; elapsed, 0.183s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 7.140s; elapsed, 0.447s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.950s; elapsed, 1.941s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 30.950s; elapsed, 1.941s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.220s; elapsed, 0.231s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.610s; elapsed, 0.163s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.046s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 7.200s; elapsed, 0.455s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 7.140s; elapsed, 0.447s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.930s; elapsed, 0.183s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.236s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 28.890s; elapsed, 1.823s, #calls: 1', 'TOTAL : CPU, 80.710s; elapsed, 5.525s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.240s; elapsed, 0.238s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.232s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.023s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.330s; elapsed, 0.212s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 29.220s; elapsed, 1.835s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.750s; elapsed, 0.109s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.750s; elapsed, 0.109s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.090s; elapsed, 0.194s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 28.990s; elapsed, 1.830s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 28.990s; elapsed, 1.830s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.240s; elapsed, 0.238s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.750s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.210s; elapsed, 0.023s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.330s; elapsed, 0.212s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 3.090s; elapsed, 0.194s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.750s; elapsed, 0.109s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.240s; elapsed, 0.232s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 29.220s; elapsed, 1.835s, #calls: 1', 'TOTAL : CPU, 68.820s; elapsed, 4.782s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 3.880s; elapsed, 3.883s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 4.090s; elapsed, 4.098s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.050s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 6.870s; elapsed, 0.435s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 37.710s; elapsed, 2.365s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.460s; elapsed, 0.154s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.680s; elapsed, 0.167s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 7.180s; elapsed, 0.450s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 40.580s; elapsed, 2.539s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 40.580s; elapsed, 2.539s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 3.880s; elapsed, 3.883s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.460s; elapsed, 0.154s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.550s; elapsed, 0.050s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 6.870s; elapsed, 0.435s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 7.180s; elapsed, 0.450s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.680s; elapsed, 0.167s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 4.090s; elapsed, 4.098s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 37.710s; elapsed, 2.365s, #calls: 1', 'TOTAL : CPU, 106.000s; elapsed, 14.140s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 2.020s; elapsed, 2.023s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 2.110s; elapsed, 2.119s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.048s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 6.330s; elapsed, 0.401s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 38.360s; elapsed, 2.411s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.420s; elapsed, 0.151s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.580s; elapsed, 0.161s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.920s; elapsed, 0.434s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 37.560s; elapsed, 2.354s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 37.560s; elapsed, 2.354s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 2.020s; elapsed, 2.023s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.420s; elapsed, 0.151s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.530s; elapsed, 0.048s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 6.330s; elapsed, 0.401s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.920s; elapsed, 0.434s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.580s; elapsed, 0.161s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 2.110s; elapsed, 2.119s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 38.360s; elapsed, 2.411s, #calls: 1', 'TOTAL : CPU, 98.830s; elapsed, 10.102s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 1.990s; elapsed, 1.994s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 2.120s; elapsed, 2.119s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.024s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 3.440s; elapsed, 0.220s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 34.260s; elapsed, 2.157s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.770s; elapsed, 0.111s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 1.670s; elapsed, 0.104s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.980s; elapsed, 0.187s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 37.390s; elapsed, 2.350s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 37.390s; elapsed, 2.350s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 1.990s; elapsed, 1.994s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 1.770s; elapsed, 0.111s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.024s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 3.440s; elapsed, 0.220s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 2.980s; elapsed, 0.187s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 1.670s; elapsed, 0.104s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 2.120s; elapsed, 2.119s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 34.260s; elapsed, 2.157s, #calls: 1', 'TOTAL : CPU, 85.860s; elapsed, 9.266s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 0.830s; elapsed, 0.835s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 0.870s; elapsed, 0.863s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.540s; elapsed, 0.051s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 7.370s; elapsed, 0.466s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 32.030s; elapsed, 2.025s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.370s; elapsed, 0.148s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.660s; elapsed, 0.166s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.810s; elapsed, 0.426s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 33.250s; elapsed, 2.094s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 33.250s; elapsed, 2.094s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 0.830s; elapsed, 0.835s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.370s; elapsed, 0.148s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.540s; elapsed, 0.051s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 7.370s; elapsed, 0.466s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.810s; elapsed, 0.426s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.660s; elapsed, 0.166s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 0.870s; elapsed, 0.863s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 32.030s; elapsed, 2.025s, #calls: 1', 'TOTAL : CPU, 86.730s; elapsed, 7.075s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU, 3.890s; elapsed, 3.903s', 'individual call time for EIGEN_LDLT_CHOL: CPU, 4.070s; elapsed, 4.073s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.047s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU, 7.190s; elapsed, 0.454s', 'individual call time for STRUMPACK_METIS_AUTO: CPU, 38.310s; elapsed, 2.408s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.510s; elapsed, 0.157s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU, 2.690s; elapsed, 0.168s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.390s; elapsed, 0.401s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 41.380s; elapsed, 2.597s', 'Traceback (most recent call last):', ' File "./OMP_SOLVER.py", line 103, in <module>', ' run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', ' File "./OMP_SOLVER.py", line 93, in run_solver', ' for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for STRUMPACK_METIS_PRECBICGSTAB: CPU, 41.380s; elapsed, 2.597s, #calls: 1', 'time for EIGEN_LLT_CHOL: CPU, 3.890s; elapsed, 3.903s, #calls: 1', 'time for STRUMPACK_SCOTCH_BICGSTAB: CPU, 2.510s; elapsed, 0.157s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.520s; elapsed, 0.047s, #calls: 1', 'time for STRUMPACK_SCOTCH_AUTO: CPU, 7.190s; elapsed, 0.454s, #calls: 1', 'time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU, 6.390s; elapsed, 0.401s, #calls: 1', 'time for STRUMPACK_METIS_BICGSTAB: CPU, 2.690s; elapsed, 0.168s, #calls: 1', 'time for EIGEN_LDLT_CHOL: CPU, 4.070s; elapsed, 4.073s, #calls: 1', 'time for STRUMPACK_METIS_AUTO: CPU, 38.310s; elapsed, 2.408s, #calls: 1', 'TOTAL : CPU, 106.950s; elapsed, 14.208s']
Data Set Size:=5k
Skipping 5k
Data Set Size:=32k
Skipping 32k
In the event of a crash, it is wise to save the resulting data (JSON and Pickle) to disk…
import json, cPickle
with open("OMP_SOLVER_TIME_1k_all.json", "w") as OMP_SOLVER_TIME_FILE:
OMP_SOLVER_TIME_FILE.write(json.dumps(str_out))
with open("OMP_SOLVER_TIME_1k_all.pickle", "w") as OMP_SOLVER_TIME_FILE:
OMP_SOLVER_TIME_FILE.write(cPickle.dumps(str_out))
…which can then be easily reloaded later. Now, we parse the string, extract the timing data, and add it to the python object at the specified thread number index. This allows for easily sorting and plot the data later.
import cPickle
str_out=cPickle.load(open('OMP_SOLVER_TIME_1k_all.pickle', 'rb'))
To simplify indexing later for plots, we create shortened keys. Additionally, we ca also separate the data based upon unique index set and thread numbers.
str_out_sm={}
for k in str_out.keys():
str_out_sm.update({k.replace('levmar.parameter_flags=','').replace('_strum_1k_omp1_params','_'):str_out[k]})
import pandas as pd
u_dat={}
threads_list=[16]
uniq_ref = set([k.split('_')[1] for k in str_out_sm.keys()])
df_list=[]
for u in uniq_ref:
same_t = lambda: None #Functions are objects, so legit
same_t.strum_scotch_a = []
same_t.strum_metis_a = []
same_t.strum_scotch_bi = []
same_t.strum_metis_bi = []
same_t.strum_scotch_pr = []
same_t.strum_metis_pr = []
same_t.eig_llt = []
same_t.eig_ldlt = []
same_t.eig_bicgstab = []
for t in threads_list:
same_t.strum_metis_a.append(str_out_sm["omp%d_%s"%(t,u)][-2].rsplit(',')[-2].strip()[0:-1])
same_t.eig_ldlt.append(str_out_sm["omp%d_%s"%(t,u)][-3].rsplit(',')[-2].strip()[0:-1])
same_t.strum_metis_bi.append(str_out_sm["omp%d_%s"%(t,u)][-4].rsplit(',')[-2].strip()[0:-1])
same_t.strum_scotch_pr.append(str_out_sm["omp%d_%s"%(t,u)][-5].rsplit(',')[-2].strip()[0:-1])
same_t.strum_scotch_a.append(str_out_sm["omp%d_%s"%(t,u)][-6].rsplit(',')[-2].strip()[0:-1])
same_t.eig_bicgstab.append(str_out_sm["omp%d_%s"%(t,u)][-7].rsplit(',')[-2].strip()[0:-1])
same_t.strum_scotch_bi.append(str_out_sm["omp%d_%s"%(t,u)][-8].rsplit(',')[-2].strip()[0:-1])
same_t.eig_llt.append(str_out_sm["omp%d_%s"%(t,u)][-9].rsplit(',')[-2].strip()[0:-1])
same_t.strum_metis_pr.append(str_out_sm["omp%d_%s"%(t,u)][-10].rsplit(',')[-2].strip()[0:-1])
u_dat.update({u:same_t})
str_E_LLT = "EIG_LLT_%s"%u
str_E_LDLT = "EIG_LDLT_%s"%u
str_E_BICGSTAB = "EIG_BICGSTAB_%s"%u
str_S_SCOTCH_A = "STRUM_SCOTCHA_%s"%u
str_S_METIS_A = "STRUM_METISA_%s"%u
str_S_SCOTCH_BI = "STRUM_SCOTCHBI_%s"%u
str_S_METIS_BI = "STRUM_METISBI_%s"%u
str_S_SCOTCH_PR = "STRUM_SCOTCHPR_%s"%u
str_S_METIS_PR = "STRUM_METISPR_%s"%u
df_list.append( pd.DataFrame({str_E_LLT:same_t.eig_llt, str_E_LDLT:same_t.eig_ldlt, str_E_BICGSTAB:same_t.eig_bicgstab, str_S_SCOTCH_A:same_t.strum_scotch_a, str_S_METIS_A:same_t.strum_metis_a, str_S_SCOTCH_BI:same_t.strum_scotch_bi, str_S_METIS_BI:same_t.strum_metis_bi, str_S_SCOTCH_PR:same_t.strum_scotch_pr, str_S_METIS_PR:same_t.strum_metis_pr }, index=threads_list).transpose() )
We now have a list of Pandas Dataframes, with which we can combine to a single entity. We can then proceed to plot the resulting columns, comparing both Eigen and STRUMPACK for each specific set of refined parameters.
perf_s = pd.concat(df_list).transpose()
perf_s.transpose()
—CLICK FOR OUTPUT—
| 16 | |
|---|---|
| EIG_BICGSTAB_Rxy-Deff-Eta- | 0.024 |
| EIG_LDLT_Rxy-Deff-Eta- | 2.119 |
| EIG_LLT_Rxy-Deff-Eta- | 1.994 |
| STRUM_METISA_Rxy-Deff-Eta- | 2.157 |
| STRUM_METISBI_Rxy-Deff-Eta- | 0.104 |
| STRUM_METISPR_Rxy-Deff-Eta- | 2.350 |
| STRUM_SCOTCHA_Rxy-Deff-Eta- | 0.220 |
| STRUM_SCOTCHBI_Rxy-Deff-Eta- | 0.111 |
| STRUM_SCOTCHPR_Rxy-Deff-Eta- | 0.187 |
| EIG_BICGSTAB_Bfactor- | 0.049 |
| EIG_LDLT_Bfactor- | 0.238 |
| EIG_LLT_Bfactor- | 0.232 |
| STRUM_METISA_Bfactor- | 1.831 |
| STRUM_METISBI_Bfactor- | 0.177 |
| STRUM_METISPR_Bfactor- | 2.001 |
| STRUM_SCOTCHA_Bfactor- | 0.447 |
| STRUM_SCOTCHBI_Bfactor- | 0.169 |
| STRUM_SCOTCHPR_Bfactor- | 0.422 |
| EIG_BICGSTAB_Bfactor-Deff-Eta- | 0.051 |
| EIG_LDLT_Bfactor-Deff-Eta- | 0.863 |
| EIG_LLT_Bfactor-Deff-Eta- | 0.835 |
| STRUM_METISA_Bfactor-Deff-Eta- | 2.025 |
| STRUM_METISBI_Bfactor-Deff-Eta- | 0.166 |
| STRUM_METISPR_Bfactor-Deff-Eta- | 2.094 |
| STRUM_SCOTCHA_Bfactor-Deff-Eta- | 0.466 |
| STRUM_SCOTCHBI_Bfactor-Deff-Eta- | 0.148 |
| STRUM_SCOTCHPR_Bfactor-Deff-Eta- | 0.426 |
| EIG_BICGSTAB_Rxy-Bfactor-Deff-Eta- | 0.047 |
| EIG_LDLT_Rxy-Bfactor-Deff-Eta- | 4.073 |
| EIG_LLT_Rxy-Bfactor-Deff-Eta- | 3.903 |
| ... | ... |
| STRUM_SCOTCHA_Deff- | 0.215 |
| STRUM_SCOTCHBI_Deff- | 0.109 |
| STRUM_SCOTCHPR_Deff- | 0.192 |
| EIG_BICGSTAB_Rxy-Bfactor- | 0.048 |
| EIG_LDLT_Rxy-Bfactor- | 2.120 |
| EIG_LLT_Rxy-Bfactor- | 1.995 |
| STRUM_METISA_Rxy-Bfactor- | 2.191 |
| STRUM_METISBI_Rxy-Bfactor- | 0.163 |
| STRUM_METISPR_Rxy-Bfactor- | 2.391 |
| STRUM_SCOTCHA_Rxy-Bfactor- | 0.417 |
| STRUM_SCOTCHBI_Rxy-Bfactor- | 0.153 |
| STRUM_SCOTCHPR_Rxy-Bfactor- | 0.433 |
| EIG_BICGSTAB_Rxy-Bfactor-Eta- | 0.048 |
| EIG_LDLT_Rxy-Bfactor-Eta- | 2.119 |
| EIG_LLT_Rxy-Bfactor-Eta- | 2.023 |
| STRUM_METISA_Rxy-Bfactor-Eta- | 2.411 |
| STRUM_METISBI_Rxy-Bfactor-Eta- | 0.161 |
| STRUM_METISPR_Rxy-Bfactor-Eta- | 2.354 |
| STRUM_SCOTCHA_Rxy-Bfactor-Eta- | 0.401 |
| STRUM_SCOTCHBI_Rxy-Bfactor-Eta- | 0.151 |
| STRUM_SCOTCHPR_Rxy-Bfactor-Eta- | 0.434 |
| EIG_BICGSTAB_Rxy-Bfactor-Deff- | 0.050 |
| EIG_LDLT_Rxy-Bfactor-Deff- | 4.098 |
| EIG_LLT_Rxy-Bfactor-Deff- | 3.883 |
| STRUM_METISA_Rxy-Bfactor-Deff- | 2.365 |
| STRUM_METISBI_Rxy-Bfactor-Deff- | 0.167 |
| STRUM_METISPR_Rxy-Bfactor-Deff- | 2.539 |
| STRUM_SCOTCHA_Rxy-Bfactor-Deff- | 0.435 |
| STRUM_SCOTCHBI_Rxy-Bfactor-Deff- | 0.154 |
| STRUM_SCOTCHPR_Rxy-Bfactor-Deff- | 0.450 |
135 rows × 1 columns
With the performance data in an easily accessible format, we can plot each respective data set for all solvers. Each bar corresponds to the parameters being refined, where the colour indicates the respective solver used, and the y-axis is the time to solve the system in seconds.
# define the figure size and grid layout properties
matplotlib.rcParams.update({'font.size': 22})
figsize = (20, 16)
cols = 4
fig3 = plt.figure(num=3, figsize=figsize)
ax3 = plt.axes()
ii = []
uu=[]
keys=['EIG_LLT','EIG_LDLT','STRUM_METISA','STRUM_SCOTCHA','STRUM_METISBI','STRUM_SCOTCHBI','STRUM_METISPR','STRUM_SCOTCHPR','EIG_BICGSTAB']
colours = ['#a6cee3','#1f78b4','#b2df8a','#33a02c','#fb9a99','#e31a1c','#fdbf6f','#ff7f00','#cab2d6']
key_col = {k:v for (k,v) in zip(keys,colours)}
hatch_pattern = [ '/' , '\\' , '|' , '-' , '+' , 'x' ,'o' ,'O' , '.', '*']
for i, u in enumerate(uniq_ref):
p_list=[]
ax3.set_ylabel('time [s]')
keys_df = [k+'_'+u for k in keys]
dfl=perf_s[keys_df]
m = zip(dfl.as_matrix().astype(np.float)[0], dfl.keys().tolist() )
m = sorted(m,key=lambda x: x[0])
for p in reversed(xrange(len(m))):
c_label = "_".join(m[p][1].split('_')[0:2])
p_list.append( plt.bar(i, m[p][0], label=m[p][1], color=key_col[c_label], edgecolor='k') )
ii.append(i)
uu.append(u)
ax3.set_yscale('log',basey=10)
plt.title('1k frames OMP_NUM_THREADS=16', fontsize=24)
plt.xticks(ii, uu)
plt.setp(ax3.get_xticklabels(), rotation=90, ha='left')
ax3.legend(p_list,keys,bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.,)
plt.savefig('omp_1kframes_allsolvers_omp16.pdf', pad_inches=10)
 |
So far we have examined the single threaded and OpenMP enabled solver performance. Strumpack also allows us to use the distributed MPI backend. We can set this up as follows using the ipyparallel backend, or invoke an mpirun call using the cell bash bash environment. For completeness, the ipyparallel environment can be set up as follows:
conda install ipyparallel
jupyter serverextension enable --py ipyparallel --user
jupyter nbextension install --py ipyparallel --user
jupyter nbextension enable --py ipyparallel --user
ipcluster start --profile=mpi -n 4
Take note of the controller client json file <HOME>/.ipython/profile_<PROFILE_NAME>/security/ipcontroller-client.json. This will be used to allow the notebook to connect to the ipyparallel cluster profile mpi. We can then return the handle of the cluster environment with:
import ipyparallel as ipp
#c = ipp.Client('/net/cci-filer2/raid1/home/mlxd/.ipython/profile_mpi/security/ipcontroller-client.json')
c = ipp.Client()
rc=c[:]
c.ids #Instance IDs
[0]
For our purposes it can be easier to call the solver using the ! bash environment, as was used for the OpenMP studies earlier. A simple test script can be performed as follows, which tests the listed solvers against the MPI backend.
!OMP_NUM_THREADS=1 mpirun -np 2 libtbx.python \
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/\
modules/cctbx_project/scitbx/examples/bevington/strumpack_eigen_solver_mpi_dist.py \
/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/\
out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv \
/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/\
out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.
try: mod = __import__(name)
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.
try: mod = __import__(name)
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.
try: mod = __import__(name)
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.
try: mod = __import__(name)
14151 7075.5
# ***** WARNING ****************************************************
# Detected a large number of levels in the frontal/elimination tree.
# STRUMPACK currently does not handle this safely, which
# could lead to segmentation faults due to stack overflows.
# As a remedy, you can try to increase the stack size,
# or try a different ordering (metis, scotch, ..).
# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,
# (enabled by default) iso --sp_enable_METIS_NodeND.
# ******************************************************************
individual call time for STRUMPACK_OMP: CPU, 0.350s; elapsed, 0.344s
individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.050s
individual call time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.027s
CPP -2
# Initializing STRUMPACK
# using 1 OpenMP thread
CPP -2
# using 2 MPI processes
# matching job: maximum matching with row and column scaling
# initial matrix:
# - number of unknowns = 14,151
# - number of nonzeros = 72,305
# nested dissection reordering:
# - Scotch reordering
# - strategy parameter = 8
# - number of separators = 4,075
# - number of levels = 68
# ***** WARNING ****************************************************
# Detected a large number of levels in the frontal/elimination tree.
# STRUMPACK currently does not handle this safely, which
# could lead to segmentation faults due to stack overflows.
# As a remedy, you can try to increase the stack size,
# or try a different ordering (metis, scotch, ..).
# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,
# (enabled by default) iso --sp_enable_METIS_NodeND.
# ******************************************************************
# - nd time = 0.0810261
# - matching time = 0.0116782
# - symmetrization time = 0.00337219
# symbolic factorization:
# - nr of dense Frontal matrices = 4,075
# - nr of HSS Frontal matrices = 0
# - symb-factor time = 0.021524
# multifrontal factorization:
# - estimated memory usage (exact solver) = 7.70359 MB
# - factor time = 0.223322
# - factor nonzeros = 962,949
# - factor memory = 7.70359 MB
# - total flops = 2.49112e+08, min = 3.46562e+07, max = 2.14456e+08
# - flop rate = 1.11548 GFlop/s
# - factor memory/nonzeros = 100 % of multifrontal
# - maximum HSS rank = 0
# - HSS compression = false
# - relative compression tolerance = 0.01
# - absolute compression tolerance = 1e-08
# - normal(0,1) distribution with minstd_rand engine
# ----- FLOP BREAKDOWN ---------------------
# compression = 0
# random = 0
# ID = 0
# QR = 0
# ortho = 0
# reduce_samples = 0
# update_samples = 0
# extraction = 0
# sampling = 0
# CB_sample = 0
# sparse_sampling = 0
# ULV_factor = 0
# Schur = 0
# full_rank = 2.63271e+08
# --------------------------------------------
# total = 2.63271e+08
# --------------------------------------------
REFINEMENT it. 0 res = 15322 rel.res = 1 bw.error = 1
REFINEMENT it. 1 res = 1.1079e-11 rel.res = 7.23077e-16 bw.error = 1.25632e-15
# DIRECT/GMRES solve:
# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000
# - number of Krylov iterations = 1
# - solve time = 0.011821
# - total flops = 2.47602e+06, min = 838849, max = 1.63717e+06
# - flop rate = 0.209459 GFlop/s
# - bytes moved = 21.8361 MB, min = 0 MB, max = 0 MB
# - byte rate = 1.84722 GByte/s
# - solve arithmetic intensity = 0.113391 flop/byte
individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.350s; elapsed, 0.359s
individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.370s; elapsed, 0.365s
Exiting profiler
time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.350s; elapsed, 0.359s, #calls: 1
TOTAL : CPU, 0.350s; elapsed, 0.359s
Strumpack solutions agree with Eigen
Exiting profiler
time for STRUMPACK_OMP: CPU, 0.350s; elapsed, 0.344s, #calls: 1
time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.027s, #calls: 1
time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.050s, #calls: 1
time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.370s; elapsed, 0.365s, #calls: 1
TOTAL : CPU, 0.790s; elapsed, 0.786s
We now use the same procedure of creating a solver file, write to disk, and call with the respective matrix arguments with OMP_NUM_THREADS and mpirun -n. For simplicity, we examine the OpenMP STRUM_SCOTCHA, EIGEN_LDLT and EIGEN_BICGSTAB, compared with the MPI-enabled STRUM_SCOTCHA (we may also examine the full range of algorithms as used in the previous example).
MPI_SOLVER='''
from __future__ import division
import mpi4py
mpi4py.rc.threads = True
mpi4py.rc.thread_level = "funneled"
from mpi4py import MPI
assert MPI.Is_initialized()
assert MPI.Query_thread() == MPI.THREAD_FUNNELED
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size()
from scitbx.matrix import sqr,col
from cctbx.array_family import flex
from libtbx.test_utils import approx_equal
from libtbx.development.timers import Profiler
import boost.python
ext_omp = boost.python.import_ext("scitbx_examples_strumpack_solver_ext")
ext_mpi = boost.python.import_ext("scitbx_examples_strumpack_mpi_dist_solver_ext")
import sys
import numpy as np
import scipy.sparse as sps
if rank==0:
A_mat = np.loadtxt(sys.argv[1],dtype={'names':('rows','cols','vals'),'formats':('i8','i8','f8')})
b_vec = np.loadtxt(sys.argv[2])
n_rows = len(b_vec)
n_cols = n_rows
nnz = len(A_mat['vals'])
print n_rows, n_rows/size
#Convert the sparse CSR to flex doubles, then use them to solve using the implemented framework
A_sp = sps.csr_matrix((A_mat['vals'],(A_mat['rows'],A_mat['cols'])))
#Check if upper/lower triangular only, and generate full if so
tu=sps.triu(A_sp)
tl=sps.tril(A_sp)
sd=sps.diags(A_sp.diagonal())
A_spS = A_sp
if tu.nnz == sd.getnnz() or tl.nnz == sd.getnnz():
A_spS = A_sp + A_sp.transpose() - sd
import numpy as np
row_idx_split = np.array_split(np.arange(n_rows),size)
len_row_idx_split = flex.int( np.cumsum( np.append([0], [len(i) for i in row_idx_split]) ).tolist() )
P = Profiler("STRUMPACK_OMP")
A_indptr = flex.int( A_spS.indptr )
A_indices = flex.int( A_spS.indices )
A_data = flex.double( A_spS.data )
b = flex.double( b_vec )
res_strum_omp = ext_omp.strumpack_solver(
n_rows, n_cols,
A_indptr, A_indices,
A_data, b,
ext_omp.scotch,
ext_omp.auto
)
del P
P = Profiler("EIGEN_LDLT")
res_eig_ldlt = ext_omp.eigen_solver(1, n_rows, n_cols, A_indptr, A_indices, A_data, b)
del P
P = Profiler("EIGEN_BICGSTAB")
res_eig_bicgstab = ext_omp.eigen_solver(2, n_rows, n_cols, A_indptr, A_indices, A_data, b)
del P
else:
A_spS=None
row_idx_split=None
len_row_idx_split=None
b_vec=None
n_cols=None
if size>1:
#Broadcast data to each rank
A_spS = comm.bcast(A_spS, root=0)
row_idx_split = comm.bcast(row_idx_split, root=0)
len_row_idx_split = comm.bcast(len_row_idx_split, root=0)
b_vec = comm.bcast(b_vec, root=0)
n_cols = comm.bcast(n_cols, root=0)
#Take subset of data for each rank
A_row_offset = flex.int(A_spS[row_idx_split[rank],:].indptr)
A_col_offset = flex.int(A_spS[row_idx_split[rank],:].indices)
A_values = flex.double(A_spS[row_idx_split[rank],:].data)
b = flex.double(b_vec[row_idx_split[rank]])
P = Profiler("STRUMPACK_MPI_DIST_RANK=%d"%rank)
res_strum_mpi_local = ext_mpi.strumpack_mpi_dist_solver(len(row_idx_split[rank]), n_cols, comm, A_row_offset, A_col_offset, A_values, b, len_row_idx_split, ext_mpi.scotch, ext_mpi.auto)
strum_result_mpi_list = comm.gather(res_strum_mpi_local.x, root=0)
del P
if rank==0:
strum_result_mpi = flex.double()
for l in strum_result_mpi_list:
strum_result_mpi.extend(l)
#MPI.Finalize()
if rank==0:
strum_result_omp = res_strum_omp.x
eig_result_ldlt = res_eig_ldlt.x
eig_result_bicgstab = res_eig_bicgstab.x
for ii in xrange(len(strum_result_omp)):
assert approx_equal( strum_result_omp[ii], eig_result_ldlt[ii] )
assert approx_equal( strum_result_omp[ii], eig_result_bicgstab[ii] )
if size>1:
assert approx_equal( strum_result_omp[ii], strum_result_mpi[ii] )
print "Strumpack solutions agree with Eigen"
'''
MPI_SOLVER_FILE = open("MPI_SOLVER.py", "w")
MPI_SOLVER_FILE.write(MPI_SOLVER)
MPI_SOLVER_FILE.close()
DATAPATH="/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/"
A_LIST = !find {DATAPATH} -iname "A*.csv"
B_LIST = [ii.replace('/A_','/b_') for ii in A_LIST]
We intend to examine performance while varying both OMP_NUM_THREADS and the MPI process number to determine an optimal set of parameters for the distributed backend. In the following example we test between 1 to 16 OpenMP threads per process, and from 1 to 16 MPI processes, assuming a total of threads*processes <= 32 cores. The resulting data set will be a scalar field of time as a function of both variables, which we can plot as a heatmap. We can begin by running the analysis on a 1000 image dataset.
str_out={}
import os
threads_list = [1,2,4,8,16]
mpi_proc_list = [1,2,4,8,16]
for imgs_size in list_idx:
print "Data Set Size:=%s"%imgs_size
#Subselect the smallest data size for now
if imgs_size != "1k":
print "Skipping %s"%imgs_size
continue
for imgs_idx in list_idx[imgs_size]:
A_path = A_LIST[imgs_idx]; b_path = B_LIST[imgs_idx]
dat_name = A_path.split('/')[-1][2:-4]
print "Data Set Name:=%s"%(dat_name)
#Ensure the A and b data are matched correctly
assert(os.path.dirname(A_path) == os.path.dirname(b_path))
print {A_path}, {b_path}
for procs in mpi_proc_list:
for threads in threads_list:
if threads*procs <= 32:
print "mpirun -n %d"%procs
print "OMP_NUM_THREADS:=%d"%threads
val = !OMP_NUM_THREADS={threads} mpirun -n {procs} libtbx.python ./MPI_SOLVER.py {A_path} {b_path}
print val
for s in val:
if "assert" in s:
raise Exception("Solutions not equal! Halting with log: %s"%val)
key = 'mpi' + str(procs) + 'omp' + str(threads) + '_' + dat_name
str_out.update({key:val})
—CLICK FOR OUTPUT—
Data Set Size:=10k
Skipping 10k
Data Set Size:=1k
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 14151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.330s; elapsed, 0.332s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.047s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.025s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.330s; elapsed, 0.332s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.025s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.047s, #calls: 1', 'TOTAL : CPU, 0.400s; elapsed, 0.404s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 14151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.490s; elapsed, 0.251s', 'individual call time for EIGEN_LDLT: CPU, 0.090s; elapsed, 0.047s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.023s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.490s; elapsed, 0.251s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.090s; elapsed, 0.047s, #calls: 1', 'TOTAL : CPU, 0.630s; elapsed, 0.321s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 14151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.720s; elapsed, 0.189s', 'individual call time for EIGEN_LDLT: CPU, 0.210s; elapsed, 0.053s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.022s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.720s; elapsed, 0.189s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.022s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.210s; elapsed, 0.053s, #calls: 1', 'TOTAL : CPU, 1.020s; elapsed, 0.264s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 14151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.570s; elapsed, 0.208s', 'individual call time for EIGEN_LDLT: CPU, 0.390s; elapsed, 0.048s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.020s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.570s; elapsed, 0.208s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.020s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.390s; elapsed, 0.048s, #calls: 1', 'TOTAL : CPU, 2.120s; elapsed, 0.277s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 14151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.140s; elapsed, 0.213s', 'individual call time for EIGEN_LDLT: CPU, 1.260s; elapsed, 0.079s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.370s; elapsed, 0.023s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.140s; elapsed, 0.213s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.370s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.260s; elapsed, 0.079s, #calls: 1', 'TOTAL : CPU, 4.770s; elapsed, 0.315s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 7075.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.340s; elapsed, 0.351s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.050s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.026s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.# - nd time = ', '# ******************************************************************', '0.0834329', '# - matching time = 0.0113571', '# - symmetrization time = 0.00335288', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0144382', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.70359 MB', '# - factor time = 0.188955', '# - factor nonzeros = 962,949', '# - factor memory = 7.70359 MB', '# - total flops = 2.49112e+08, min = 3.46562e+07, max = 2.14456e+08', '# - flop rate = 1.31837 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.63271e+08', '# --------------------------------------------', '# total = 2.63271e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.1079e-11\trel.res = 7.23077e-16\tbw.error = 1.25632e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0111809', '# - total flops = 2.47602e+06, min = 838849, max = 1.63717e+06', '# - flop rate = 0.221451 GFlop/s', '# - bytes moved = 21.8361 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.95298 GByte/s', '# - solve arithmetic intensity = 0.113391 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.320s; elapsed, 0.319s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.310s; elapsed, 0.323s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.320s; elapsed, 0.319s, #calls: 1', 'TOTAL : CPU, 0.320s; elapsed, 0.319s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.340s; elapsed, 0.351s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.026s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.050s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.310s; elapsed, 0.323s, #calls: 1', 'TOTAL : CPU, 0.730s; elapsed, 0.750s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 7075.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.520s; elapsed, 0.265s', 'individual call time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.050s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.023s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0841191', '# - matching time = 0.0118749', '# - symmetrization time = 0.00408697', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.019855', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.70359 MB', '# - factor time = 0.112911', '# - factor nonzeros = 962,949', '# - factor memory = 7.70359 MB', '# - total flops = 2.49139e+08, min = 3.46562e+07, max = 2.14483e+08', '# - flop rate = 2.20651 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.63271e+08', '# --------------------------------------------', '# total = 2.63271e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.04075e-11\trel.res = 6.79254e-16\tbw.error = 1.20867e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.010448', '# - total flops = 2.47602e+06, min = 838849, max = 1.63717e+06', '# - flop rate = 0.236985 GFlop/s', '# - bytes moved = 21.8492 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.09124 GByte/s', '# - solve arithmetic intensity = 0.113323 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.500s; elapsed, 0.253s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.510s; elapsed, 0.259s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.500s; elapsed, 0.253s, #calls: 1', 'TOTAL : CPU, 0.500s; elapsed, 0.253s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.520s; elapsed, 0.265s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.050s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.510s; elapsed, 0.259s, #calls: 1', 'TOTAL : CPU, 1.180s; elapsed, 0.596s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 7075.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.940s; elapsed, 0.248s', 'individual call time for EIGEN_LDLT: CPU, 0.300s; elapsed, 0.076s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.025s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0842121', '# - matching time = 0.0123389', '# - symmetrization time = 0.00392413', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.016829', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.70359 MB', '# - factor time = 0.09586', '# - factor nonzeros = 962,949', '# - factor memory = 7.70359 MB', '# - total flops = 2.49144e+08, min = 3.46562e+07, max = 2.14488e+08', '# - flop rate = 2.59904 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.63271e+08', '# --------------------------------------------', '# total = 2.63271e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.02827e-11\trel.res = 6.71106e-16\tbw.error = 1.20855e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.01089', '# - total flops = 2.47602e+06, min = 838849, max = 1.63717e+06', '# - flop rate = 0.227366 GFlop/s', '# - bytes moved = 21.8668 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.00797 GByte/s', '# - solve arithmetic intensity = 0.113232 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.910s; elapsed, 0.234s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.960s; elapsed, 0.240s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.910s; elapsed, 0.234s, #calls: 1', 'TOTAL : CPU, 0.910s; elapsed, 0.234s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.940s; elapsed, 0.248s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.025s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.300s; elapsed, 0.076s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.960s; elapsed, 0.240s, #calls: 1', 'TOTAL : CPU, 2.300s; elapsed, 0.588s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 7075.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.440s; elapsed, 0.191s', 'individual call time for EIGEN_LDLT: CPU, 0.390s; elapsed, 0.049s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.020s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0945621', '# - matching time = 0.013525', '# - symmetrization time = 0.0044291', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.019686', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.70359 MB', '# - factor time = 0.102882', '# - factor nonzeros = 962,949', '# - factor memory = 7.70359 MB', '# - total flops = 2.49146e+08, min = 3.46578e+07, max = 2.14488e+08', '# - flop rate = 2.42166 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.63271e+08', '# --------------------------------------------', '# total = 2.63271e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.00272e-11\trel.res = 6.54433e-16\tbw.error = 1.2116e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0105138', '# - total flops = 2.47602e+06, min = 838849, max = 1.63717e+06', '# - flop rate = 0.235502 GFlop/s', '# - bytes moved = 21.8836 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.08142 GByte/s', '# - solve arithmetic intensity = 0.113145 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.010s; elapsed, 0.256s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.090s; elapsed, 0.262s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.010s; elapsed, 0.256s, #calls: 1', 'TOTAL : CPU, 2.010s; elapsed, 0.256s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.440s; elapsed, 0.191s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.020s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.390s; elapsed, 0.049s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.090s; elapsed, 0.262s, #calls: 1', 'TOTAL : CPU, 4.080s; elapsed, 0.522s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 7075.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.370s; elapsed, 0.231s', 'individual call time for EIGEN_LDLT: CPU, 0.920s; elapsed, 0.057s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.350s; elapsed, 0.022s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', '# using 2 MPI processesCPP -2', '', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0956161', '# - matching time = 0.0146439', '# - symmetrization time = 0.00466204', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0220499', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.70359 MB', '# - factor time = 0.113796', '# - factor nonzeros = 962,949', '# - factor memory = 7.70359 MB', '# - total flops = 2.49152e+08, min = 3.46594e+07, max = 2.14492e+08', '# - flop rate = 2.18946 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.63271e+08', '# --------------------------------------------', '# total = 2.63271e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.00817e-11\trel.res = 6.57988e-16\tbw.error = 1.2116e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.010931', '# - total flops = 2.47602e+06, min = 838849, max = 1.63717e+06', '# - flop rate = 0.226513 GFlop/s', '# - bytes moved = 21.8961 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.00312 GByte/s', '# - solve arithmetic intensity = 0.11308 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.050s; elapsed, 0.273s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.440s; elapsed, 0.279s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.050s; elapsed, 0.273s, #calls: 1', 'TOTAL : CPU, 4.050s; elapsed, 0.273s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.370s; elapsed, 0.231s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.350s; elapsed, 0.022s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.920s; elapsed, 0.057s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.440s; elapsed, 0.279s, #calls: 1', 'TOTAL : CPU, 9.080s; elapsed, 0.589s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 3537.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.380s; elapsed, 0.383s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.054s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.029s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************# - nd time = ', '0.0921421', '# - matching time = 0.012584', '# - symmetrization time = 0.00509715', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0135608', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.71692 MB', '# - factor time = 0.142919', '# - factor nonzeros = 964,615', '# - factor memory = 7.71692 MB', '# - total flops = 2.5035e+08, min = 2.90159e+07, max = 1.34565e+08', '# - flop rate = 1.75169 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.63777e+08', '# --------------------------------------------', '# total = 2.63777e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.13122e-11\trel.res = 7.383e-16\tbw.error = 2.51177e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.013602', '# - total flops = 2.86909e+06, min = 493941, max = 897810', '# - flop rate = 0.210931 GFlop/s', '# - bytes moved = 20.6003 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.5145 GByte/s', '# - solve arithmetic intensity = 0.139274 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.260s; elapsed, 0.286s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.250s; elapsed, 0.288s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.270s; elapsed, 0.284s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.290s; elapsed, 0.290s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.250s; elapsed, 0.288s, #calls: 1', 'TOTAL : CPU, 0.250s; elapsed, 0.288s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.270s; elapsed, 0.284s, #calls: 1', 'TOTAL : CPU, 0.270s; elapsed, 0.284s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.260s; elapsed, 0.286s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.286s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.380s; elapsed, 0.383s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.029s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.054s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.290s; elapsed, 0.290s, #calls: 1', 'TOTAL : CPU, 0.750s; elapsed, 0.756s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 3537.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.530s; elapsed, 0.270s', 'individual call time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.053s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.024s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.093828', '# - matching time = 0.0128682', '# - symmetrization time = 0.00457501', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0154769', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.71692 MB', '# - factor time = 0.103127', '# - factor nonzeros = 964,615', '# - factor memory = 7.71692 MB', '# - total flops = 2.50361e+08, min = 2.90159e+07, max = 1.34577e+08', '# - flop rate = 2.4277 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.63777e+08', '# --------------------------------------------', '# total = 2.63777e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.13019e-11\trel.res = 7.37623e-16\tbw.error = 2.51177e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0116901', '# - total flops = 2.86909e+06, min = 493941, max = 897810', '# - flop rate = 0.245428 GFlop/s', '# - bytes moved = 20.6104 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.76305 GByte/s', '# - solve arithmetic intensity = 0.139206 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.490s; elapsed, 0.253s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.490s; elapsed, 0.250s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.500s; elapsed, 0.252s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.510s; elapsed, 0.256s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.490s; elapsed, 0.253s, #calls: 1', 'TOTAL : CPU, 0.490s; elapsed, 0.253s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.490s; elapsed, 0.250s, #calls: 1', 'TOTAL : CPU, 0.490s; elapsed, 0.250s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.500s; elapsed, 0.252s, #calls: 1', 'TOTAL : CPU, 0.500s; elapsed, 0.252s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.530s; elapsed, 0.270s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.024s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.053s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.510s; elapsed, 0.256s, #calls: 1', 'TOTAL : CPU, 1.200s; elapsed, 0.603s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 3537.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.900s; elapsed, 0.237s', 'individual call time for EIGEN_LDLT: CPU, 0.220s; elapsed, 0.056s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.021s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0950179', '# - matching time = 0.01349', '# - symmetrization time = 0.00556684', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.016129', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.71692 MB', '# - factor time = 0.103751', '# - factor nonzeros = 964,615', '# - factor memory = 7.71692 MB', '# - total flops = 2.50364e+08, min = 2.90159e+07, max = 1.34579e+08', '# - flop rate = 2.41312 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.63777e+08', '# --------------------------------------------', '# total = 2.63777e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.12775e-11\trel.res = 7.3603e-16\tbw.error = 2.51177e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0118899', '# - total flops = 2.86909e+06, min = 493941, max = 897810', '# - flop rate = 0.241304 GFlop/s', '# - bytes moved = 20.623 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.73449 GByte/s', '# - solve arithmetic intensity = 0.139121 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.000s; elapsed, 0.254s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.000s; elapsed, 0.256s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.020s; elapsed, 0.257s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.050s; elapsed, 0.261s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.000s; elapsed, 0.256s, #calls: 1', 'TOTAL : CPU, 1.000s; elapsed, 0.256s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.000s; elapsed, 0.254s, #calls: 1', 'TOTAL : CPU, 1.000s; elapsed, 0.254s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.020s; elapsed, 0.257s, #calls: 1', 'TOTAL : CPU, 1.020s; elapsed, 0.257s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.900s; elapsed, 0.237s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.220s; elapsed, 0.056s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.050s; elapsed, 0.261s, #calls: 1', 'TOTAL : CPU, 2.240s; elapsed, 0.574s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 3537.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.670s; elapsed, 0.223s', 'individual call time for EIGEN_LDLT: CPU, 0.650s; elapsed, 0.081s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.024s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.095717', '# - matching time = 0.0142171', '# - symmetrization time = 0.00552583', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0187118', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.71692 MB', '# - factor time = 0.131734', '# - factor nonzeros = 964,615', '# - factor memory = 7.71692 MB', '# - total flops = 2.50371e+08, min = 2.90175e+07, max = 1.34581e+08', '# - flop rate = 1.90058 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.63777e+08', '# --------------------------------------------', '# total = 2.63777e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.13334e-11\trel.res = 7.3968e-16\tbw.error = 2.51177e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.013279', '# - total flops = 2.86909e+06, min = 493941, max = 897810', '# - flop rate = 0.216063 GFlop/s', '# - bytes moved = 20.6319 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.55373 GByte/s', '# - solve arithmetic intensity = 0.139061 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.290s; elapsed, 0.293s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.280s; elapsed, 0.291sindividual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.300s; elapsed, 0.294s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.370s; elapsed, 0.297s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.290s; elapsed, 0.293s, #calls: 1', 'TOTAL : CPU, 2.290s; elapsed, 0.293s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.300s; elapsed, 0.294s, #calls: 1', 'TOTAL : CPU, 2.300s; elapsed, 0.294s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.280s; elapsed, 0.291s, #calls: 1', 'TOTAL : CPU, 2.280s; elapsed, 0.291s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.670s; elapsed, 0.223s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.190s; elapsed, 0.024s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.650s; elapsed, 0.081s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.370s; elapsed, 0.297s, #calls: 1', 'TOTAL : CPU, 4.880s; elapsed, 0.625s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 1768.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.370s; elapsed, 0.374s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.054s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.029s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.092581', '# - matching time = 0.0129061', '# - symmetrization time = 0.00472403', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0125499', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.6429 MB', '# - factor time = 0.120092', '# - factor nonzeros = 955,363', '# - factor memory = 7.6429 MB', '# - total flops = 2.43618e+08, min = 181487, max = 9.78935e+07', '# - flop rate = 2.02859 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.55532e+08', '# --------------------------------------------', '# total = 2.55532e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.95445e-12\trel.res = 6.49683e-16\tbw.error = 8.94762e-16', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0136209', '# - total flops = 3.54215e+06, min = 18825, max = 800361', '# - flop rate = 0.260053 GFlop/s', '# - bytes moved = 19.924 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.46276 GByte/s', '# - solve arithmetic intensity = 0.177783 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.240s; elapsed, 0.262s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.240s; elapsed, 0.261s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.230s; elapsed, 0.261s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.250s; elapsed, 0.262s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.250s; elapsed, 0.262s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.260s; elapsed, 0.261s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.240s; elapsed, 0.262s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.270s; elapsed, 0.266s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.260s; elapsed, 0.261s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.261s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.240s; elapsed, 0.262s, #calls: 1', 'TOTAL : CPU, 0.240s; elapsed, 0.262s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.240s; elapsed, 0.262s, #calls: 1', 'TOTAL : CPU, 0.240s; elapsed, 0.262s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.250s; elapsed, 0.262s, #calls: 1', 'TOTAL : CPU, 0.250s; elapsed, 0.262s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.240s; elapsed, 0.261s, #calls: 1', 'TOTAL : CPU, 0.240s; elapsed, 0.261s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.230s; elapsed, 0.261s, #calls: 1', 'TOTAL : CPU, 0.230s; elapsed, 0.261s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.250s; elapsed, 0.262s, #calls: 1', 'TOTAL : CPU, 0.250s; elapsed, 0.262s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.370s; elapsed, 0.374s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.029s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.054s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.270s; elapsed, 0.266s, #calls: 1', 'TOTAL : CPU, 0.720s; elapsed, 0.723s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 1768.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.490s; elapsed, 0.256s', 'individual call time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.055s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.026s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', '2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2CPP -2', '', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 66', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.087255', '# - matching time = 0.015729', '# - symmetrization time = 0.00505519', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.013561', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.6429 MB', '# - factor time = 0.111239', '# - factor nonzeros = 955,363', '# - factor memory = 7.6429 MB', '# - total flops = 2.43625e+08, min = 181487, max = 9.79012e+07', '# - flop rate = 2.19011 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.55532e+08', '# --------------------------------------------', '# total = 2.55532e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.96659e-12\trel.res = 6.50475e-16\tbw.error = 8.85161e-16', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0127609', '# - total flops = 3.54215e+06, min = 18825, max = 800361', '# - flop rate = 0.277579 GFlop/s', '# - bytes moved = 19.9308 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.56187 GByte/s', '# - solve arithmetic intensity = 0.177722 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.500s; elapsed, 0.255s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.490s; elapsed, 0.255s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.490s; elapsed, 0.253s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.490s; elapsed, 0.255s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.500s; elapsed, 0.255s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.500s; elapsed, 0.256s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.500s; elapsed, 0.257s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.520s; elapsed, 0.260s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.500s; elapsed, 0.255s, #calls: 1', 'TOTAL : CPU, 0.500s; elapsed, 0.255s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.500s; elapsed, 0.255s, #calls: 1', 'TOTAL : CPU, 0.500s; elapsed, 0.255s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.490s; elapsed, 0.255s, #calls: 1', 'TOTAL : CPU, 0.490s; elapsed, 0.255s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.490s; elapsed, 0.255s, #calls: 1', 'TOTAL : CPU, 0.490s; elapsed, 0.255s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.500s; elapsed, 0.256s, #calls: 1', 'TOTAL : CPU, 0.500s; elapsed, 0.256s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.490s; elapsed, 0.253s, #calls: 1', 'TOTAL : CPU, 0.490s; elapsed, 0.253s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.500s; elapsed, 0.257s, #calls: 1', 'TOTAL : CPU, 0.500s; elapsed, 0.257s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.490s; elapsed, 0.256s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.026s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.055s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.520s; elapsed, 0.260s, #calls: 1', 'TOTAL : CPU, 1.180s; elapsed, 0.597s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 1768.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.900s; elapsed, 0.237s', 'individual call time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.054s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.023s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.103038', '# - matching time = 0.0141149', '# - symmetrization time = 0.00636292', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.018019', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.6429 MB', '# - factor time = 0.14164', '# - factor nonzeros = 955,363', '# - factor memory = 7.6429 MB', '# - total flops = 2.43632e+08, min = 183182, max = 9.79029e+07', '# - flop rate = 1.72008 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.55532e+08', '# --------------------------------------------', '# total = 2.55532e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.72427e-12\trel.res = 6.3466e-16\tbw.error = 8.82623e-16', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0138209', '# - total flops = 3.54215e+06, min = 18825, max = 800361', '# - flop rate = 0.256289 GFlop/s', '# - bytes moved = 19.9395 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.44271 GByte/s', '# - solve arithmetic intensity = 0.177644 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.200s; elapsed, 0.307s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.210s; elapsed, 0.309sindividual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.210s; elapsed, 0.308s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.150s; elapsed, 0.309s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.220s; elapsed, 0.310sindividual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.210s; elapsed, 0.310s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.200s; elapsed, 0.310s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.260s; elapsed, 0.314s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.200s; elapsed, 0.307s, #calls: 1', 'TOTAL : CPU, 1.200s; elapsed, 0.307s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.210s; elapsed, 0.309s, #calls: 1', 'TOTAL : CPU, 1.210s; elapsed, 0.309s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.210s; elapsed, 0.308s, #calls: 1', 'TOTAL : CPU, 1.210s; elapsed, 0.308s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.220s; elapsed, 0.310s, #calls: 1', 'TOTAL : CPU, 1.220s; elapsed, 0.310s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.210s; elapsed, 0.310s, #calls: 1', 'Exiting profilerTOTAL : CPU, 1.210s; elapsed, 0.310s', '', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.150s; elapsed, 0.309s, #calls: 1', 'TOTAL : CPU, 1.150s; elapsed, 0.309s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.200s; elapsed, 0.310s, #calls: 1', 'TOTAL : CPU, 1.200s; elapsed, 0.310s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.900s; elapsed, 0.237s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.054s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.260s; elapsed, 0.314s, #calls: 1', 'TOTAL : CPU, 2.470s; elapsed, 0.629s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 884.4375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.400s; elapsed, 0.399s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.056s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.030s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.096596', '# - matching time = 0.0133419', '# - symmetrization time = 0.00511479', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.014358', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.6429 MB', '# - factor time = 0.155044', '# - factor nonzeros = 955,363', '# - factor memory = 7.6429 MB', '# - total flops = 2.43618e+08, min = 178295, max = 4.76965e+07', '# - flop rate = 1.57128 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.52727e+08', '# --------------------------------------------', '# total = 2.52727e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.03051e-11\trel.res = 6.72566e-16\tbw.error = 9.34049e-16', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0169559', '# - total flops = 5.0589e+06, min = 14405, max = 774758', '# - flop rate = 0.298357 GFlop/s', '# - bytes moved = 18.8309 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.11058 GByte/s', '# - solve arithmetic intensity = 0.268649 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.240s; elapsed, 0.314sindividual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.300s; elapsed, 0.315s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.250s; elapsed, 0.314s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.290s; elapsed, 0.315s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.210s; elapsed, 0.314s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.310s; elapsed, 0.314s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.220s; elapsed, 0.314s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.300s; elapsed, 0.312sindividual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.310s; elapsed, 0.314s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.260s; elapsed, 0.313s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.280s; elapsed, 0.314s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.270s; elapsed, 0.313s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.290s; elapsed, 0.314s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.280s; elapsed, 0.314s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.280s; elapsed, 0.315s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.320s; elapsed, 0.320s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.240s; elapsed, 0.314s, #calls: 1', 'TOTAL : CPU, 0.240s; elapsed, 0.314s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.300s; elapsed, 0.315s, #calls: 1', 'TOTAL : CPU, 0.300s; elapsed, 0.315s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.270s; elapsed, 0.313s, #calls: 1', 'TOTAL : CPU, 0.270s; elapsed, 0.313s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.310s; elapsed, 0.314s, #calls: 1', 'TOTAL : CPU, 0.310s; elapsed, 0.314s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.290s; elapsed, 0.315s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.315s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.250s; elapsed, 0.314s, #calls: 1', 'TOTAL : CPU, 0.250s; elapsed, 0.314s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.290s; elapsed, 0.314s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.314s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.210s; elapsed, 0.314s, #calls: 1', 'TOTAL : CPU, 0.210s; elapsed, 0.314s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.260s; elapsed, 0.313s, #calls: 1', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.310s; elapsed, 0.314s, #calls: 1TOTAL : CPU, 0.260s; elapsed, 0.313s', '', 'TOTAL : CPU, 0.310s; elapsed, 0.314s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.280s; elapsed, 0.314s, #calls: 1', 'TOTAL : CPU, 0.280s; elapsed, 0.314s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.220s; elapsed, 0.314s, #calls: 1', 'TOTAL : CPU, 0.220s; elapsed, 0.314s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.280s; elapsed, 0.314s, #calls: 1Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.300s; elapsed, 0.312s, #calls: 1', 'TOTAL : CPU, 0.300s; elapsed, 0.312s', '', 'TOTAL : CPU, 0.280s; elapsed, 0.314s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.280s; elapsed, 0.315s, #calls: 1', 'TOTAL : CPU, 0.280s; elapsed, 0.315s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.400s; elapsed, 0.399s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.030s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.056s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.320s; elapsed, 0.320s, #calls: 1', 'TOTAL : CPU, 0.800s; elapsed, 0.805s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 884.4375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.540s; elapsed, 0.278s', 'individual call time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.054s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.024s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.09727', '# - matching time = 0.014107', '# - symmetrization time = 0.00612402', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0187101', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.6429 MB', '# - factor time = 0.138211', '# - factor nonzeros = 955,363', '# - factor memory = 7.6429 MB', '# - total flops = 2.43622e+08, min = 178295, max = 4.76965e+07', '# - flop rate = 1.76268 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.52727e+08', '# --------------------------------------------', '# total = 2.52727e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.0207e-11\trel.res = 6.66167e-16\tbw.error = 9.34544e-16', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.01372', '# - total flops = 5.0589e+06, min = 14405, max = 774758', '# - flop rate = 0.368723 GFlop/s', '# - bytes moved = 18.8371 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.37296 GByte/s', '# - solve arithmetic intensity = 0.26856 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.610s; elapsed, 0.305s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.590s; elapsed, 0.307sindividual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.560s; elapsed, 0.306s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.590s; elapsed, 0.304s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.590s; elapsed, 0.303s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.580s; elapsed, 0.305s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.610s; elapsed, 0.306s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.580s; elapsed, 0.307s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.570s; elapsed, 0.305s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.590s; elapsed, 0.306s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.570s; elapsed, 0.306s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.590s; elapsed, 0.306s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.580s; elapsed, 0.304s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.570s; elapsed, 0.305s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.570s; elapsed, 0.307s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.610s; elapsed, 0.311s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.610s; elapsed, 0.305s, #calls: 1', 'TOTAL : CPU, 0.610s; elapsed, 0.305s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.560s; elapsed, 0.306s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.306s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.590s; elapsed, 0.307s, #calls: 1', 'TOTAL : CPU, 0.590s; elapsed, 0.307s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.590s; elapsed, 0.303s, #calls: 1', 'TOTAL : CPU, 0.590s; elapsed, 0.303s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.590s; elapsed, 0.304s, #calls: 1', 'TOTAL : CPU, 0.590s; elapsed, 0.304s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.590s; elapsed, 0.306s, #calls: 1', 'TOTAL : CPU, 0.590s; elapsed, 0.306s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.570s; elapsed, 0.307s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.307sExiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.610s; elapsed, 0.306s, #calls: 1', 'TOTAL : CPU, 0.610s; elapsed, 0.306s', '', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.580s; elapsed, 0.305s, #calls: 1', 'TOTAL : CPU, 0.580s; elapsed, 0.305s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.580s; elapsed, 0.304s, #calls: 1', 'TOTAL : CPU, 0.580s; elapsed, 0.304s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.570s; elapsed, 0.305s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.305s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.580s; elapsed, 0.307s, #calls: 1', 'TOTAL : CPU, 0.580s; elapsed, 0.307s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.570s; elapsed, 0.306s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.306s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.590s; elapsed, 0.306s, #calls: 1', 'TOTAL : CPU, 0.590s; elapsed, 0.306s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.570s; elapsed, 0.305s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.305s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.540s; elapsed, 0.278s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.024s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.054s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.610s; elapsed, 0.311s, #calls: 1', 'TOTAL : CPU, 1.300s; elapsed, 0.667s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 13651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.940s; elapsed, 0.944s', 'individual call time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.219s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.066s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.940s; elapsed, 0.944s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.066s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.219s, #calls: 1', 'TOTAL : CPU, 1.230s; elapsed, 1.229s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 13651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.440s; elapsed, 0.736s', 'individual call time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.219s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.056s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.440s; elapsed, 0.736s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.056s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.219s, #calls: 1', 'TOTAL : CPU, 1.950s; elapsed, 1.011s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 13651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.680s; elapsed, 0.441s', 'individual call time for EIGEN_LDLT: CPU, 0.770s; elapsed, 0.231s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.048s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.680s; elapsed, 0.441s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.048s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.770s; elapsed, 0.231s, #calls: 1', 'TOTAL : CPU, 2.620s; elapsed, 0.720s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 13651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.330s; elapsed, 0.436s', 'individual call time for EIGEN_LDLT: CPU, 1.520s; elapsed, 0.248s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.042s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.330s; elapsed, 0.436s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.042s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.520s; elapsed, 0.248s, #calls: 1', 'TOTAL : CPU, 5.110s; elapsed, 0.727s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 13651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.930s; elapsed, 0.460s', 'individual call time for EIGEN_LDLT: CPU, 2.980s; elapsed, 0.242s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.039s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.930s; elapsed, 0.460s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.039s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.980s; elapsed, 0.242s, #calls: 1', 'TOTAL : CPU, 10.380s; elapsed, 0.740s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 6825.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.940s; elapsed, 0.941s', 'individual call time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.230s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.066s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,057', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.# - nd time = ', '# ******************************************************************', '0.0845411', '# - matching time = 0.0167019', '# - symmetrization time = 0.00645304', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,057', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0218749', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.3681 MB', '# - factor time = 0.613879', '# - factor nonzeros = 2,171,017', '# - factor memory = 17.3681 MB', '# - total flops = 1.1267e+09, min = 2.48891e+08, max = 8.77805e+08', '# - flop rate = 1.83537 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.18662e+09', '# --------------------------------------------', '# total = 1.18662e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.66736e-11\trel.res = 1.02959e-15\tbw.error = 2.06992e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0215859', '# - total flops = 5.32738e+06, min = 2.07883e+06, max = 3.24855e+06', '# - flop rate = 0.246799 GFlop/s', '# - bytes moved = 42.9647 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.9904 GByte/s', '# - solve arithmetic intensity = 0.123994 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.770s; elapsed, 0.776s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.790s; elapsed, 0.786s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.770s; elapsed, 0.776s, #calls: 1', 'TOTAL : CPU, 0.770s; elapsed, 0.776s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.940s; elapsed, 0.941s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.066s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.230s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.790s; elapsed, 0.786s, #calls: 1', 'TOTAL : CPU, 2.020s; elapsed, 2.023s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 6825.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.310s; elapsed, 0.672s', 'individual call time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.230s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.053s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,057', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0888541', '# - matching time = 0.018559', '# - symmetrization time = 0.00735211', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,057', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0272582', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.3681 MB', '# - factor time = 0.34786', '# - factor nonzeros = 2,171,017', '# - factor memory = 17.3681 MB', '# - total flops = 1.12676e+09, min = 2.48891e+08, max = 8.77865e+08', '# - flop rate = 3.23911 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.18662e+09', '# --------------------------------------------', '# total = 1.18662e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.67788e-11\trel.res = 1.03609e-15\tbw.error = 2.06057e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0152318', '# - total flops = 5.32999e+06, min = 2.07883e+06, max = 3.25116e+06', '# - flop rate = 0.349924 GFlop/s', '# - bytes moved = 43.0716 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.82773 GByte/s', '# - solve arithmetic intensity = 0.123747 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.020s; elapsed, 0.516s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.050s; elapsed, 0.525s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.020s; elapsed, 0.516s, #calls: 1', 'TOTAL : CPU, 1.020s; elapsed, 0.516s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.310s; elapsed, 0.672s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.053s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.230s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.050s; elapsed, 0.525s, #calls: 1', 'TOTAL : CPU, 2.880s; elapsed, 1.480s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 6825.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 2.000s; elapsed, 0.516s', 'individual call time for EIGEN_LDLT: CPU, 0.790s; elapsed, 0.225s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.046s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,057', '# - number of levels = 68', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.086098', '# - matching time = 0.0174599', '# - symmetrization time = 0.00688505', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,057', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0286288', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.3681 MB', '# - factor time = 0.277136', '# - factor nonzeros = 2,171,017', '# - factor memory = 17.3681 MB', '# - total flops = 1.12676e+09, min = 2.48891e+08, max = 8.77866e+08', '# - flop rate = 4.06572 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.18662e+09', '# --------------------------------------------', '# total = 1.18662e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.66738e-11\trel.res = 1.02961e-15\tbw.error = 2.06014e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0182171', '# - total flops = 5.32999e+06, min = 2.07883e+06, max = 3.25116e+06', '# - flop rate = 0.292582 GFlop/s', '# - bytes moved = 43.0824 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.36495 GByte/s', '# - solve arithmetic intensity = 0.123716 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.760s; elapsed, 0.446s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.810s; elapsed, 0.455s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.760s; elapsed, 0.446s, #calls: 1', 'TOTAL : CPU, 1.760s; elapsed, 0.446s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 2.000s; elapsed, 0.516s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.046s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.790s; elapsed, 0.225s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.810s; elapsed, 0.455s, #calls: 1', 'TOTAL : CPU, 4.760s; elapsed, 1.242s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 6825.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.720s; elapsed, 0.489s', 'individual call time for EIGEN_LDLT: CPU, 1.550s; elapsed, 0.243s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.040s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,057', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.10344', '# - matching time = 0.0205929', '# - symmetrization time = 0.00799203', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,057', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.029407', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.3681 MB', '# - factor time = 0.320545', '# - factor nonzeros = 2,171,017', '# - factor memory = 17.3681 MB', '# - total flops = 1.1268e+09, min = 2.48892e+08, max = 8.77908e+08', '# - flop rate = 3.51526 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.18662e+09', '# --------------------------------------------', '# total = 1.18662e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.71226e-11\trel.res = 1.05732e-15\tbw.error = 2.05844e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.019094', '# - total flops = 5.32999e+06, min = 2.07883e+06, max = 3.25116e+06', '# - flop rate = 0.279145 GFlop/s', '# - bytes moved = 43.1071 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.25763 GByte/s', '# - solve arithmetic intensity = 0.123645 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.060s; elapsed, 0.514s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.180s; elapsed, 0.524s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.060s; elapsed, 0.514s, #calls: 1', 'TOTAL : CPU, 4.060s; elapsed, 0.514s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.720s; elapsed, 0.489s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.040s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.550s; elapsed, 0.243s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.180s; elapsed, 0.524s, #calls: 1', 'TOTAL : CPU, 9.700s; elapsed, 1.297s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 6825.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.340s; elapsed, 0.486s', 'individual call time for EIGEN_LDLT: CPU, 3.000s; elapsed, 0.234s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.039s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,057', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0929561', '# - matching time = 0.026655', '# - symmetrization time = 0.00831103', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,057', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.052108', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.3681 MB', '# - factor time = 0.369107', '# - factor nonzeros = 2,171,017', '# - factor memory = 17.3681 MB', '# - total flops = 1.12684e+09, min = 2.48893e+08, max = 8.77944e+08', '# - flop rate = 3.05287 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.18662e+09', '# --------------------------------------------', '# total = 1.18662e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.65852e-11\trel.res = 1.02414e-15\tbw.error = 2.05914e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.019182', '# - total flops = 5.32999e+06, min = 2.07883e+06, max = 3.25116e+06', '# - flop rate = 0.277865 GFlop/s', '# - bytes moved = 43.1462 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.24931 GByte/s', '# - solve arithmetic intensity = 0.123533 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.860s; elapsed, 0.594s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.620s; elapsed, 0.602s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.860s; elapsed, 0.594s, #calls: 1', 'TOTAL : CPU, 8.860s; elapsed, 0.594s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.340s; elapsed, 0.486s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.039s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.000s; elapsed, 0.234s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.620s; elapsed, 0.602s, #calls: 1', 'TOTAL : CPU, 20.430s; elapsed, 1.360s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 3412.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.000s; elapsed, 1.006s', 'individual call time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.236s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.069s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,063', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.090996', '# - matching time = 0.0176051', '# - symmetrization time = 0.00897717', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.019057', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.6027 MB', '# - factor time = 0.380739', '# - factor nonzeros = 2,200,341', '# - factor memory = 17.6027 MB', '# - total flops = 1.17047e+09, min = 1.89709e+08, max = 4.51898e+08', '# - flop rate = 3.07421 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.22692e+09', '# --------------------------------------------', '# total = 1.22692e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.60273e-11\trel.res = 9.89682e-16\tbw.error = 2.4436e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0205159', '# - total flops = 6.73405e+06, min = 1.27554e+06, max = 2.06214e+06', '# - flop rate = 0.328235 GFlop/s', '# - bytes moved = 39.2925 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.91522 GByte/s', '# - solve arithmetic intensity = 0.171383 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.530s; elapsed, 0.546s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.550s; elapsed, 0.546s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.540s; elapsed, 0.543s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.550s; elapsed, 0.551s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.550s; elapsed, 0.546s, #calls: 1', 'TOTAL : CPU, 0.550s; elapsed, 0.546s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.540s; elapsed, 0.543s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.543s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.530s; elapsed, 0.546s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.546s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.000s; elapsed, 1.006s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.069s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.236s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.550s; elapsed, 0.551s, #calls: 1', 'TOTAL : CPU, 1.860s; elapsed, 1.862s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 3412.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.410s; elapsed, 0.719s', 'individual call time for EIGEN_LDLT: CPU, 0.420s; elapsed, 0.240s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.057s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,063', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.104213', '# - matching time = 0.0206001', '# - symmetrization time = 0.0108202', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0242069', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.6027 MB', '# - factor time = 0.340766', '# - factor nonzeros = 2,200,341', '# - factor memory = 17.6027 MB', '# - total flops = 1.17048e+09, min = 1.89709e+08, max = 4.51908e+08', '# - flop rate = 3.43486 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.22692e+09', '# --------------------------------------------', '# total = 1.22692e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.55832e-11\trel.res = 9.62261e-16\tbw.error = 2.45106e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0194521', '# - total flops = 6.73405e+06, min = 1.27554e+06, max = 2.06214e+06', '# - flop rate = 0.346186 GFlop/s', '# - bytes moved = 39.3012 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.02041 GByte/s', '# - solve arithmetic intensity = 0.171345 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.050s; elapsed, 0.533s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.050s; elapsed, 0.529s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.060s; elapsed, 0.535s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.070s; elapsed, 0.539s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.050s; elapsed, 0.533s, #calls: 1', 'TOTAL : CPU, 1.050s; elapsed, 0.533s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.050s; elapsed, 0.529s, #calls: 1', 'TOTAL : CPU, 1.050s; elapsed, 0.529s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.060s; elapsed, 0.535s, #calls: 1', 'TOTAL : CPU, 1.060s; elapsed, 0.535s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.410s; elapsed, 0.719s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.057s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.420s; elapsed, 0.240s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.070s; elapsed, 0.539s, #calls: 1', 'TOTAL : CPU, 3.010s; elapsed, 1.555s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 3412.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.930s; elapsed, 0.502s', 'individual call time for EIGEN_LDLT: CPU, 0.800s; elapsed, 0.226s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.052s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,063', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.10354', '# - matching time = 0.0195551', '# - symmetrization time = 0.0104699', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0286989', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.6027 MB', '# - factor time = 0.296537', '# - factor nonzeros = 2,200,341', '# - factor memory = 17.6027 MB', '# - total flops = 1.17048e+09, min = 1.89709e+08, max = 4.51908e+08', '# - flop rate = 3.94717 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.22692e+09', '# --------------------------------------------', '# total = 1.22692e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.5551e-11\trel.res = 9.60268e-16\tbw.error = 2.45161e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0215471', '# - total flops = 6.73405e+06, min = 1.27554e+06, max = 2.06214e+06', '# - flop rate = 0.312527 GFlop/s', '# - bytes moved = 39.3122 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.82448 GByte/s', '# - solve arithmetic intensity = 0.171297 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.950s; elapsed, 0.494s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.930s; elapsed, 0.493s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.920s; elapsed, 0.489s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.990s; elapsed, 0.497s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.950s; elapsed, 0.494s, #calls: 1', 'TOTAL : CPU, 1.950s; elapsed, 0.494s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.920s; elapsed, 0.489s, #calls: 1', 'TOTAL : CPU, 1.920s; elapsed, 0.489s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.930s; elapsed, 0.493s, #calls: 1', 'TOTAL : CPU, 1.930s; elapsed, 0.493s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.930s; elapsed, 0.502s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.052s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.800s; elapsed, 0.226s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.990s; elapsed, 0.497s, #calls: 1', 'TOTAL : CPU, 4.900s; elapsed, 1.277s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 3412.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.460s; elapsed, 0.454s', 'individual call time for EIGEN_LDLT: CPU, 1.530s; elapsed, 0.236s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.040s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,063', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.104447', '# - matching time = 0.021564', '# - symmetrization time = 0.0102301', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0260451', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.6027 MB', '# - factor time = 0.373722', '# - factor nonzeros = 2,200,341', '# - factor memory = 17.6027 MB', '# - total flops = 1.17049e+09, min = 1.8971e+08, max = 4.51911e+08', '# - flop rate = 3.13198 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.22692e+09', '# --------------------------------------------', '# total = 1.22692e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.57108e-11\trel.res = 9.70137e-16\tbw.error = 2.45357e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0213079', '# - total flops = 6.73405e+06, min = 1.27554e+06, max = 2.06214e+06', '# - flop rate = 0.316035 GFlop/s', '# - bytes moved = 39.335 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.84603 GByte/s', '# - solve arithmetic intensity = 0.171197 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.490s; elapsed, 0.571s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.500s; elapsed, 0.572s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.480s; elapsed, 0.566s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.610s; elapsed, 0.576s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.500s; elapsed, 0.572s, #calls: 1', 'TOTAL : CPU, 4.500s; elapsed, 0.572s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.490s; elapsed, 0.571s, #calls: 1', 'TOTAL : CPU, 4.490s; elapsed, 0.571s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.480s; elapsed, 0.566s, #calls: 1', 'TOTAL : CPU, 4.480s; elapsed, 0.566s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.460s; elapsed, 0.454s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.040s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.530s; elapsed, 0.236s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.610s; elapsed, 0.576s, #calls: 1', 'TOTAL : CPU, 9.840s; elapsed, 1.306s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 1706.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.090s; elapsed, 1.097s', 'individual call time for EIGEN_LDLT: CPU, 0.270s; elapsed, 0.267s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.080s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,063', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.106138', '# - matching time = 0.0202501', '# - symmetrization time = 0.011622', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.022666', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.6027 MB', '# - factor time = 0.368763', '# - factor nonzeros = 2,200,341', '# - factor memory = 17.6027 MB', '# - total flops = 1.17047e+09, min = 908142, max = 2.96546e+08', '# - flop rate = 3.17405 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.22123e+09', '# --------------------------------------------', '# total = 1.22123e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.41958e-11\trel.res = 8.7659e-16\tbw.error = 2.56115e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.022249', '# - total flops = 8.86617e+06, min = 29121, max = 2.0318e+06', '# - flop rate = 0.398498 GFlop/s', '# - bytes moved = 37.7608 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.69719 GByte/s', '# - solve arithmetic intensity = 0.234798 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.550s; elapsed, 0.562s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.520s; elapsed, 0.561s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.540s; elapsed, 0.563s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.560s; elapsed, 0.562s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.530s; elapsed, 0.561s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.530s; elapsed, 0.563s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.540s; elapsed, 0.557s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.560s; elapsed, 0.567s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.550s; elapsed, 0.562s, #calls: 1', 'TOTAL : CPU, 0.550s; elapsed, 0.562s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.530s; elapsed, 0.563s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.563s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.520s; elapsed, 0.561s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.561s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.530s; elapsed, 0.561s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.561s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.540s; elapsed, 0.563s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.563s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.540s; elapsed, 0.557s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.557s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.560s; elapsed, 0.562s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.562s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.090s; elapsed, 1.097s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.080s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.270s; elapsed, 0.267s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.560s; elapsed, 0.567s, #calls: 1', 'TOTAL : CPU, 2.000s; elapsed, 2.010s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 1706.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.290s; elapsed, 0.658s', 'individual call time for EIGEN_LDLT: CPU, 0.440s; elapsed, 0.254s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.064s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = CPP -2', 'CPP -2', 'CPP -2', '4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,063', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************# - nd time = ', '0.099072', '# - matching time = 0.0206611', '# - symmetrization time = 0.0122371', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.026561', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.6027 MB', '# - factor time = 0.33117', '# - factor nonzeros = 2,200,341', '# - factor memory = 17.6027 MB', '# - total flops = 1.17048e+09, min = 908142, max = 2.96551e+08', '# - flop rate = 3.53437 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.22123e+09', '# --------------------------------------------', '# total = 1.22123e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.41384e-11\trel.res = 8.73046e-16\tbw.error = 2.56115e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0206521', '# - total flops = 8.86617e+06, min = 29121, max = 2.0318e+06', '# - flop rate = 0.429312 GFlop/s', '# - bytes moved = 37.7669 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.82872 GByte/s', '# - solve arithmetic intensity = 0.23476 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.000s; elapsed, 0.524s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.980s; elapsed, 0.523s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.020s; elapsed, 0.524s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.030s; elapsed, 0.526s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.010s; elapsed, 0.519s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.960s; elapsed, 0.525s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.990s; elapsed, 0.525s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.040s; elapsed, 0.529s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.000s; elapsed, 0.524s, #calls: 1', 'TOTAL : CPU, 1.000s; elapsed, 0.524s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.030s; elapsed, 0.526s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.526s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.980s; elapsed, 0.523s, #calls: 1', 'TOTAL : CPU, 0.980s; elapsed, 0.523s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.020s; elapsed, 0.524s, #calls: 1', 'TOTAL : CPU, 1.020s; elapsed, 0.524s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.010s; elapsed, 0.519s, #calls: 1', 'TOTAL : CPU, 1.010s; elapsed, 0.519s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.960s; elapsed, 0.525s, #calls: 1', 'TOTAL : CPU, 0.960s; elapsed, 0.525s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.990s; elapsed, 0.525s, #calls: 1', 'TOTAL : CPU, 0.990s; elapsed, 0.525s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.290s; elapsed, 0.658s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.064s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.440s; elapsed, 0.254s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.040s; elapsed, 0.529s, #calls: 1', 'TOTAL : CPU, 2.880s; elapsed, 1.505s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 1706.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.820s; elapsed, 0.477s', 'individual call time for EIGEN_LDLT: CPU, 0.810s; elapsed, 0.262s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.054s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,063', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************# - nd time = ', '0.105998', '# - matching time = 0.021929', '# - symmetrization time = 0.0116129', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0274742', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.6027 MB', '# - factor time = 0.349132', '# - factor nonzeros = 2,200,341', '# - factor memory = 17.6027 MB', '# - total flops = 1.17048e+09, min = 909542, max = 2.96554e+08', '# - flop rate = 3.35256 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.22123e+09', '# --------------------------------------------', '# total = 1.22123e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.38528e-11\trel.res = 8.55407e-16\tbw.error = 2.55981e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.024472', '# - total flops = 8.86617e+06, min = 29121, max = 2.0318e+06', '# - flop rate = 0.362299 GFlop/s', '# - bytes moved = 37.7751 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.54361 GByte/s', '# - solve arithmetic intensity = 0.234709 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.180s; elapsed, 0.555s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.960s; elapsed, 0.557s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.130s; elapsed, 0.556s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.940s; elapsed, 0.556s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.170s; elapsed, 0.551s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.940s; elapsed, 0.557s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.180s; elapsed, 0.556s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.220s; elapsed, 0.562s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.180s; elapsed, 0.555s, #calls: 1', 'TOTAL : CPU, 2.180s; elapsed, 0.555s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.170s; elapsed, 0.551s, #calls: 1', 'TOTAL : CPU, 2.170s; elapsed, 0.551s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.130s; elapsed, 0.556s, #calls: 1', 'TOTAL : CPU, 2.130s; elapsed, 0.556s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.180s; elapsed, 0.556s, #calls: 1', 'TOTAL : CPU, 2.180s; elapsed, 0.556s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.940s; elapsed, 0.557s, #calls: 1', 'TOTAL : CPU, 1.940s; elapsed, 0.557s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.960s; elapsed, 0.557s, #calls: 1', 'TOTAL : CPU, 1.960s; elapsed, 0.557s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.940s; elapsed, 0.556s, #calls: 1', 'TOTAL : CPU, 1.940s; elapsed, 0.556s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.820s; elapsed, 0.477s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.054s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.810s; elapsed, 0.262s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.220s; elapsed, 0.562s, #calls: 1', 'TOTAL : CPU, 5.030s; elapsed, 1.355s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 853.1875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.140s; elapsed, 1.136s', 'individual call time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.244s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.080s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,063', '# - number of levels = 65', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.0906', '# - matching time = 0.0179701', '# - symmetrization time = 0.0105698', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.021188', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.6382 MB', '# - factor time = 0.409723', '# - factor nonzeros = 2,204,769', '# - factor memory = 17.6382 MB', '# - total flops = 1.17308e+09, min = 594238, max = 2.95348e+08', '# - flop rate = 2.8631 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.21323e+09', '# --------------------------------------------', '# total = 1.21323e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.285e-11\trel.res = 7.93488e-16\tbw.error = 1.53534e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.022825', '# - total flops = 1.19466e+07, min = 20012, max = 1.96858e+06', '# - flop rate = 0.523399 GFlop/s', '# - bytes moved = 36.41 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.59518 GByte/s', '# - solve arithmetic intensity = 0.328113 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.540s; elapsed, 0.582s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.580s; elapsed, 0.582s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.530s; elapsed, 0.581s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.560s; elapsed, 0.582s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.560s; elapsed, 0.580s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.520s; elapsed, 0.583s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.540s; elapsed, 0.581sindividual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.510s; elapsed, 0.580s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.560s; elapsed, 0.581s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.530s; elapsed, 0.582s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.560s; elapsed, 0.578s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.520s; elapsed, 0.581sindividual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.540s; elapsed, 0.581s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.540s; elapsed, 0.582s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.580s; elapsed, 0.582s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.590s; elapsed, 0.587s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.540s; elapsed, 0.582s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.582s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.580s; elapsed, 0.582s, #calls: 1', 'TOTAL : CPU, 0.580s; elapsed, 0.582s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.540s; elapsed, 0.581s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.581s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.520s; elapsed, 0.581s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.581s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.560s; elapsed, 0.582s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.582s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.560s; elapsed, 0.580s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.580s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.520s; elapsed, 0.583s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.583s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.560s; elapsed, 0.578s, #calls: 1', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.560s; elapsed, 0.581s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.578s', 'TOTAL : CPU, 0.560s; elapsed, 0.581s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.580s; elapsed, 0.582s, #calls: 1', 'TOTAL : CPU, 0.580s; elapsed, 0.582s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.530s; elapsed, 0.582s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.582s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.540s; elapsed, 0.581s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.581s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.510s; elapsed, 0.580s, #calls: 1', 'TOTAL : CPU, 0.510s; elapsed, 0.580s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.530s; elapsed, 0.581s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.581s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.540s; elapsed, 0.582s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.582s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.140s; elapsed, 1.136s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.080s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.244s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.590s; elapsed, 0.587s, #calls: 1', 'TOTAL : CPU, 2.050s; elapsed, 2.047s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 853.1875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.450s; elapsed, 0.733s', 'individual call time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.247s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.067s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 16 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,063', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.10632', '# - matching time = 0.0212049', '# - symmetrization time = 0.0107839', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0248349', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.6382 MB', '# - factor time = 0.363948', '# - factor nonzeros = 2,204,769', '# - factor memory = 17.6382 MB', '# - total flops = 1.17308e+09, min = 594238, max = 2.95348e+08', '# - flop rate = 3.22322 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.21323e+09', '# --------------------------------------------', '# total = 1.21323e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.2812e-11\trel.res = 7.91139e-16\tbw.error = 1.53398e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0214942', '# - total flops = 1.19466e+07, min = 20012, max = 1.96858e+06', '# - flop rate = 0.555806 GFlop/s', '# - bytes moved = 36.4164 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.69425 GByte/s', '# - solve arithmetic intensity = 0.328055 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.990s; elapsed, 0.561s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.010s; elapsed, 0.560s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 1.090s; elapsed, 0.559s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.010s; elapsed, 0.559s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.010s; elapsed, 0.559s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.020s; elapsed, 0.558s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.000s; elapsed, 0.561s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 1.090s; elapsed, 0.558s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 1.080s; elapsed, 0.559s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 1.100s; elapsed, 0.558s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 1.090s; elapsed, 0.557s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.080s; elapsed, 0.559s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 1.000s; elapsed, 0.558s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 1.090s; elapsed, 0.558s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.010s; elapsed, 0.558s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.120s; elapsed, 0.564s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.990s; elapsed, 0.561s, #calls: 1', 'TOTAL : CPU, 0.990s; elapsed, 0.561s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.010s; elapsed, 0.560s, #calls: 1', 'TOTAL : CPU, 1.010s; elapsed, 0.560s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 1.090s; elapsed, 0.559s, #calls: 1', 'TOTAL : CPU, 1.090s; elapsed, 0.559s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 1.090s; elapsed, 0.558s, #calls: 1', 'TOTAL : CPU, 1.090s; elapsed, 0.558s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 1.000s; elapsed, 0.558s, #calls: 1', 'TOTAL : CPU, 1.000s; elapsed, 0.558s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.000s; elapsed, 0.561s, #calls: 1', 'TOTAL : CPU, 1.000s; elapsed, 0.561s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 1.090s; elapsed, 0.558s, #calls: 1', 'TOTAL : CPU, 1.090s; elapsed, 0.558s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.020s; elapsed, 0.558s, #calls: 1', 'TOTAL : CPU, 1.020s; elapsed, 0.558s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.010s; elapsed, 0.559s, #calls: 1', 'TOTAL : CPU, 1.010s; elapsed, 0.559s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 1.090s; elapsed, 0.557s, #calls: 1', 'TOTAL : CPU, 1.090s; elapsed, 0.557s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 1.100s; elapsed, 0.558s, #calls: 1', 'TOTAL : CPU, 1.100s; elapsed, 0.558s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.080s; elapsed, 0.559s, #calls: 1', 'TOTAL : CPU, 1.080s; elapsed, 0.559s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.010s; elapsed, 0.559s, #calls: 1', 'TOTAL : CPU, 1.010s; elapsed, 0.559s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.010s; elapsed, 0.558s, #calls: 1', 'TOTAL : CPU, 1.010s; elapsed, 0.558s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 1.080s; elapsed, 0.559s, #calls: 1', 'TOTAL : CPU, 1.080s; elapsed, 0.559s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.450s; elapsed, 0.733s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.067s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.247s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.120s; elapsed, 0.564s, #calls: 1', 'TOTAL : CPU, 3.120s; elapsed, 1.611s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Deff-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 13651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.330s; elapsed, 0.333s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.047s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.025s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.330s; elapsed, 0.333s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.025s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.047s, #calls: 1', 'TOTAL : CPU, 0.410s; elapsed, 0.405s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 13651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.490s; elapsed, 0.248s', 'individual call time for EIGEN_LDLT: CPU, 0.090s; elapsed, 0.048s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.022s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.490s; elapsed, 0.248s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.022s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.090s; elapsed, 0.048s, #calls: 1', 'TOTAL : CPU, 0.620s; elapsed, 0.318s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 13651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.820s; elapsed, 0.215s', 'individual call time for EIGEN_LDLT: CPU, 0.190s; elapsed, 0.049s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.021s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.820s; elapsed, 0.215s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.190s; elapsed, 0.049s, #calls: 1', 'TOTAL : CPU, 1.090s; elapsed, 0.284s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 13651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.520s; elapsed, 0.203s', 'individual call time for EIGEN_LDLT: CPU, 0.650s; elapsed, 0.080s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.024s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.520s; elapsed, 0.203s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.024s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.650s; elapsed, 0.080s, #calls: 1', 'TOTAL : CPU, 2.350s; elapsed, 0.307s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 13651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.310s; elapsed, 0.223s', 'individual call time for EIGEN_LDLT: CPU, 0.760s; elapsed, 0.048s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.019s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.310s; elapsed, 0.223s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.019s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.760s; elapsed, 0.048s, #calls: 1', 'TOTAL : CPU, 4.380s; elapsed, 0.290s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 6825.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.350s; elapsed, 0.349s', 'individual call time for EIGEN_LDLT: CPU, 0.040s; elapsed, 0.047s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.025s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.07551', '# - matching time = 0.0125248', '# - symmetrization time = 0.00372195', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0142961', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.19591 MB', '# - factor time = 0.177088', '# - factor nonzeros = 899,489', '# - factor memory = 7.19591 MB', '# - total flops = 2.17387e+08, min = 3.70215e+07, max = 1.80366e+08', '# - flop rate = 1.22757 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.31866e+08', '# --------------------------------------------', '# total = 2.31866e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.24011e-11\trel.res = 8.09365e-16\tbw.error = 4.06316e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0116749', '# - total flops = 2.3471e+06, min = 856409, max = 1.49069e+06', '# - flop rate = 0.201039 GFlop/s', '# - bytes moved = 21.3289 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.8269 GByte/s', '# - solve arithmetic intensity = 0.110043 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.300s; elapsed, 0.302s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.310s; elapsed, 0.306s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.300s; elapsed, 0.302s, #calls: 1', 'TOTAL : CPU, 0.300s; elapsed, 0.302s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.350s; elapsed, 0.349s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.025s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.040s; elapsed, 0.047s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.310s; elapsed, 0.306s, #calls: 1', 'TOTAL : CPU, 0.730s; elapsed, 0.727s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 6825.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.530s; elapsed, 0.276s', 'individual call time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.055s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.023s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0913858', '# - matching time = 0.012563', '# - symmetrization time = 0.00397682', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0184171', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.19591 MB', '# - factor time = 0.120989', '# - factor nonzeros = 899,489', '# - factor memory = 7.19591 MB', '# - total flops = 2.17416e+08, min = 3.70215e+07, max = 1.80395e+08', '# - flop rate = 1.79699 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.31866e+08', '# --------------------------------------------', '# total = 2.31866e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.18614e-11\trel.res = 7.74142e-16\tbw.error = 3.26284e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.00985694', '# - total flops = 2.34774e+06, min = 856409, max = 1.49133e+06', '# - flop rate = 0.238182 GFlop/s', '# - bytes moved = 21.3528 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.16627 GByte/s', '# - solve arithmetic intensity = 0.10995 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.530s; elapsed, 0.268s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.550s; elapsed, 0.276s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.530s; elapsed, 0.268s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.268s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.530s; elapsed, 0.276s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.055s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.550s; elapsed, 0.276s, #calls: 1', 'TOTAL : CPU, 1.230s; elapsed, 0.630s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 6825.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.790s; elapsed, 0.208s', 'individual call time for EIGEN_LDLT: CPU, 0.210s; elapsed, 0.052s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.021s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.081831', '# - matching time = 0.0115721', '# - symmetrization time = 0.0036571', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0159321', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.19591 MB', '# - factor time = 0.099793', '# - factor nonzeros = 899,489', '# - factor memory = 7.19591 MB', '# - total flops = 2.17437e+08, min = 3.70215e+07, max = 1.80416e+08', '# - flop rate = 2.17889 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.31866e+08', '# --------------------------------------------', '# total = 2.31866e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.18802e-11\trel.res = 7.75366e-16\tbw.error = 3.26284e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0115058', '# - total flops = 2.34774e+06, min = 856409, max = 1.49133e+06', '# - flop rate = 0.204048 GFlop/s', '# - bytes moved = 21.364 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.8568 GByte/s', '# - solve arithmetic intensity = 0.109892 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.910s; elapsed, 0.235s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.950s; elapsed, 0.240s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.910s; elapsed, 0.235s, #calls: 1', 'TOTAL : CPU, 0.910s; elapsed, 0.235s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.790s; elapsed, 0.208s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.210s; elapsed, 0.052s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.950s; elapsed, 0.240s, #calls: 1', 'TOTAL : CPU, 2.040s; elapsed, 0.522s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 6825.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.580s; elapsed, 0.212s', 'individual call time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.051s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.019s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0818939', '# - matching time = 0.0117681', '# - symmetrization time = 0.00494599', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0197721', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.19591 MB', '# - factor time = 0.098726', '# - factor nonzeros = 899,489', '# - factor memory = 7.19591 MB', '# - total flops = 2.17439e+08, min = 3.70231e+07, max = 1.80416e+08', '# - flop rate = 2.20245 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.31866e+08', '# --------------------------------------------', '# total = 2.31866e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.16124e-11\trel.res = 7.57889e-16\tbw.error = 3.26284e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0093739', '# - total flops = 2.34774e+06, min = 856409, max = 1.49133e+06', '# - flop rate = 0.250455 GFlop/s', '# - bytes moved = 21.3673 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.27944 GByte/s', '# - solve arithmetic intensity = 0.109876 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.870s; elapsed, 0.236s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.930s; elapsed, 0.242s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.870s; elapsed, 0.236s, #calls: 1', 'TOTAL : CPU, 1.870s; elapsed, 0.236s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.580s; elapsed, 0.212s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.019s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.051s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.930s; elapsed, 0.242s, #calls: 1', 'TOTAL : CPU, 4.070s; elapsed, 0.525s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 6825.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.170s; elapsed, 0.216s', 'individual call time for EIGEN_LDLT: CPU, 0.890s; elapsed, 0.056s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.340s; elapsed, 0.021s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 68', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.0917859', '# - matching time = 0.0135388', '# - symmetrization time = 0.00417686', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.020745', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.19591 MB', '# - factor time = 0.117176', '# - factor nonzeros = 899,489', '# - factor memory = 7.19591 MB', '# - total flops = 2.17444e+08, min = 3.70246e+07, max = 1.8042e+08', '# - flop rate = 1.85571 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.31866e+08', '# --------------------------------------------', '# total = 2.31866e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.16336e-11\trel.res = 7.59275e-16\tbw.error = 3.26284e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0136292', '# - total flops = 2.34774e+06, min = 856409, max = 1.49133e+06', '# - flop rate = 0.172258 GFlop/s', '# - bytes moved = 21.3718 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.56809 GByte/s', '# - solve arithmetic intensity = 0.109852 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.200s; elapsed, 0.272s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.410s; elapsed, 0.276s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.200s; elapsed, 0.272s, #calls: 1', 'TOTAL : CPU, 4.200s; elapsed, 0.272s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.170s; elapsed, 0.216s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.340s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.890s; elapsed, 0.056s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.410s; elapsed, 0.276s, #calls: 1', 'TOTAL : CPU, 8.810s; elapsed, 0.569s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 3412.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.410s; elapsed, 0.410s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.054s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.029s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 67', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.0900881', '# - matching time = 0.0127611', '# - symmetrization time = 0.00457215', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0137699', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.05593 MB', '# - factor time = 0.164755', '# - factor nonzeros = 881,991', '# - factor memory = 7.05593 MB', '# - total flops = 2.06216e+08, min = 3.70215e+07, max = 8.34426e+07', '# - flop rate = 1.25165 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.19939e+08', '# --------------------------------------------', '# total = 2.19939e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.13219e-11\trel.res = 7.38927e-16\tbw.error = 1.08941e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0145369', '# - total flops = 2.70089e+06, min = 511448, max = 839344', '# - flop rate = 0.185796 GFlop/s', '# - bytes moved = 19.8444 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.36511 GByte/s', '# - solve arithmetic intensity = 0.136103 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.280s; elapsed, 0.307s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.300s; elapsed, 0.305s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.300s; elapsed, 0.307s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.300s; elapsed, 0.310s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.280s; elapsed, 0.307s, #calls: 1', 'TOTAL : CPU, 0.280s; elapsed, 0.307s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.300s; elapsed, 0.305s, #calls: 1', 'TOTAL : CPU, 0.300s; elapsed, 0.305s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.300s; elapsed, 0.307s, #calls: 1', 'TOTAL : CPU, 0.300s; elapsed, 0.307s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.410s; elapsed, 0.410s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.029s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.054s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.300s; elapsed, 0.310s, #calls: 1', 'TOTAL : CPU, 0.790s; elapsed, 0.803s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 3412.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.610s; elapsed, 0.311s', 'individual call time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.056s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.024s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.094619', '# - matching time = 0.0131378', '# - symmetrization time = 0.00457406', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.014739', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.05593 MB', '# - factor time = 0.123206', '# - factor nonzeros = 881,991', '# - factor memory = 7.05593 MB', '# - total flops = 2.06223e+08, min = 3.70215e+07, max = 8.34491e+07', '# - flop rate = 1.6738 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.19939e+08', '# --------------------------------------------', '# total = 2.19939e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.1215e-11\trel.res = 7.31952e-16\tbw.error = 1.10174e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.012388', '# - total flops = 2.70089e+06, min = 511448, max = 839344', '# - flop rate = 0.218025 GFlop/s', '# - bytes moved = 19.8514 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.60247 GByte/s', '# - solve arithmetic intensity = 0.136055 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.540s; elapsed, 0.273s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.540s; elapsed, 0.275s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.530s; elapsed, 0.272s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.550s; elapsed, 0.277s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.540s; elapsed, 0.275s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.275s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.540s; elapsed, 0.273s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.273s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.530s; elapsed, 0.272s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.272s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.610s; elapsed, 0.311s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.024s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.056s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.550s; elapsed, 0.277s, #calls: 1', 'TOTAL : CPU, 1.320s; elapsed, 0.668s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 3412.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.990s; elapsed, 0.259s', 'individual call time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.058s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.023s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', 'CPP -2', '4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0952041', '# - matching time = 0.0139439', '# - symmetrization time = 0.00526214', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0158701', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.05593 MB', '# - factor time = 0.118561', '# - factor nonzeros = 881,991', '# - factor memory = 7.05593 MB', '# - total flops = 2.06223e+08, min = 3.70215e+07, max = 8.34491e+07', '# - flop rate = 1.73938 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.19939e+08', '# --------------------------------------------', '# total = 2.19939e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.12111e-11\trel.res = 7.317e-16\tbw.error = 1.10174e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0124111', '# - total flops = 2.70089e+06, min = 511448, max = 839344', '# - flop rate = 0.217618 GFlop/s', '# - bytes moved = 19.8548 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.59976 GByte/s', '# - solve arithmetic intensity = 0.136032 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.070s; elapsed, 0.273s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.070s; elapsed, 0.274sindividual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.090s; elapsed, 0.274s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.110s; elapsed, 0.279s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.070s; elapsed, 0.273s, #calls: 1', 'TOTAL : CPU, 1.070s; elapsed, 0.273s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.070s; elapsed, 0.274s, #calls: 1', 'TOTAL : CPU, 1.070s; elapsed, 0.274s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.090s; elapsed, 0.274s, #calls: 1', 'TOTAL : CPU, 1.090s; elapsed, 0.274s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.990s; elapsed, 0.259s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.058s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.110s; elapsed, 0.279s, #calls: 1', 'TOTAL : CPU, 2.430s; elapsed, 0.620s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 3412.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.750s; elapsed, 0.232s', 'individual call time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.054s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.020s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.092514', '# - matching time = 0.014462', '# - symmetrization time = 0.0052669', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0163271', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.05593 MB', '# - factor time = 0.110125', '# - factor nonzeros = 881,991', '# - factor memory = 7.05593 MB', '# - total flops = 2.06229e+08, min = 3.70231e+07, max = 8.34514e+07', '# - flop rate = 1.87268 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.19939e+08', '# --------------------------------------------', '# total = 2.19939e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.13871e-11\trel.res = 7.43184e-16\tbw.error = 1.10247e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.01301', '# - total flops = 2.70089e+06, min = 511448, max = 839344', '# - flop rate = 0.2076 GFlop/s', '# - bytes moved = 19.8621 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.52668 GByte/s', '# - solve arithmetic intensity = 0.135982 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.060s; elapsed, 0.263s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.070s; elapsed, 0.264s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.080s; elapsed, 0.264s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.140s; elapsed, 0.269s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.060s; elapsed, 0.263s, #calls: 1', 'TOTAL : CPU, 2.060s; elapsed, 0.263s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.080s; elapsed, 0.264s, #calls: 1', 'TOTAL : CPU, 2.080s; elapsed, 0.264s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.070s; elapsed, 0.264s, #calls: 1', 'TOTAL : CPU, 2.070s; elapsed, 0.264s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.750s; elapsed, 0.232s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.020s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.054s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.140s; elapsed, 0.269s, #calls: 1', 'TOTAL : CPU, 4.480s; elapsed, 0.575s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 1706.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.430s; elapsed, 0.420s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.057s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.030s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.090615', '# - matching time = 0.0129662', '# - symmetrization time = 0.00462198', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0124819', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.05593 MB', '# - factor time = 0.176365', '# - factor nonzeros = 881,991', '# - factor memory = 7.05593 MB', '# - total flops = 2.06216e+08, min = 225576, max = 6.3946e+07', '# - flop rate = 1.16926 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.18436e+08', '# --------------------------------------------', '# total = 2.18436e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.06023e-11\trel.res = 6.91968e-16\tbw.error = 1.08348e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.013438', '# - total flops = 3.41102e+06, min = 19352, max = 819391', '# - flop rate = 0.253834 GFlop/s', '# - bytes moved = 19.4123 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.44458 GByte/s', '# - solve arithmetic intensity = 0.175715 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.280s; elapsed, 0.316s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.310s; elapsed, 0.315s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.300s; elapsed, 0.316s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.290s; elapsed, 0.315s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.300s; elapsed, 0.314s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.290s; elapsed, 0.315s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.280s; elapsed, 0.315s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.310s; elapsed, 0.317s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.280s; elapsed, 0.316s, #calls: 1', 'TOTAL : CPU, 0.280s; elapsed, 0.316s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.290s; elapsed, 0.315s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.315s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.310s; elapsed, 0.315s, #calls: 1', 'TOTAL : CPU, 0.310s; elapsed, 0.315s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.280s; elapsed, 0.315s, #calls: 1', 'TOTAL : CPU, 0.280s; elapsed, 0.315s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.300s; elapsed, 0.314s, #calls: 1', 'TOTAL : CPU, 0.300s; elapsed, 0.314s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.300s; elapsed, 0.316s, #calls: 1', 'TOTAL : CPU, 0.300s; elapsed, 0.316s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.290s; elapsed, 0.315s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.315s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.430s; elapsed, 0.420s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.030s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.057s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.310s; elapsed, 0.317s, #calls: 1', 'TOTAL : CPU, 0.820s; elapsed, 0.824s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 1706.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.510s; elapsed, 0.262s', 'individual call time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.058s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.025s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.096503', '# - matching time = 0.0136571', '# - symmetrization time = 0.00595999', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0173061', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.05593 MB', '# - factor time = 0.11608', '# - factor nonzeros = 881,991', '# - factor memory = 7.05593 MB', '# - total flops = 2.0622e+08, min = 225576, max = 6.39502e+07', '# - flop rate = 1.77654 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.18436e+08', '# --------------------------------------------', '# total = 2.18436e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.05418e-11\trel.res = 6.88014e-16\tbw.error = 1.08348e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0143149', '# - total flops = 3.41102e+06, min = 19352, max = 819391', '# - flop rate = 0.238285 GFlop/s', '# - bytes moved = 19.4183 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.35651 GByte/s', '# - solve arithmetic intensity = 0.175661 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.540s; elapsed, 0.274s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.530s; elapsed, 0.274s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.530s; elapsed, 0.274s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.530s; elapsed, 0.272s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.540s; elapsed, 0.274s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.530s; elapsed, 0.274s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.540s; elapsed, 0.275s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.540s; elapsed, 0.276s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.540s; elapsed, 0.274s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.274s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.540s; elapsed, 0.274s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.274s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.530s; elapsed, 0.272s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.272s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.530s; elapsed, 0.274s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.274s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.530s; elapsed, 0.274s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.274s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.540s; elapsed, 0.275s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.275s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.530s; elapsed, 0.274s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.274s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.510s; elapsed, 0.262s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.025s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.058s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.540s; elapsed, 0.276s, #calls: 1', 'TOTAL : CPU, 1.190s; elapsed, 0.620s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 1706.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.950s; elapsed, 0.251s', 'individual call time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.056s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.021s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.094913', '# - matching time = 0.0139389', '# - symmetrization time = 0.00638509', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0184851', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.05593 MB', '# - factor time = 0.111166', '# - factor nonzeros = 881,991', '# - factor memory = 7.05593 MB', '# - total flops = 2.06227e+08, min = 227131, max = 6.39525e+07', '# - flop rate = 1.85513 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.18436e+08', '# --------------------------------------------', '# total = 2.18436e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.07523e-11\trel.res = 7.01753e-16\tbw.error = 1.08377e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.015413', '# - total flops = 3.41102e+06, min = 19352, max = 819391', '# - flop rate = 0.221308 GFlop/s', '# - bytes moved = 19.4256 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.26034 GByte/s', '# - solve arithmetic intensity = 0.175594 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.070s; elapsed, 0.271s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.080s; elapsed, 0.272sindividual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.040s; elapsed, 0.274s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.050s; elapsed, 0.273s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.060s; elapsed, 0.273s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.060s; elapsed, 0.274s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.060s; elapsed, 0.274s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.100s; elapsed, 0.279s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.060s; elapsed, 0.274s, #calls: 1', 'TOTAL : CPU, 1.060s; elapsed, 0.274s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.070s; elapsed, 0.271s, #calls: 1', 'TOTAL : CPU, 1.070s; elapsed, 0.271s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.050s; elapsed, 0.273s, #calls: 1', 'TOTAL : CPU, 1.050s; elapsed, 0.273s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.060s; elapsed, 0.273s, #calls: 1', 'TOTAL : CPU, 1.060s; elapsed, 0.273s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.060s; elapsed, 0.274s, #calls: 1', 'TOTAL : CPU, 1.060s; elapsed, 0.274s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.040s; elapsed, 0.274s, #calls: 1', 'TOTAL : CPU, 1.040s; elapsed, 0.274s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.080s; elapsed, 0.272s, #calls: 1', 'TOTAL : CPU, 1.080s; elapsed, 0.272s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.950s; elapsed, 0.251s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.056s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.100s; elapsed, 0.279s, #calls: 1', 'TOTAL : CPU, 2.360s; elapsed, 0.608s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 853.1875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.420s; elapsed, 0.418s', 'individual call time for EIGEN_LDLT: CPU, 0.060s; elapsed, 0.056s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.030s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0943608', '# - matching time = 0.0138721', '# - symmetrization time = 0.00418782', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0163059', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.05593 MB', '# - factor time = 0.160943', '# - factor nonzeros = 881,991', '# - factor memory = 7.05593 MB', '# - total flops = 2.06216e+08, min = 172869, max = 4.26352e+07', '# - flop rate = 1.2813 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.15638e+08', '# --------------------------------------------', '# total = 2.15638e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.75506e-12\trel.res = 6.36669e-16\tbw.error = 1.09812e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0164499', '# - total flops = 4.7536e+06, min = 13772, max = 794351', '# - flop rate = 0.288974 GFlop/s', '# - bytes moved = 18.5117 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.12534 GByte/s', '# - solve arithmetic intensity = 0.256789 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.300s; elapsed, 0.318s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.310s; elapsed, 0.320s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.220s; elapsed, 0.317s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.200s; elapsed, 0.318s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.260s; elapsed, 0.321s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.270s; elapsed, 0.319s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.220s; elapsed, 0.319s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.250s; elapsed, 0.319s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.270s; elapsed, 0.319s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.280s; elapsed, 0.319s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.280s; elapsed, 0.318s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.290s; elapsed, 0.318s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.310s; elapsed, 0.318s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.290s; elapsed, 0.316s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.290s; elapsed, 0.319s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.320s; elapsed, 0.325s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.200s; elapsed, 0.318s, #calls: 1', 'TOTAL : CPU, 0.200s; elapsed, 0.318s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.270s; elapsed, 0.319s, #calls: 1', 'TOTAL : CPU, 0.270s; elapsed, 0.319s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.270s; elapsed, 0.319s, #calls: 1', 'TOTAL : CPU, 0.270s; elapsed, 0.319s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.310s; elapsed, 0.320s, #calls: 1', 'TOTAL : CPU, 0.310s; elapsed, 0.320s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.220s; elapsed, 0.317s, #calls: 1', 'TOTAL : CPU, 0.220s; elapsed, 0.317s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.250s; elapsed, 0.319s, #calls: 1', 'TOTAL : CPU, 0.250s; elapsed, 0.319s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.300s; elapsed, 0.318s, #calls: 1', 'TOTAL : CPU, 0.300s; elapsed, 0.318s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.280s; elapsed, 0.318s, #calls: 1', 'TOTAL : CPU, 0.280s; elapsed, 0.318s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.290s; elapsed, 0.318s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.318s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.260s; elapsed, 0.321s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.321s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.290s; elapsed, 0.316s, #calls: 1', 'Exiting profiler', 'TOTAL : CPU, 0.290s; elapsed, 0.316s', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.310s; elapsed, 0.318s, #calls: 1', 'TOTAL : CPU, 0.310s; elapsed, 0.318s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.220s; elapsed, 0.319s, #calls: 1', 'TOTAL : CPU, 0.220s; elapsed, 0.319s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.290s; elapsed, 0.319s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.319s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.280s; elapsed, 0.319s, #calls: 1', 'TOTAL : CPU, 0.280s; elapsed, 0.319s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.420s; elapsed, 0.418s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.030s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.060s; elapsed, 0.056s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.320s; elapsed, 0.325s, #calls: 1', 'TOTAL : CPU, 0.830s; elapsed, 0.829s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 853.1875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.540s; elapsed, 0.278s', 'individual call time for EIGEN_LDLT: CPU, 0.120s; elapsed, 0.056s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.026s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0963531', '# - matching time = 0.013932', '# - symmetrization time = 0.0042038', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0167351', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.05593 MB', '# - factor time = 0.13779', '# - factor nonzeros = 881,991', '# - factor memory = 7.05593 MB', '# - total flops = 2.06219e+08, min = 172869, max = 4.26352e+07', '# - flop rate = 1.49662 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.15638e+08', '# --------------------------------------------', '# total = 2.15638e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.7554e-12\trel.res = 6.36691e-16\tbw.error = 1.09812e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.01367', '# - total flops = 4.7536e+06, min = 13772, max = 794351', '# - flop rate = 0.34774 GFlop/s', '# - bytes moved = 18.5136 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.35432 GByte/s', '# - solve arithmetic intensity = 0.256763 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.590s; elapsed, 0.298s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.560s; elapsed, 0.300s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.570s; elapsed, 0.301s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.560s; elapsed, 0.300s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.590s; elapsed, 0.301s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.580s; elapsed, 0.300s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.570s; elapsed, 0.299s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.570s; elapsed, 0.300s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.570s; elapsed, 0.300s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.570s; elapsed, 0.300s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.580s; elapsed, 0.299s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.560s; elapsed, 0.299s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.560s; elapsed, 0.299s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.570s; elapsed, 0.300s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.590s; elapsed, 0.302s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.600s; elapsed, 0.305s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.560s; elapsed, 0.300s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.300s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.590s; elapsed, 0.298s, #calls: 1', 'TOTAL : CPU, 0.590s; elapsed, 0.298s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.570s; elapsed, 0.301s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.301s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.580s; elapsed, 0.299s, #calls: 1', 'TOTAL : CPU, 0.580s; elapsed, 0.299s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.570s; elapsed, 0.300s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.300s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.590s; elapsed, 0.301s, #calls: 1', 'TOTAL : CPU, 0.590s; elapsed, 0.301s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.560s; elapsed, 0.299s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.299s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.570s; elapsed, 0.299s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.299s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.560s; elapsed, 0.300s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.300s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.580s; elapsed, 0.300s, #calls: 1', 'TOTAL : CPU, 0.580s; elapsed, 0.300s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.590s; elapsed, 0.302s, #calls: 1', 'Exiting profilerTOTAL : CPU, 0.590s; elapsed, 0.302s', '', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.570s; elapsed, 0.300s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.300s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.570s; elapsed, 0.300s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.300s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.570s; elapsed, 0.300s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.300s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.560s; elapsed, 0.299s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.299s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.540s; elapsed, 0.278s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.026s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.120s; elapsed, 0.056s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.600s; elapsed, 0.305s, #calls: 1', 'TOTAL : CPU, 1.310s; elapsed, 0.665s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13151 13151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.340s; elapsed, 0.335s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.047s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.024s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.340s; elapsed, 0.335s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.024s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.047s, #calls: 1', 'TOTAL : CPU, 0.410s; elapsed, 0.406s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13151 13151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.480s; elapsed, 0.247s', 'individual call time for EIGEN_LDLT: CPU, 0.090s; elapsed, 0.047s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.022s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.480s; elapsed, 0.247s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.022s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.090s; elapsed, 0.047s, #calls: 1', 'TOTAL : CPU, 0.610s; elapsed, 0.315s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13151 13151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.790s; elapsed, 0.207s', 'individual call time for EIGEN_LDLT: CPU, 0.190s; elapsed, 0.048s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.021s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.790s; elapsed, 0.207s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.190s; elapsed, 0.048s, #calls: 1', 'TOTAL : CPU, 1.070s; elapsed, 0.276s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13151 13151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.650s; elapsed, 0.220s', 'individual call time for EIGEN_LDLT: CPU, 0.470s; elapsed, 0.057s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.021s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.650s; elapsed, 0.220s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.470s; elapsed, 0.057s, #calls: 1', 'TOTAL : CPU, 2.280s; elapsed, 0.299s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13151 13151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.520s; elapsed, 0.236s', 'individual call time for EIGEN_LDLT: CPU, 1.060s; elapsed, 0.066s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.330s; elapsed, 0.020s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.520s; elapsed, 0.236s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.330s; elapsed, 0.020s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.060s; elapsed, 0.066s, #calls: 1', 'TOTAL : CPU, 4.910s; elapsed, 0.322s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13151 6575.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.360s; elapsed, 0.351s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.052s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.027s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', '1 OpenMP thread', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,151', '# - number of nonzeros = 71,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# - nd time = # ******************************************************************', '0.0779922', '# - matching time = 0.010932', '# - symmetrization time = 0.003438', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0142422', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.48993 MB', '# - factor time = 0.18613', '# - factor nonzeros = 936,241', '# - factor memory = 7.48993 MB', '# - total flops = 2.53572e+08, min = 4.10212e+07, max = 2.1255e+08', '# - flop rate = 1.36234 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.68407e+08', '# --------------------------------------------', '# total = 2.68407e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.25063e-11\trel.res = 8.16234e-16\tbw.error = 1.35875e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0109029', '# - total flops = 2.41526e+06, min = 879234, max = 1.53603e+06', '# - flop rate = 0.221525 GFlop/s', '# - bytes moved = 21.4806 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.97017 GByte/s', '# - solve arithmetic intensity = 0.11244 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.300s; elapsed, 0.310s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.320s; elapsed, 0.314s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.300s; elapsed, 0.310s, #calls: 1', 'TOTAL : CPU, 0.300s; elapsed, 0.310s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.360s; elapsed, 0.351s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.027s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.052s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.320s; elapsed, 0.314s, #calls: 1', 'TOTAL : CPU, 0.750s; elapsed, 0.744s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13151 6575.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.500s; elapsed, 0.256s', 'individual call time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.049s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.023s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,151', '# - number of nonzeros = 71,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0792282', '# - matching time = 0.0108502', '# - symmetrization time = 0.00356913', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.015748', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.48993 MB', '# - factor time = 0.119972', '# - factor nonzeros = 936,241', '# - factor memory = 7.48993 MB', '# - total flops = 2.53597e+08, min = 4.10212e+07, max = 2.12576e+08', '# - flop rate = 2.1138 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.68407e+08', '# --------------------------------------------', '# total = 2.68407e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.21473e-11\trel.res = 7.92799e-16\tbw.error = 1.22195e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.00921702', '# - total flops = 2.41526e+06, min = 879234, max = 1.53603e+06', '# - flop rate = 0.262044 GFlop/s', '# - bytes moved = 21.4941 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.332 GByte/s', '# - solve arithmetic intensity = 0.112368 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.480s; elapsed, 0.248s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.510s; elapsed, 0.251s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.480s; elapsed, 0.248s, #calls: 1', 'TOTAL : CPU, 0.480s; elapsed, 0.248s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.500s; elapsed, 0.256s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.049s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.510s; elapsed, 0.251s, #calls: 1', 'TOTAL : CPU, 1.150s; elapsed, 0.580s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13151 6575.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.840s; elapsed, 0.221s', 'individual call time for EIGEN_LDLT: CPU, 0.210s; elapsed, 0.051s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.022s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,151', '# - number of nonzeros = 71,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0806141', '# - matching time = 0.0107949', '# - symmetrization time = 0.0034461', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0149159', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.48993 MB', '# - factor time = 0.0945809', '# - factor nonzeros = 936,241', '# - factor memory = 7.48993 MB', '# - total flops = 2.53602e+08, min = 4.10212e+07, max = 2.1258e+08', '# - flop rate = 2.68132 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.68407e+08', '# --------------------------------------------', '# total = 2.68407e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.19812e-11\trel.res = 7.81957e-16\tbw.error = 1.22195e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.00903296', '# - total flops = 2.41526e+06, min = 879234, max = 1.53603e+06', '# - flop rate = 0.267383 GFlop/s', '# - bytes moved = 21.5116 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.38146 GByte/s', '# - solve arithmetic intensity = 0.112277 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.860s; elapsed, 0.223s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.900s; elapsed, 0.226s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.860s; elapsed, 0.223s, #calls: 1', 'TOTAL : CPU, 0.860s; elapsed, 0.223s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.840s; elapsed, 0.221s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.022s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.210s; elapsed, 0.051s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.900s; elapsed, 0.226s, #calls: 1', 'TOTAL : CPU, 2.040s; elapsed, 0.521s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13151 6575.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.630s; elapsed, 0.218s', 'individual call time for EIGEN_LDLT: CPU, 0.440s; elapsed, 0.055s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.021s', 'CPP -2', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,151', '# - number of nonzeros = 71,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0907271', '# - matching time = 0.0138259', '# - symmetrization time = 0.00546503', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.016654', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.48993 MB', '# - factor time = 0.102614', '# - factor nonzeros = 936,241', '# - factor memory = 7.48993 MB', '# - total flops = 2.53606e+08, min = 4.10226e+07, max = 2.12584e+08', '# - flop rate = 2.47146 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.68407e+08', '# --------------------------------------------', '# total = 2.68407e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.15724e-11\trel.res = 7.55278e-16\tbw.error = 1.22195e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0105209', '# - total flops = 2.41526e+06, min = 879234, max = 1.53603e+06', '# - flop rate = 0.229567 GFlop/s', '# - bytes moved = 21.5265 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.04607 GByte/s', '# - solve arithmetic intensity = 0.112199 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.980s; elapsed, 0.253s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.080s; elapsed, 0.261s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.980s; elapsed, 0.253s, #calls: 1', 'TOTAL : CPU, 1.980s; elapsed, 0.253s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.630s; elapsed, 0.218s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.440s; elapsed, 0.055s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.080s; elapsed, 0.261s, #calls: 1', 'TOTAL : CPU, 4.330s; elapsed, 0.555s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13151 6575.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.590s; elapsed, 0.242s', 'individual call time for EIGEN_LDLT: CPU, 1.080s; elapsed, 0.068s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.370s; elapsed, 0.023s', 'CPP -2CPP -2', '', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,151', '# - number of nonzeros = 71,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0914981', '# - matching time = 0.0147059', '# - symmetrization time = 0.00418305', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0185521', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.48993 MB', '# - factor time = 0.108894', '# - factor nonzeros = 936,241', '# - factor memory = 7.48993 MB', '# - total flops = 2.5361e+08, min = 4.10239e+07, max = 2.12586e+08', '# - flop rate = 2.32896 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.68407e+08', '# --------------------------------------------', '# total = 2.68407e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.15387e-11\trel.res = 7.5308e-16\tbw.error = 1.07124e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0108631', '# - total flops = 2.41526e+06, min = 879234, max = 1.53603e+06', '# - flop rate = 0.222337 GFlop/s', '# - bytes moved = 21.5398 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.98285 GByte/s', '# - solve arithmetic intensity = 0.11213 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.030s; elapsed, 0.260s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.200s; elapsed, 0.263s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.030s; elapsed, 0.260s, #calls: 1', 'TOTAL : CPU, 4.030s; elapsed, 0.260s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.590s; elapsed, 0.242s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.370s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.080s; elapsed, 0.068s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.200s; elapsed, 0.263s, #calls: 1', 'TOTAL : CPU, 9.240s; elapsed, 0.595s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13151 3287.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.380s; elapsed, 0.380s', 'individual call time for EIGEN_LDLT: CPU, 0.060s; elapsed, 0.056s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.029s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', 'CPP -2', 'CPP -2', '1 OpenMP thread', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,151', '# - number of nonzeros = 71,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.095643', '# - matching time = 0.0124459', '# - symmetrization time = 0.00487208', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.015172', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.12607 MB', '# - factor time = 0.128586', '# - factor nonzeros = 890,759', '# - factor memory = 7.12607 MB', '# - total flops = 2.22417e+08, min = 3.03195e+07, max = 1.02905e+08', '# - flop rate = 1.72971 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.36424e+08', '# --------------------------------------------', '# total = 2.36424e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.07646e-11\trel.res = 7.02559e-16\tbw.error = 9.96274e-16', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.013422', '# - total flops = 2.71052e+06, min = 467563, max = 853409', '# - flop rate = 0.201946 GFlop/s', '# - bytes moved = 19.9615 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.48722 GByte/s', '# - solve arithmetic intensity = 0.135788 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.260s; elapsed, 0.279s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.270s; elapsed, 0.279s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.260s; elapsed, 0.277s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.280s; elapsed, 0.281s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.260s; elapsed, 0.277s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.277s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.260s; elapsed, 0.279s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.279s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.270s; elapsed, 0.279s, #calls: 1', 'TOTAL : CPU, 0.270s; elapsed, 0.279s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.380s; elapsed, 0.380s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.029s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.060s; elapsed, 0.056s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.280s; elapsed, 0.281s, #calls: 1', 'TOTAL : CPU, 0.750s; elapsed, 0.747s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13151 3287.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.550s; elapsed, 0.285s', 'individual call time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.053s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.023s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,151', '# - number of nonzeros = 71,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,# - nd time = ', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.088805', '# - matching time = 0.0126319', '# - symmetrization time = 0.00530505', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.015986', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.12607 MB', '# - factor time = 0.125342', '# - factor nonzeros = 890,759', '# - factor memory = 7.12607 MB', '# - total flops = 2.22424e+08, min = 3.03195e+07, max = 1.02912e+08', '# - flop rate = 1.77453 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.36424e+08', '# --------------------------------------------', '# total = 2.36424e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.05664e-11\trel.res = 6.89622e-16\tbw.error = 1.00389e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.014498', '# - total flops = 2.71052e+06, min = 467563, max = 853409', '# - flop rate = 0.186959 GFlop/s', '# - bytes moved = 19.9654 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.37711 GByte/s', '# - solve arithmetic intensity = 0.135761 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.540s; elapsed, 0.274s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.540s; elapsed, 0.275s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.550s; elapsed, 0.273s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.540s; elapsed, 0.279s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.540s; elapsed, 0.275s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.275s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.540s; elapsed, 0.274s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.274s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.550s; elapsed, 0.273s, #calls: 1', 'TOTAL : CPU, 0.550s; elapsed, 0.273s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.550s; elapsed, 0.285s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.053s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.540s; elapsed, 0.279s, #calls: 1', 'TOTAL : CPU, 1.240s; elapsed, 0.640s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13151 3287.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.850s; elapsed, 0.224s', 'individual call time for EIGEN_LDLT: CPU, 0.220s; elapsed, 0.056s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.023s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', 'CPP -2', 'CPP -2', 'CPP -2', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,151', '# - number of nonzeros = 71,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0922999', '# - matching time = 0.0134981', '# - symmetrization time = 0.00544405', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0157189', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.12607 MB', '# - factor time = 0.159256', '# - factor nonzeros = 890,759', '# - factor memory = 7.12607 MB', '# - total flops = 2.22424e+08, min = 3.03195e+07, max = 1.02912e+08', '# - flop rate = 1.39665 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.36424e+08', '# --------------------------------------------', '# total = 2.36424e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.05917e-11\trel.res = 6.91271e-16\tbw.error = 1.00389e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.011466', '# - total flops = 2.71052e+06, min = 467563, max = 853409', '# - flop rate = 0.236396 GFlop/s', '# - bytes moved = 19.9702 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.74168 GByte/s', '# - solve arithmetic intensity = 0.135728 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.200s; elapsed, 0.309s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.200s; elapsed, 0.308s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.200s; elapsed, 0.306s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.230s; elapsed, 0.309s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.200s; elapsed, 0.309s, #calls: 1', 'TOTAL : CPU, 1.200s; elapsed, 0.309s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.200s; elapsed, 0.308s, #calls: 1', 'TOTAL : CPU, 1.200s; elapsed, 0.308s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.200s; elapsed, 0.306s, #calls: 1', 'TOTAL : CPU, 1.200s; elapsed, 0.306s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.850s; elapsed, 0.224s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.220s; elapsed, 0.056s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.230s; elapsed, 0.309s, #calls: 1', 'TOTAL : CPU, 2.390s; elapsed, 0.613s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13151 3287.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.540s; elapsed, 0.208s', 'individual call time for EIGEN_LDLT: CPU, 0.470s; elapsed, 0.058s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.022s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', 'CPP -2', 'CPP -2', '8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,151', '# - number of nonzeros = 71,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.091068', '# - matching time = 0.0135081', '# - symmetrization time = 0.00524521', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0167642', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.12607 MB', '# - factor time = 0.121722', '# - factor nonzeros = 890,759', '# - factor memory = 7.12607 MB', '# - total flops = 2.2243e+08, min = 3.03207e+07, max = 1.02914e+08', '# - flop rate = 1.82736 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.36424e+08', '# --------------------------------------------', '# total = 2.36424e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.05977e-11\trel.res = 6.91666e-16\tbw.error = 1.00432e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0120201', '# - total flops = 2.71052e+06, min = 467563, max = 853409', '# - flop rate = 0.225499 GFlop/s', '# - bytes moved = 19.9803 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.66224 GByte/s', '# - solve arithmetic intensity = 0.13566 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.100s; elapsed, 0.268s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.110s; elapsed, 0.270s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.110s; elapsed, 0.270s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.170s; elapsed, 0.272s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.100s; elapsed, 0.268s, #calls: 1', 'TOTAL : CPU, 2.100s; elapsed, 0.268s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.110s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 2.110s; elapsed, 0.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.110s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 2.110s; elapsed, 0.270s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.540s; elapsed, 0.208s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.022s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.470s; elapsed, 0.058s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.170s; elapsed, 0.272s, #calls: 1', 'TOTAL : CPU, 4.350s; elapsed, 0.560s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13151 1643.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.370s; elapsed, 0.364s', 'individual call time for EIGEN_LDLT: CPU, 0.060s; elapsed, 0.053s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.028s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,151', '# - number of nonzeros = 71,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0925171', '# - matching time = 0.0123351', '# - symmetrization time = 0.00547099', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0149798', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.42935 MB', '# - factor time = 0.174405', '# - factor nonzeros = 928,669', '# - factor memory = 7.42935 MB', '# - total flops = 2.46999e+08, min = 203065, max = 1.04596e+08', '# - flop rate = 1.41624 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.59517e+08', '# --------------------------------------------', '# total = 2.59517e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.14133e-11\trel.res = 7.44898e-16\tbw.error = 1.25235e-14', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.013133', '# - total flops = 3.47987e+06, min = 18269, max = 845631', '# - flop rate = 0.26497 GFlop/s', '# - bytes moved = 19.7676 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.50518 GByte/s', '# - solve arithmetic intensity = 0.176039 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.290s; elapsed, 0.318s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.300s; elapsed, 0.320s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.300s; elapsed, 0.321s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.310s; elapsed, 0.321s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.290s; elapsed, 0.321s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.310s; elapsed, 0.321s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.310s; elapsed, 0.321s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.320s; elapsed, 0.323s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.290s; elapsed, 0.318s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.318s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.300s; elapsed, 0.321s, #calls: 1', 'TOTAL : CPU, 0.300s; elapsed, 0.321s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.300s; elapsed, 0.320s, #calls: 1', 'TOTAL : CPU, 0.300s; elapsed, 0.320s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.290s; elapsed, 0.321s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.321s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.310s; elapsed, 0.321s, #calls: 1', 'TOTAL : CPU, 0.310s; elapsed, 0.321s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.310s; elapsed, 0.321s, #calls: 1', 'TOTAL : CPU, 0.310s; elapsed, 0.321s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.310s; elapsed, 0.321s, #calls: 1', 'TOTAL : CPU, 0.310s; elapsed, 0.321s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.370s; elapsed, 0.364s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.028s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.060s; elapsed, 0.053s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.320s; elapsed, 0.323s, #calls: 1', 'TOTAL : CPU, 0.770s; elapsed, 0.768s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13151 1643.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.540s; elapsed, 0.272s', 'individual call time for EIGEN_LDLT: CPU, 0.090s; elapsed, 0.046s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.023s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,151', '# - number of nonzeros = 71,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 66', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.074728', '# - matching time = 0.010653', '# - symmetrization time = 0.00514889', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0134952', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.42935 MB', '# - factor time = 0.148522', '# - factor nonzeros = 928,669', '# - factor memory = 7.42935 MB', '# - total flops = 2.47008e+08, min = 203065, max = 1.04605e+08', '# - flop rate = 1.66311 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.59517e+08', '# --------------------------------------------', '# total = 2.59517e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.13746e-11\trel.res = 7.42368e-16\tbw.error = 1.25235e-14', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0109618', '# - total flops = 3.47987e+06, min = 18269, max = 845631', '# - flop rate = 0.317455 GFlop/s', '# - bytes moved = 19.7761 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.8041 GByte/s', '# - solve arithmetic intensity = 0.175963 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.530s; elapsed, 0.271s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.520s; elapsed, 0.271s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.520s; elapsed, 0.271s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.530s; elapsed, 0.271s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.530s; elapsed, 0.270s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.530s; elapsed, 0.271s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.530s; elapsed, 0.271s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.540s; elapsed, 0.273s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.530s; elapsed, 0.271s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.271s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.530s; elapsed, 0.271s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.271s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.530s; elapsed, 0.271s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.271s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.530s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.520s; elapsed, 0.271s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.271s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.530s; elapsed, 0.271s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.271s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.520s; elapsed, 0.271s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.271s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.540s; elapsed, 0.272s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.090s; elapsed, 0.046s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.540s; elapsed, 0.273s, #calls: 1', 'TOTAL : CPU, 1.210s; elapsed, 0.614s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13151 1643.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.910s; elapsed, 0.235s', 'individual call time for EIGEN_LDLT: CPU, 0.210s; elapsed, 0.054s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.022s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,151', '# - number of nonzeros = 71,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0929', '# - matching time = 0.0132482', '# - symmetrization time = 0.00585699', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.017561', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.42935 MB', '# - factor time = 0.127258', '# - factor nonzeros = 928,669', '# - factor memory = 7.42935 MB', '# - total flops = 2.47012e+08, min = 204465, max = 1.04605e+08', '# - flop rate = 1.94103 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.59517e+08', '# --------------------------------------------', '# total = 2.59517e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.14175e-11\trel.res = 7.45172e-16\tbw.error = 1.25235e-14', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0143731', '# - total flops = 3.47987e+06, min = 18269, max = 845631', '# - flop rate = 0.24211 GFlop/s', '# - bytes moved = 19.787 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.37668 GByte/s', '# - solve arithmetic intensity = 0.175866 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.120s; elapsed, 0.283s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.110s; elapsed, 0.283s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.100s; elapsed, 0.282s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.120s; elapsed, 0.283s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.120s; elapsed, 0.283s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.110s; elapsed, 0.283sindividual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.120s; elapsed, 0.283s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.160s; elapsed, 0.288s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.120s; elapsed, 0.283s, #calls: 1', 'TOTAL : CPU, 1.120s; elapsed, 0.283s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.100s; elapsed, 0.282s, #calls: 1', 'TOTAL : CPU, 1.100s; elapsed, 0.282s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.110s; elapsed, 0.283s, #calls: 1', 'TOTAL : CPU, 1.110s; elapsed, 0.283s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.120s; elapsed, 0.283s, #calls: 1', 'TOTAL : CPU, 1.120s; elapsed, 0.283s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.120s; elapsed, 0.283s, #calls: 1', 'Exiting profilerTOTAL : CPU, 1.120s; elapsed, 0.283s', '', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.120s; elapsed, 0.283s, #calls: 1', 'TOTAL : CPU, 1.120s; elapsed, 0.283s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.110s; elapsed, 0.283s, #calls: 1', 'TOTAL : CPU, 1.110s; elapsed, 0.283s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.910s; elapsed, 0.235s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.022s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.210s; elapsed, 0.054s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.160s; elapsed, 0.288s, #calls: 1', 'TOTAL : CPU, 2.370s; elapsed, 0.599s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13151 821.9375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.390s; elapsed, 0.396s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.058s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.029s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,151', '# - number of nonzeros = 71,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0908659', '# - matching time = 0.013247', '# - symmetrization time = 0.00605607', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.015811', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.42935 MB', '# - factor time = 0.146544', '# - factor nonzeros = 928,669', '# - factor memory = 7.42935 MB', '# - total flops = 2.46999e+08, min = 203065, max = 5.38251e+07', '# - flop rate = 1.68549 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.56676e+08', '# --------------------------------------------', '# total = 2.56676e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.11213e-11\trel.res = 7.25836e-16\tbw.error = 1.05596e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0142581', '# - total flops = 4.95377e+06, min = 14159, max = 819300', '# - flop rate = 0.347434 GFlop/s', '# - bytes moved = 18.7103 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.31225 GByte/s', '# - solve arithmetic intensity = 0.264762 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.280s; elapsed, 0.292sindividual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.250s; elapsed, 0.292s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.270s; elapsed, 0.290s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.290s; elapsed, 0.295s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.270s; elapsed, 0.291s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.260s; elapsed, 0.292s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.230s; elapsed, 0.292s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.240s; elapsed, 0.292s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.280s; elapsed, 0.292sindividual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.240s; elapsed, 0.293s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.280s; elapsed, 0.291s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.270s; elapsed, 0.292s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.250s; elapsed, 0.292s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.260s; elapsed, 0.293s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.260s; elapsed, 0.294s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.300s; elapsed, 0.297s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.250s; elapsed, 0.292s, #calls: 1', 'TOTAL : CPU, 0.250s; elapsed, 0.292s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.270s; elapsed, 0.290s, #calls: 1', 'TOTAL : CPU, 0.270s; elapsed, 0.290s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.280s; elapsed, 0.292s, #calls: 1', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.270s; elapsed, 0.291s, #calls: 1', 'TOTAL : CPU, 0.270s; elapsed, 0.291s', 'TOTAL : CPU, 0.280s; elapsed, 0.292s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.280s; elapsed, 0.292s, #calls: 1', 'TOTAL : CPU, 0.280s; elapsed, 0.292s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.230s; elapsed, 0.292s, #calls: 1', 'TOTAL : CPU, 0.230s; elapsed, 0.292s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.250s; elapsed, 0.292s, #calls: 1', 'TOTAL : CPU, 0.250s; elapsed, 0.292s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.260s; elapsed, 0.294s, #calls: 1', 'Exiting profiler', 'TOTAL : CPU, 0.260s; elapsed, 0.294s', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.240s; elapsed, 0.292s, #calls: 1', 'TOTAL : CPU, 0.240s; elapsed, 0.292s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.290s; elapsed, 0.295s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.295s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.260s; elapsed, 0.292s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.292s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.240s; elapsed, 0.293s, #calls: 1', 'TOTAL : CPU, 0.240s; elapsed, 0.293s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.270s; elapsed, 0.292s, #calls: 1', 'TOTAL : CPU, 0.270s; elapsed, 0.292s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.260s; elapsed, 0.293s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.293s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.280s; elapsed, 0.291s, #calls: 1', 'TOTAL : CPU, 0.280s; elapsed, 0.291s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.390s; elapsed, 0.396s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.029s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.058s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.300s; elapsed, 0.297s, #calls: 1', 'TOTAL : CPU, 0.770s; elapsed, 0.780s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13151 821.9375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.580s; elapsed, 0.299s', 'individual call time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.054s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.025s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,151', '# - number of nonzeros = 71,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0939062', '# - matching time = 0.01335', '# - symmetrization time = 0.00614715', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.016891', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.42935 MB', '# - factor time = 0.12631', '# - factor nonzeros = 928,669', '# - factor memory = 7.42935 MB', '# - total flops = 2.47002e+08, min = 203065, max = 5.38279e+07', '# - flop rate = 1.95552 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.56676e+08', '# --------------------------------------------', '# total = 2.56676e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.15102e-11\trel.res = 7.51218e-16\tbw.error = 1.0554e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0170059', '# - total flops = 4.95377e+06, min = 14159, max = 819300', '# - flop rate = 0.291297 GFlop/s', '# - bytes moved = 18.7152 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.10051 GByte/s', '# - solve arithmetic intensity = 0.264692 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.540s; elapsed, 0.287s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.560s; elapsed, 0.286s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.540s; elapsed, 0.287s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.530s; elapsed, 0.285s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.560s; elapsed, 0.286s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.520s; elapsed, 0.284s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.560s; elapsed, 0.286sindividual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.550s; elapsed, 0.285s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.530s; elapsed, 0.284s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.540s; elapsed, 0.285s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.540s; elapsed, 0.284s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.540s; elapsed, 0.285s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.550s; elapsed, 0.286s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.530s; elapsed, 0.286s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.540s; elapsed, 0.288s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.570s; elapsed, 0.292s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.540s; elapsed, 0.287s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.287s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.540s; elapsed, 0.287s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.287s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.530s; elapsed, 0.285s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.285s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.540s; elapsed, 0.285s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.285s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.560s; elapsed, 0.286s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.286s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.530s; elapsed, 0.286s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.286s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.520s; elapsed, 0.284s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.284s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.550s; elapsed, 0.285s, #calls: 1', 'TOTAL : CPU, 0.550s; elapsed, 0.285s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.560s; elapsed, 0.286s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.286s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.530s; elapsed, 0.284s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.284s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.540s; elapsed, 0.284s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.284s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.540s; elapsed, 0.285s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.285s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.560s; elapsed, 0.286s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.286s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.550s; elapsed, 0.286s, #calls: 1', 'TOTAL : CPU, 0.550s; elapsed, 0.286s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.540s; elapsed, 0.288s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.288s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.580s; elapsed, 0.299s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.025s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.054s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.570s; elapsed, 0.292s, #calls: 1', 'TOTAL : CPU, 1.310s; elapsed, 0.669s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 14651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.920s; elapsed, 0.927s', 'individual call time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.223s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.065s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.920s; elapsed, 0.927s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.065s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.223s, #calls: 1', 'TOTAL : CPU, 1.210s; elapsed, 1.215s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 14651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.440s; elapsed, 0.736s', 'individual call time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.221s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.051s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.440s; elapsed, 0.736s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.051s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.221s, #calls: 1', 'TOTAL : CPU, 1.940s; elapsed, 1.009s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 14651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.690s; elapsed, 0.441s', 'individual call time for EIGEN_LDLT: CPU, 0.780s; elapsed, 0.236s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.049s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.690s; elapsed, 0.441s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.049s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.780s; elapsed, 0.236s, #calls: 1', 'TOTAL : CPU, 2.640s; elapsed, 0.725s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 14651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.540s; elapsed, 0.464s', 'individual call time for EIGEN_LDLT: CPU, 1.560s; elapsed, 0.243s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.040s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.540s; elapsed, 0.464s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.040s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.560s; elapsed, 0.243s, #calls: 1', 'TOTAL : CPU, 5.350s; elapsed, 0.747s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 14651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.530s; elapsed, 0.434s', 'individual call time for EIGEN_LDLT: CPU, 2.970s; elapsed, 0.255s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.460s; elapsed, 0.040s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.530s; elapsed, 0.434s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.460s; elapsed, 0.040s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.970s; elapsed, 0.255s, #calls: 1', 'TOTAL : CPU, 9.960s; elapsed, 0.729s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 7325.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.990s; elapsed, 0.979s', 'individual call time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.236s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.068s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processesCPP -2', '', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.# - nd time = ', '# ******************************************************************', '0.0969272', '# - matching time = 0.018106', '# - symmetrization time = 0.00695801', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.022372', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.600411', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.08505e+09, min = 2.09373e+08, max = 8.75678e+08', '# - flop rate = 1.80718 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.14259e+09', '# --------------------------------------------', '# total = 1.14259e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.62462e-11\trel.res = 1.0032e-15\tbw.error = 1.73909e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.018826', '# - total flops = 5.3468e+06, min = 1.93797e+06, max = 3.40882e+06', '# - flop rate = 0.284011 GFlop/s', '# - bytes moved = 43.3862 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.30459 GByte/s', '# - solve arithmetic intensity = 0.123237 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.760s; elapsed, 0.774s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.780s; elapsed, 0.782s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.760s; elapsed, 0.774s, #calls: 1', 'TOTAL : CPU, 0.760s; elapsed, 0.774s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.990s; elapsed, 0.979s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.068s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.236s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.780s; elapsed, 0.782s, #calls: 1', 'TOTAL : CPU, 2.070s; elapsed, 2.065s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 7325.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.220s; elapsed, 0.619s', 'individual call time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.231s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.055s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.098866', '# - matching time = 0.019407', '# - symmetrization time = 0.007658', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0257959', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.443069', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.0851e+09, min = 2.09373e+08, max = 8.75729e+08', '# - flop rate = 2.44906 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.14259e+09', '# --------------------------------------------', '# total = 1.14259e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.55563e-11\trel.res = 9.60601e-16\tbw.error = 1.7363e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0172482', '# - total flops = 5.34832e+06, min = 1.93797e+06, max = 3.41034e+06', '# - flop rate = 0.310081 GFlop/s', '# - bytes moved = 43.4535 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.51931 GByte/s', '# - solve arithmetic intensity = 0.123081 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.230s; elapsed, 0.625s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.240s; elapsed, 0.634s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.230s; elapsed, 0.625s, #calls: 1', 'TOTAL : CPU, 1.230s; elapsed, 0.625s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.220s; elapsed, 0.619s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.055s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.231s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.240s; elapsed, 0.634s, #calls: 1', 'TOTAL : CPU, 2.960s; elapsed, 1.538s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 7325.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.640s; elapsed, 0.428s', 'individual call time for EIGEN_LDLT: CPU, 0.790s; elapsed, 0.237s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.049s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0974638', '# - matching time = 0.0189638', '# - symmetrization time = 0.00763202', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0290601', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.313763', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.08511e+09, min = 2.09373e+08, max = 8.75736e+08', '# - flop rate = 3.45837 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.14259e+09', '# --------------------------------------------', '# total = 1.14259e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.55568e-11\trel.res = 9.60632e-16\tbw.error = 1.72833e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0242651', '# - total flops = 5.34832e+06, min = 1.93797e+06, max = 3.41034e+06', '# - flop rate = 0.220412 GFlop/s', '# - bytes moved = 43.4677 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.79137 GByte/s', '# - solve arithmetic intensity = 0.123041 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.980s; elapsed, 0.505s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.050s; elapsed, 0.514s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.980s; elapsed, 0.505s, #calls: 1', 'TOTAL : CPU, 1.980s; elapsed, 0.505s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.640s; elapsed, 0.428s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.049s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.790s; elapsed, 0.237s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.050s; elapsed, 0.514s, #calls: 1', 'TOTAL : CPU, 4.640s; elapsed, 1.228s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 7325.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.190s; elapsed, 0.422s', 'individual call time for EIGEN_LDLT: CPU, 1.540s; elapsed, 0.255s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.042s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.109184', '# - matching time = 0.0204589', '# - symmetrization time = 0.00823522', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0277951', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.353601', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.08515e+09, min = 2.09375e+08, max = 8.75776e+08', '# - flop rate = 3.06886 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.14259e+09', '# --------------------------------------------', '# total = 1.14259e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.56172e-11\trel.res = 9.6436e-16\tbw.error = 1.79793e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0278351', '# - total flops = 5.34832e+06, min = 1.93797e+06, max = 3.41034e+06', '# - flop rate = 0.192143 GFlop/s', '# - bytes moved = 43.4973 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.56268 GByte/s', '# - solve arithmetic intensity = 0.122957 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.440s; elapsed, 0.564s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.570s; elapsed, 0.572s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.440s; elapsed, 0.564s, #calls: 1', 'TOTAL : CPU, 4.440s; elapsed, 0.564s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.190s; elapsed, 0.422s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.042s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.540s; elapsed, 0.255s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.570s; elapsed, 0.572s, #calls: 1', 'TOTAL : CPU, 9.550s; elapsed, 1.291s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 7325.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.490s; elapsed, 0.436s', 'individual call time for EIGEN_LDLT: CPU, 3.040s; elapsed, 0.247s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.460s; elapsed, 0.040s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.110718', '# - matching time = 0.0465579', '# - symmetrization time = 0.00915909', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.044347', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.378584', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.08516e+09, min = 2.09376e+08, max = 8.75779e+08', '# - flop rate = 2.86635 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.14259e+09', '# --------------------------------------------', '# total = 1.14259e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.55885e-11\trel.res = 9.62587e-16\tbw.error = 1.78159e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0291791', '# - total flops = 5.34832e+06, min = 1.93797e+06, max = 3.41034e+06', '# - flop rate = 0.183293 GFlop/s', '# - bytes moved = 43.5424 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.49225 GByte/s', '# - solve arithmetic intensity = 0.12283 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.170s; elapsed, 0.638s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.290s; elapsed, 0.647s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.170s; elapsed, 0.638s, #calls: 1', 'TOTAL : CPU, 9.170s; elapsed, 0.638s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.490s; elapsed, 0.436s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.460s; elapsed, 0.040s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.040s; elapsed, 0.247s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.290s; elapsed, 0.647s, #calls: 1', 'TOTAL : CPU, 20.280s; elapsed, 1.370s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 3662.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.040s; elapsed, 1.040s', 'individual call time for EIGEN_LDLT: CPU, 0.250s; elapsed, 0.249s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.076s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.107642', '# - matching time = 0.0207858', '# - symmetrization time = 0.010478', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0208499', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.379176', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.08505e+09, min = 1.79002e+08, max = 4.05598e+08', '# - flop rate = 2.8616 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.1396e+09', '# --------------------------------------------', '# total = 1.1396e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.61383e-11\trel.res = 9.96538e-16\tbw.error = 1.81756e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0278521', '# - total flops = 6.70752e+06, min = 1.30999e+06, max = 1.98026e+06', '# - flop rate = 0.240827 GFlop/s', '# - bytes moved = 39.3021 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.4111 GByte/s', '# - solve arithmetic intensity = 0.170665 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.570s; elapsed, 0.578s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.580s; elapsed, 0.577s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.550s; elapsed, 0.572s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.580s; elapsed, 0.582s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.570s; elapsed, 0.578s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.578s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.580s; elapsed, 0.577s, #calls: 1', 'TOTAL : CPU, 0.580s; elapsed, 0.577s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.550s; elapsed, 0.572s, #calls: 1', 'TOTAL : CPU, 0.550s; elapsed, 0.572s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.040s; elapsed, 1.040s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.076s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.250s; elapsed, 0.249s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.580s; elapsed, 0.582s, #calls: 1', 'TOTAL : CPU, 1.950s; elapsed, 1.948s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 3662.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.270s; elapsed, 0.646s', 'individual call time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.229s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.057s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0984101', '# - matching time = 0.0185878', '# - symmetrization time = 0.00966001', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.022687', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.329532', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.08506e+09, min = 1.79002e+08, max = 4.0561e+08', '# - flop rate = 3.29274 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.1396e+09', '# --------------------------------------------', '# total = 1.1396e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.60231e-11\trel.res = 9.89423e-16\tbw.error = 1.72892e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0164199', '# - total flops = 6.70752e+06, min = 1.30999e+06, max = 1.98026e+06', '# - flop rate = 0.4085 GFlop/s', '# - bytes moved = 39.3115 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.39414 GByte/s', '# - solve arithmetic intensity = 0.170625 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.000s; elapsed, 0.504s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.010s; elapsed, 0.508s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.000s; elapsed, 0.509s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.020s; elapsed, 0.513s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.000s; elapsed, 0.504s, #calls: 1', 'TOTAL : CPU, 1.000s; elapsed, 0.504s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.010s; elapsed, 0.508s, #calls: 1', 'TOTAL : CPU, 1.010s; elapsed, 0.508s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.000s; elapsed, 0.509s, #calls: 1', 'TOTAL : CPU, 1.000s; elapsed, 0.509s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.270s; elapsed, 0.646s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.057s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.229s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.020s; elapsed, 0.513s, #calls: 1', 'TOTAL : CPU, 2.800s; elapsed, 1.445s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 3662.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.800s; elapsed, 0.465s', 'individual call time for EIGEN_LDLT: CPU, 0.800s; elapsed, 0.249s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.051s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using CPP -2', 'CPP -2', 'CPP -2', '4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0988998', '# - matching time = 0.0194359', '# - symmetrization time = 0.00939083', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.022419', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.333732', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.08506e+09, min = 1.79002e+08, max = 4.05612e+08', '# - flop rate = 3.2513 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.1396e+09', '# --------------------------------------------', '# total = 1.1396e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.60077e-11\trel.res = 9.88473e-16\tbw.error = 1.73483e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0188949', '# - total flops = 6.70752e+06, min = 1.30999e+06, max = 1.98026e+06', '# - flop rate = 0.354991 GFlop/s', '# - bytes moved = 39.3235 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.08117 GByte/s', '# - solve arithmetic intensity = 0.170573 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.020s; elapsed, 0.509s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.010s; elapsed, 0.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.030s; elapsed, 0.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.070s; elapsed, 0.517s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.020s; elapsed, 0.509s, #calls: 1', 'TOTAL : CPU, 2.020s; elapsed, 0.509s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.010s; elapsed, 0.513s, #calls: 1', 'TOTAL : CPU, 2.010s; elapsed, 0.513s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.030s; elapsed, 0.513s, #calls: 1', 'TOTAL : CPU, 2.030s; elapsed, 0.513s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.800s; elapsed, 0.465s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.051s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.800s; elapsed, 0.249s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.070s; elapsed, 0.517s, #calls: 1', 'TOTAL : CPU, 4.840s; elapsed, 1.281s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 3662.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.160s; elapsed, 0.415s', 'individual call time for EIGEN_LDLT: CPU, 1.510s; elapsed, 0.246s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.040s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', 'CPP -2', '8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.111128', '# - matching time = 0.0222631', '# - symmetrization time = 0.0106661', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0243969', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.364013', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.08507e+09, min = 1.79004e+08, max = 4.05614e+08', '# - flop rate = 2.98086 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.1396e+09', '# --------------------------------------------', '# total = 1.1396e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.58963e-11\trel.res = 9.81591e-16\tbw.error = 1.7112e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0280299', '# - total flops = 6.70752e+06, min = 1.30999e+06, max = 1.98026e+06', '# - flop rate = 0.239299 GFlop/s', '# - bytes moved = 39.3473 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.40376 GByte/s', '# - solve arithmetic intensity = 0.17047 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.500s; elapsed, 0.569s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.510s; elapsed, 0.572s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.520s; elapsed, 0.573s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.600s; elapsed, 0.577s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.500s; elapsed, 0.569s, #calls: 1', 'TOTAL : CPU, 4.500s; elapsed, 0.569s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.510s; elapsed, 0.572s, #calls: 1', 'TOTAL : CPU, 4.510s; elapsed, 0.572s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.520s; elapsed, 0.573s, #calls: 1', 'TOTAL : CPU, 4.520s; elapsed, 0.573s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.160s; elapsed, 0.415s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.040s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.510s; elapsed, 0.246s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.600s; elapsed, 0.577s, #calls: 1', 'TOTAL : CPU, 9.510s; elapsed, 1.278s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 1831.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.980s; elapsed, 0.988s', 'individual call time for EIGEN_LDLT: CPU, 0.260s; elapsed, 0.246s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.070s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 66', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.0969429', '# - matching time = 0.0183339', '# - symmetrization time = 0.00886297', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0190101', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4218 MB', '# - factor time = 0.343356', '# - factor nonzeros = 2,177,723', '# - factor memory = 17.4218 MB', '# - total flops = 1.08505e+09, min = 746150, max = 2.89001e+08', '# - flop rate = 3.16014 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.13413e+09', '# --------------------------------------------', '# total = 1.13413e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.45204e-11\trel.res = 8.96633e-16\tbw.error = 1.36052e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0186291', '# - total flops = 8.80018e+06, min = 28189, max = 1.89032e+06', '# - flop rate = 0.472389 GFlop/s', '# - bytes moved = 38.099 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.04514 GByte/s', '# - solve arithmetic intensity = 0.230982 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.500s; elapsed, 0.509s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.480s; elapsed, 0.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.480s; elapsed, 0.514s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.500s; elapsed, 0.514s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.490s; elapsed, 0.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.490s; elapsed, 0.511s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.500s; elapsed, 0.511s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.520s; elapsed, 0.518s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.490s; elapsed, 0.513s, #calls: 1', 'TOTAL : CPU, 0.490s; elapsed, 0.513s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.480s; elapsed, 0.514s, #calls: 1', 'TOTAL : CPU, 0.480s; elapsed, 0.514s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.500s; elapsed, 0.514s, #calls: 1', 'TOTAL : CPU, 0.500s; elapsed, 0.514s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.480s; elapsed, 0.513s, #calls: 1', 'TOTAL : CPU, 0.480s; elapsed, 0.513s', 'Exiting profiler', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.500s; elapsed, 0.511s, #calls: 1', 'TOTAL : CPU, 0.500s; elapsed, 0.511s', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.490s; elapsed, 0.511s, #calls: 1', 'TOTAL : CPU, 0.490s; elapsed, 0.511s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.500s; elapsed, 0.509s, #calls: 1', 'TOTAL : CPU, 0.500s; elapsed, 0.509s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.980s; elapsed, 0.988s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.070s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.260s; elapsed, 0.246s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.520s; elapsed, 0.518s, #calls: 1', 'TOTAL : CPU, 1.820s; elapsed, 1.822s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 1831.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.240s; elapsed, 0.625s', 'individual call time for EIGEN_LDLT: CPU, 0.420s; elapsed, 0.243s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.063s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 66', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.112455', '# - matching time = 0.0211492', '# - symmetrization time = 0.00980306', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0229301', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4218 MB', '# - factor time = 0.314378', '# - factor nonzeros = 2,177,723', '# - factor memory = 17.4218 MB', '# - total flops = 1.08506e+09, min = 746150, max = 2.8901e+08', '# - flop rate = 3.45145 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.13413e+09', '# --------------------------------------------', '# total = 1.13413e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.44024e-11\trel.res = 8.89345e-16\tbw.error = 1.35888e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0194039', '# - total flops = 8.80018e+06, min = 28189, max = 1.89032e+06', '# - flop rate = 0.453525 GFlop/s', '# - bytes moved = 38.107 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.96388 GByte/s', '# - solve arithmetic intensity = 0.230933 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.010s; elapsed, 0.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.010s; elapsed, 0.508s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.990s; elapsed, 0.512s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.950s; elapsed, 0.512s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.990s; elapsed, 0.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.950s; elapsed, 0.512s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.000s; elapsed, 0.511s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.020s; elapsed, 0.517s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.010s; elapsed, 0.513s, #calls: 1', 'TOTAL : CPU, 1.010s; elapsed, 0.513s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.950s; elapsed, 0.512s, #calls: 1', 'TOTAL : CPU, 0.950s; elapsed, 0.512s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.990s; elapsed, 0.512s, #calls: 1', 'TOTAL : CPU, 0.990s; elapsed, 0.512s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.950s; elapsed, 0.512s, #calls: 1', 'TOTAL : CPU, 0.950s; elapsed, 0.512s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.000s; elapsed, 0.511s, #calls: 1', 'TOTAL : CPU, 1.000s; elapsed, 0.511s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.010s; elapsed, 0.508s, #calls: 1', 'TOTAL : CPU, 1.010s; elapsed, 0.508s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.990s; elapsed, 0.513s, #calls: 1', 'TOTAL : CPU, 0.990s; elapsed, 0.513s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.240s; elapsed, 0.625s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.063s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.420s; elapsed, 0.243s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.020s; elapsed, 0.517s, #calls: 1', 'TOTAL : CPU, 2.800s; elapsed, 1.447s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 1831.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.670s; elapsed, 0.434s', 'individual call time for EIGEN_LDLT: CPU, 0.800s; elapsed, 0.264s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.054s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.113922', '# - matching time = 0.021781', '# - symmetrization time = 0.0101352', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.024581', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4218 MB', '# - factor time = 0.32277', '# - factor nonzeros = 2,177,723', '# - factor memory = 17.4218 MB', '# - total flops = 1.08507e+09, min = 747880, max = 2.89012e+08', '# - flop rate = 3.36174 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.13413e+09', '# --------------------------------------------', '# total = 1.13413e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.43736e-11\trel.res = 8.8757e-16\tbw.error = 1.35593e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.027128', '# - total flops = 8.80018e+06, min = 28189, max = 1.89032e+06', '# - flop rate = 0.324395 GFlop/s', '# - bytes moved = 38.1167 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.40507 GByte/s', '# - solve arithmetic intensity = 0.230875 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.100s; elapsed, 0.533s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.960s; elapsed, 0.533s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.040s; elapsed, 0.533s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.940s; elapsed, 0.533s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.940s; elapsed, 0.533s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.110s; elapsed, 0.533s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.110s; elapsed, 0.531s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.130s; elapsed, 0.538s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.110s; elapsed, 0.531s, #calls: 1', 'TOTAL : CPU, 2.110s; elapsed, 0.531s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.100s; elapsed, 0.533s, #calls: 1', 'TOTAL : CPU, 2.100s; elapsed, 0.533s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.040s; elapsed, 0.533s, #calls: 1', 'TOTAL : CPU, 2.040s; elapsed, 0.533s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.110s; elapsed, 0.533s, #calls: 1', 'TOTAL : CPU, 2.110s; elapsed, 0.533s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.940s; elapsed, 0.533s, #calls: 1', 'TOTAL : CPU, 1.940s; elapsed, 0.533s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.960s; elapsed, 0.533s, #calls: 1', 'TOTAL : CPU, 1.960s; elapsed, 0.533s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.940s; elapsed, 0.533s, #calls: 1', 'TOTAL : CPU, 1.940s; elapsed, 0.533s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.670s; elapsed, 0.434s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.054s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.800s; elapsed, 0.264s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.130s; elapsed, 0.538s, #calls: 1', 'TOTAL : CPU, 4.780s; elapsed, 1.290s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 915.6875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.110s; elapsed, 1.112s', 'individual call time for EIGEN_LDLT: CPU, 0.260s; elapsed, 0.259s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.079s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.112476', '# - matching time = 0.021364', '# - symmetrization time = 0.00913787', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.021734', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.405566', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.08505e+09, min = 535689, max = 2.85362e+08', '# - flop rate = 2.6754 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.1232e+09', '# --------------------------------------------', '# total = 1.1232e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.30996e-11\trel.res = 8.08899e-16\tbw.error = 1.11099e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0186679', '# - total flops = 1.37536e+07, min = 20884, max = 1.84992e+06', '# - flop rate = 0.736748 GFlop/s', '# - bytes moved = 36.1237 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.93507 GByte/s', '# - solve arithmetic intensity = 0.380735 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.540s; elapsed, 0.597s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.560s; elapsed, 0.593s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.530s; elapsed, 0.599s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.590s; elapsed, 0.598s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.590s; elapsed, 0.597s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.520s; elapsed, 0.599sindividual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.550s; elapsed, 0.596s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.580s; elapsed, 0.598s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.590s; elapsed, 0.598s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.570s; elapsed, 0.596s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.570s; elapsed, 0.597s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.580s; elapsed, 0.596s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.570s; elapsed, 0.597s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.560s; elapsed, 0.597s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.550s; elapsed, 0.597s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.600s; elapsed, 0.602s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.540s; elapsed, 0.597s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.597s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.560s; elapsed, 0.597s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.597s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.520s; elapsed, 0.599s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.599s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.560s; elapsed, 0.593s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.593s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.530s; elapsed, 0.599s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.599s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.570s; elapsed, 0.596s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.596s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.590s; elapsed, 0.598s, #calls: 1', 'TOTAL : CPU, 0.590s; elapsed, 0.598s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.580s; elapsed, 0.598s, #calls: 1', 'TOTAL : CPU, 0.580s; elapsed, 0.598s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.570s; elapsed, 0.597s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.597s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.550s; elapsed, 0.596s, #calls: 1', 'TOTAL : CPU, 0.550s; elapsed, 0.596s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.590s; elapsed, 0.598s, #calls: 1', 'TOTAL : CPU, 0.590s; elapsed, 0.598s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.550s; elapsed, 0.597s, #calls: 1', 'TOTAL : CPU, 0.550s; elapsed, 0.597s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.570s; elapsed, 0.597s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.597s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.590s; elapsed, 0.597s, #calls: 1', 'TOTAL : CPU, 0.590s; elapsed, 0.597s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.580s; elapsed, 0.596s, #calls: 1', 'TOTAL : CPU, 0.580s; elapsed, 0.596s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.110s; elapsed, 1.112s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.079s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.260s; elapsed, 0.259s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.600s; elapsed, 0.602s, #calls: 1', 'TOTAL : CPU, 2.050s; elapsed, 2.052s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 915.6875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.330s; elapsed, 0.679s', 'individual call time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.235s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.061s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 65', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.11895', '# - matching time = 0.0219691', '# - symmetrization time = 0.011755', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0258489', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.350025', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.08505e+09, min = 535689, max = 2.85362e+08', '# - flop rate = 3.09993 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.1232e+09', '# --------------------------------------------', '# total = 1.1232e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.30501e-11\trel.res = 8.0584e-16\tbw.error = 1.07391e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.020859', '# - total flops = 1.37536e+07, min = 20884, max = 1.84992e+06', '# - flop rate = 0.659358 GFlop/s', '# - bytes moved = 36.1289 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.73206 GByte/s', '# - solve arithmetic intensity = 0.38068 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 1.100s; elapsed, 0.562s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.050s; elapsed, 0.564s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.030s; elapsed, 0.564s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.030s; elapsed, 0.564s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 1.110s; elapsed, 0.562s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.030s; elapsed, 0.565s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.080s; elapsed, 0.563s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.040s; elapsed, 0.562s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.050s; elapsed, 0.562s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 1.000s; elapsed, 0.565s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 1.090s; elapsed, 0.562s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 1.090s; elapsed, 0.558s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 1.110s; elapsed, 0.563s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 1.080s; elapsed, 0.563s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 1.080s; elapsed, 0.562s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.140s; elapsed, 0.569s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 1.000s; elapsed, 0.565s, #calls: 1', 'TOTAL : CPU, 1.000s; elapsed, 0.565s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.030s; elapsed, 0.565s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.565s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 1.100s; elapsed, 0.562s, #calls: 1', 'TOTAL : CPU, 1.100s; elapsed, 0.562s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 1.090s; elapsed, 0.562s, #calls: 1', 'TOTAL : CPU, 1.090s; elapsed, 0.562s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.030s; elapsed, 0.564s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.564s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.040s; elapsed, 0.562s, #calls: 1', 'TOTAL : CPU, 1.040s; elapsed, 0.562s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.030s; elapsed, 0.564s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.564s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.050s; elapsed, 0.562s, #calls: 1', 'TOTAL : CPU, 1.050s; elapsed, 0.562sExiting profiler', '', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.080s; elapsed, 0.563s, #calls: 1', 'TOTAL : CPU, 1.080s; elapsed, 0.563s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 1.110s; elapsed, 0.562s, #calls: 1', 'TOTAL : CPU, 1.110s; elapsed, 0.562s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.050s; elapsed, 0.564s, #calls: 1', 'TOTAL : CPU, 1.050s; elapsed, 0.564s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 1.090s; elapsed, 0.558s, #calls: 1', 'TOTAL : CPU, 1.090s; elapsed, 0.558s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 1.110s; elapsed, 0.563s, #calls: 1', 'TOTAL : CPU, 1.110s; elapsed, 0.563s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 1.080s; elapsed, 0.563s, #calls: 1', 'TOTAL : CPU, 1.080s; elapsed, 0.563s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 1.080s; elapsed, 0.562s, #calls: 1', 'TOTAL : CPU, 1.080s; elapsed, 0.562s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.330s; elapsed, 0.679s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.061s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.235s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.140s; elapsed, 0.569s, #calls: 1', 'TOTAL : CPU, 2.990s; elapsed, 1.545s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 14651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.330s; elapsed, 0.328s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.047s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.026s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.330s; elapsed, 0.328s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.026s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.047s, #calls: 1', 'TOTAL : CPU, 0.400s; elapsed, 0.401s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 14651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.570s; elapsed, 0.292s', 'individual call time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.055s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.021s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.570s; elapsed, 0.292s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.055s, #calls: 1', 'TOTAL : CPU, 0.730s; elapsed, 0.367s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 14651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.760s; elapsed, 0.198s', 'individual call time for EIGEN_LDLT: CPU, 0.200s; elapsed, 0.052s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.021s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.760s; elapsed, 0.198s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.200s; elapsed, 0.052s, #calls: 1', 'TOTAL : CPU, 1.050s; elapsed, 0.271s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 14651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.570s; elapsed, 0.209s', 'individual call time for EIGEN_LDLT: CPU, 0.670s; elapsed, 0.085s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.200s; elapsed, 0.025s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.570s; elapsed, 0.209s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.200s; elapsed, 0.025s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.670s; elapsed, 0.085s, #calls: 1', 'TOTAL : CPU, 2.440s; elapsed, 0.319s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 14651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.260s; elapsed, 0.222s', 'individual call time for EIGEN_LDLT: CPU, 0.810s; elapsed, 0.051s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.020s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.260s; elapsed, 0.222s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.310s; elapsed, 0.020s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.810s; elapsed, 0.051s, #calls: 1', 'TOTAL : CPU, 4.380s; elapsed, 0.293s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 7325.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.350s; elapsed, 0.355s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.049s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.027s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,# - nd time = ', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.0857291', '# - matching time = 0.0113728', '# - symmetrization time = 0.00339007', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.014513', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.16826 MB', '# - factor time = 0.188344', '# - factor nonzeros = 896,033', '# - factor memory = 7.16826 MB', '# - total flops = 1.94418e+08, min = 3.11578e+07, max = 1.6326e+08', '# - flop rate = 1.03225 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.08278e+08', '# --------------------------------------------', '# total = 2.08278e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.17207e-11\trel.res = 7.64957e-16\tbw.error = 1.74155e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.011441', '# - total flops = 2.3455e+06, min = 815211, max = 1.53029e+06', '# - flop rate = 0.205009 GFlop/s', '# - bytes moved = 21.3766 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.86842 GByte/s', '# - solve arithmetic intensity = 0.109723 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.320s; elapsed, 0.321s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.320s; elapsed, 0.327s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.320s; elapsed, 0.321s, #calls: 1', 'TOTAL : CPU, 0.320s; elapsed, 0.321s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.350s; elapsed, 0.355s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.027s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.049s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.320s; elapsed, 0.327s, #calls: 1', 'TOTAL : CPU, 0.750s; elapsed, 0.757s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 7325.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.500s; elapsed, 0.258s', 'individual call time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.051s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.022s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0874209', '# - matching time = 0.011538', '# - symmetrization time = 0.00375485', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.016063', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.16826 MB', '# - factor time = 0.117243', '# - factor nonzeros = 896,033', '# - factor memory = 7.16826 MB', '# - total flops = 1.94441e+08, min = 3.11578e+07, max = 1.63283e+08', '# - flop rate = 1.65844 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.08278e+08', '# --------------------------------------------', '# total = 2.08278e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.15192e-11\trel.res = 7.5181e-16\tbw.error = 1.67704e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.012059', '# - total flops = 2.3455e+06, min = 815211, max = 1.53029e+06', '# - flop rate = 0.194503 GFlop/s', '# - bytes moved = 21.3893 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.77372 GByte/s', '# - solve arithmetic intensity = 0.109658 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.510s; elapsed, 0.259s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.520s; elapsed, 0.264s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.510s; elapsed, 0.259s, #calls: 1', 'TOTAL : CPU, 0.510s; elapsed, 0.259s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.500s; elapsed, 0.258s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.022s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.051s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.520s; elapsed, 0.264s, #calls: 1', 'TOTAL : CPU, 1.170s; elapsed, 0.595s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 7325.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.910s; elapsed, 0.241s', 'individual call time for EIGEN_LDLT: CPU, 0.190s; elapsed, 0.050s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.020s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.08599', '# - matching time = 0.0115731', '# - symmetrization time = 0.00373721', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0202341', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.16826 MB', '# - factor time = 0.141523', '# - factor nonzeros = 896,033', '# - factor memory = 7.16826 MB', '# - total flops = 1.94443e+08, min = 3.11578e+07, max = 1.63285e+08', '# - flop rate = 1.37393 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.08278e+08', '# --------------------------------------------', '# total = 2.08278e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.14358e-11\trel.res = 7.46362e-16\tbw.error = 1.67704e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0101039', '# - total flops = 2.3455e+06, min = 815211, max = 1.53029e+06', '# - flop rate = 0.232138 GFlop/s', '# - bytes moved = 21.4055 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.11853 GByte/s', '# - solve arithmetic intensity = 0.109575 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.100s; elapsed, 0.283s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.160s; elapsed, 0.289s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.100s; elapsed, 0.283s, #calls: 1', 'TOTAL : CPU, 1.100s; elapsed, 0.283s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.910s; elapsed, 0.241s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.020s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.190s; elapsed, 0.050s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.160s; elapsed, 0.289s, #calls: 1', 'TOTAL : CPU, 2.340s; elapsed, 0.599s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 7325.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.540s; elapsed, 0.207s', 'individual call time for EIGEN_LDLT: CPU, 0.450s; elapsed, 0.057s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.022s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# - nd time = # (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.098012', '# - matching time = 0.0135651', '# - symmetrization time = 0.00437188', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.022089', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.16826 MB', '# - factor time = 0.103098', '# - factor nonzeros = 896,033', '# - factor memory = 7.16826 MB', '# - total flops = 1.94446e+08, min = 3.11596e+07, max = 1.63286e+08', '# - flop rate = 1.88603 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.08278e+08', '# --------------------------------------------', '# total = 2.08278e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.1387e-11\trel.res = 7.43176e-16\tbw.error = 1.64769e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.012013', '# - total flops = 2.3455e+06, min = 815211, max = 1.53029e+06', '# - flop rate = 0.195248 GFlop/s', '# - bytes moved = 21.418 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.78291 GByte/s', '# - solve arithmetic intensity = 0.109511 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.070s; elapsed, 0.265s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.170s; elapsed, 0.272s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.070s; elapsed, 0.265s, #calls: 1', 'TOTAL : CPU, 2.070s; elapsed, 0.265s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.540s; elapsed, 0.207s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.022s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.450s; elapsed, 0.057s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.170s; elapsed, 0.272s, #calls: 1', 'TOTAL : CPU, 4.340s; elapsed, 0.557s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 7325.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.270s; elapsed, 0.222s', 'individual call time for EIGEN_LDLT: CPU, 0.840s; elapsed, 0.053s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.330s; elapsed, 0.021s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0988491', '# - matching time = 0.0174532', '# - symmetrization time = 0.00879812', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.02373', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.16826 MB', '# - factor time = 0.105458', '# - factor nonzeros = 896,033', '# - factor memory = 7.16826 MB', '# - total flops = 1.94449e+08, min = 3.11613e+07, max = 1.63288e+08', '# - flop rate = 1.84385 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.08278e+08', '# --------------------------------------------', '# total = 2.08278e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.1385e-11\trel.res = 7.43046e-16\tbw.error = 1.64769e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.012285', '# - total flops = 2.3455e+06, min = 815211, max = 1.53029e+06', '# - flop rate = 0.190924 GFlop/s', '# - bytes moved = 21.4212 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.74369 GByte/s', '# - solve arithmetic intensity = 0.109494 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.980s; elapsed, 0.280s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.700s; elapsed, 0.294s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.980s; elapsed, 0.280s, #calls: 1', 'TOTAL : CPU, 3.980s; elapsed, 0.280s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.270s; elapsed, 0.222s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.330s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.840s; elapsed, 0.053s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.700s; elapsed, 0.294s, #calls: 1', 'TOTAL : CPU, 9.140s; elapsed, 0.589s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 3662.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.380s; elapsed, 0.373s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.054s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.029s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0945191', '# - matching time = 0.012991', '# - symmetrization time = 0.00453901', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0130851', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.29695 MB', '# - factor time = 0.106418', '# - factor nonzeros = 912,119', '# - factor memory = 7.29695 MB', '# - total flops = 2.03201e+08, min = 3.14439e+07, max = 8.73785e+07', '# - flop rate = 1.90946 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.1638e+08', '# --------------------------------------------', '# total = 2.1638e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.06571e-11\trel.res = 6.95539e-16\tbw.error = 1.13009e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0123899', '# - total flops = 2.7707e+06, min = 524653, max = 800713', '# - flop rate = 0.223626 GFlop/s', '# - bytes moved = 20.2222 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.63215 GByte/s', '# - solve arithmetic intensity = 0.137013 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.230s; elapsed, 0.248s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.240s; elapsed, 0.250s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.240s; elapsed, 0.250s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.240s; elapsed, 0.254s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.240s; elapsed, 0.250s, #calls: 1', 'TOTAL : CPU, 0.240s; elapsed, 0.250s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.240s; elapsed, 0.250s, #calls: 1', 'TOTAL : CPU, 0.240s; elapsed, 0.250s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.230s; elapsed, 0.248s, #calls: 1', 'TOTAL : CPU, 0.230s; elapsed, 0.248s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.380s; elapsed, 0.373s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.029s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.054s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.240s; elapsed, 0.254s, #calls: 1', 'TOTAL : CPU, 0.700s; elapsed, 0.710s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 3662.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.520s; elapsed, 0.268s', 'individual call time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.049s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.023s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.086524', '# - matching time = 0.0118148', '# - symmetrization time = 0.0049181', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.013422', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.29695 MB', '# - factor time = 0.102518', '# - factor nonzeros = 912,119', '# - factor memory = 7.29695 MB', '# - total flops = 2.03205e+08, min = 3.14439e+07, max = 8.73825e+07', '# - flop rate = 1.98214 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.1638e+08', '# --------------------------------------------', '# total = 2.1638e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.03211e-11\trel.res = 6.73609e-16\tbw.error = 1.13009e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0115921', '# - total flops = 2.7707e+06, min = 524653, max = 800713', '# - flop rate = 0.239016 GFlop/s', '# - bytes moved = 20.2271 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.7449 GByte/s', '# - solve arithmetic intensity = 0.13698 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.460s; elapsed, 0.239s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.460s; elapsed, 0.240s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.460s; elapsed, 0.241s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.480s; elapsed, 0.244s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.460s; elapsed, 0.241s, #calls: 1', 'TOTAL : CPU, 0.460s; elapsed, 0.241s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.460s; elapsed, 0.240s, #calls: 1', 'TOTAL : CPU, 0.460s; elapsed, 0.240s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.460s; elapsed, 0.239s, #calls: 1', 'TOTAL : CPU, 0.460s; elapsed, 0.239s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.520s; elapsed, 0.268s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.049s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.480s; elapsed, 0.244s, #calls: 1', 'TOTAL : CPU, 1.140s; elapsed, 0.584s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 3662.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.790s; elapsed, 0.210s', 'individual call time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.057s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.022s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# - nd time = # (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.0965018', '# - matching time = 0.0133011', '# - symmetrization time = 0.00562', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0159009', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.29695 MB', '# - factor time = 0.130877', '# - factor nonzeros = 912,119', '# - factor memory = 7.29695 MB', '# - total flops = 2.03205e+08, min = 3.14439e+07, max = 8.73825e+07', '# - flop rate = 1.55264 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.1638e+08', '# --------------------------------------------', '# total = 2.1638e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.03158e-11\trel.res = 6.73267e-16\tbw.error = 1.13009e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.010844', '# - total flops = 2.7707e+06, min = 524653, max = 800713', '# - flop rate = 0.255506 GFlop/s', '# - bytes moved = 20.2303 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.86558 GByte/s', '# - solve arithmetic intensity = 0.136958 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.110s; elapsed, 0.281s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.100s; elapsed, 0.280s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.120s; elapsed, 0.282s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.140s; elapsed, 0.286s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.110s; elapsed, 0.281s, #calls: 1', 'TOTAL : CPU, 1.110s; elapsed, 0.281s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.120s; elapsed, 0.282s, #calls: 1', 'TOTAL : CPU, 1.120s; elapsed, 0.282s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.100s; elapsed, 0.280s, #calls: 1', 'TOTAL : CPU, 1.100s; elapsed, 0.280s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.790s; elapsed, 0.210s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.022s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.057s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.140s; elapsed, 0.286s, #calls: 1', 'TOTAL : CPU, 2.250s; elapsed, 0.575s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 3662.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.820s; elapsed, 0.243s', 'individual call time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.054s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.022s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0979559', '# - matching time = 0.0155511', '# - symmetrization time = 0.00563693', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0169358', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.29695 MB', '# - factor time = 0.123922', '# - factor nonzeros = 912,119', '# - factor memory = 7.29695 MB', '# - total flops = 2.03211e+08, min = 3.14458e+07, max = 8.73836e+07', '# - flop rate = 1.63983 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.1638e+08', '# --------------------------------------------', '# total = 2.1638e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.01573e-11\trel.res = 6.62922e-16\tbw.error = 1.13691e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0123491', '# - total flops = 2.7707e+06, min = 524653, max = 800713', '# - flop rate = 0.224364 GFlop/s', '# - bytes moved = 20.2308 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.63823 GByte/s', '# - solve arithmetic intensity = 0.136955 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.210s; elapsed, 0.281s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.200s; elapsed, 0.283s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.210s; elapsed, 0.284s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.300s; elapsed, 0.288s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.210s; elapsed, 0.281s, #calls: 1', 'TOTAL : CPU, 2.210s; elapsed, 0.281s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.200s; elapsed, 0.283s, #calls: 1', 'TOTAL : CPU, 2.200s; elapsed, 0.283s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.210s; elapsed, 0.284s, #calls: 1', 'TOTAL : CPU, 2.210s; elapsed, 0.284s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.820s; elapsed, 0.243s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.022s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.054s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.300s; elapsed, 0.288s, #calls: 1', 'TOTAL : CPU, 4.720s; elapsed, 0.607s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 1831.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.390s; elapsed, 0.397s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.054s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.030s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,089', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0957708', '# - matching time = 0.0131218', '# - symmetrization time = 0.00344801', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,089', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.013046', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.52468 MB', '# - factor time = 0.135299', '# - factor nonzeros = 940,585', '# - factor memory = 7.52468 MB', '# - total flops = 2.24119e+08, min = 137603, max = 6.09266e+07', '# - flop rate = 1.65647 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.35469e+08', '# --------------------------------------------', '# total = 2.35469e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.0651e-11\trel.res = 6.95146e-16\tbw.error = 2.45888e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0128808', '# - total flops = 3.85024e+06, min = 18068, max = 796233', '# - flop rate = 0.298913 GFlop/s', '# - bytes moved = 19.5066 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.51439 GByte/s', '# - solve arithmetic intensity = 0.197382 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.270s; elapsed, 0.278s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.280s; elapsed, 0.277s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.250s; elapsed, 0.280s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.240s; elapsed, 0.278s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.270s; elapsed, 0.279s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.240s; elapsed, 0.278s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.260s; elapsed, 0.278s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.290s; elapsed, 0.283s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.270s; elapsed, 0.278s, #calls: 1', 'TOTAL : CPU, 0.270s; elapsed, 0.278s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.240s; elapsed, 0.278s, #calls: 1', 'TOTAL : CPU, 0.240s; elapsed, 0.278s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.240s; elapsed, 0.278s, #calls: 1', 'TOTAL : CPU, 0.240s; elapsed, 0.278s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.250s; elapsed, 0.280s, #calls: 1', 'TOTAL : CPU, 0.250s; elapsed, 0.280s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.260s; elapsed, 0.278s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.278s', 'Exiting profiler', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.270s; elapsed, 0.279s, #calls: 1', 'TOTAL : CPU, 0.270s; elapsed, 0.279s', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.280s; elapsed, 0.277s, #calls: 1', 'TOTAL : CPU, 0.280s; elapsed, 0.277s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.390s; elapsed, 0.397s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.030s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.054s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.290s; elapsed, 0.283s, #calls: 1', 'TOTAL : CPU, 0.760s; elapsed, 0.764s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 1831.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.590s; elapsed, 0.304s', 'individual call time for EIGEN_LDLT: CPU, 0.120s; elapsed, 0.055s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.024s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,089', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# - nd time = # ******************************************************************', '0.100895', '# - matching time = 0.013808', '# - symmetrization time = 0.00407505', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,089', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0142319', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.52468 MB', '# - factor time = 0.115712', '# - factor nonzeros = 940,585', '# - factor memory = 7.52468 MB', '# - total flops = 2.24122e+08, min = 137603, max = 6.09266e+07', '# - flop rate = 1.93689 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.35469e+08', '# --------------------------------------------', '# total = 2.35469e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.02036e-11\trel.res = 6.65946e-16\tbw.error = 2.45888e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.011698', '# - total flops = 3.85024e+06, min = 18068, max = 796233', '# - flop rate = 0.329137 GFlop/s', '# - bytes moved = 19.5114 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.66792 GByte/s', '# - solve arithmetic intensity = 0.197334 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.520s; elapsed, 0.270s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.530s; elapsed, 0.268s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.530s; elapsed, 0.269s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.530s; elapsed, 0.270s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.520s; elapsed, 0.270sindividual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.540s; elapsed, 0.271s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.530s; elapsed, 0.270s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.540s; elapsed, 0.275s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.520s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.530s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.530s; elapsed, 0.269s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.269s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.540s; elapsed, 0.271s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.271s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.530s; elapsed, 0.268s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.268s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.530s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.520s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.270s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.590s; elapsed, 0.304s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.024s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.120s; elapsed, 0.055s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.540s; elapsed, 0.275s, #calls: 1', 'TOTAL : CPU, 1.300s; elapsed, 0.658s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 1831.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.010s; elapsed, 0.263s', 'individual call time for EIGEN_LDLT: CPU, 0.250s; elapsed, 0.062s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.022s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threadsCPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,089', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.10383', '# - matching time = 0.014713', '# - symmetrization time = 0.00537801', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,089', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0157418', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.52468 MB', '# - factor time = 0.110168', '# - factor nonzeros = 940,585', '# - factor memory = 7.52468 MB', '# - total flops = 2.24127e+08, min = 139438, max = 6.09266e+07', '# - flop rate = 2.0344 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.35469e+08', '# --------------------------------------------', '# total = 2.35469e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.02458e-11\trel.res = 6.68696e-16\tbw.error = 2.45888e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0115292', '# - total flops = 3.85024e+06, min = 18068, max = 796233', '# - flop rate = 0.333956 GFlop/s', '# - bytes moved = 19.513 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.69248 GByte/s', '# - solve arithmetic intensity = 0.197317 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.070s; elapsed, 0.280s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.070s; elapsed, 0.273s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.080s; elapsed, 0.279s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.080s; elapsed, 0.278s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.080s; elapsed, 0.278s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.080s; elapsed, 0.279s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.080s; elapsed, 0.279s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.130s; elapsed, 0.284s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.070s; elapsed, 0.280s, #calls: 1', 'TOTAL : CPU, 1.070s; elapsed, 0.280s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.080s; elapsed, 0.278s, #calls: 1', 'TOTAL : CPU, 1.080s; elapsed, 0.278s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.070s; elapsed, 0.273s, #calls: 1', 'TOTAL : CPU, 1.070s; elapsed, 0.273s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.080s; elapsed, 0.279s, #calls: 1', 'TOTAL : CPU, 1.080s; elapsed, 0.279s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.080s; elapsed, 0.279s, #calls: 1', 'TOTAL : CPU, 1.080s; elapsed, 0.279s', 'Exiting profilerExiting profiler', '', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.080s; elapsed, 0.279s, #calls: 1', 'TOTAL : CPU, 1.080s; elapsed, 0.279s', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.080s; elapsed, 0.278s, #calls: 1', 'TOTAL : CPU, 1.080s; elapsed, 0.278s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.010s; elapsed, 0.263s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.022s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.250s; elapsed, 0.062s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.130s; elapsed, 0.284s, #calls: 1', 'TOTAL : CPU, 2.470s; elapsed, 0.631s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 915.6875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.400s; elapsed, 0.401s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.056s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.031s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,089', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.100725', '# - matching time = 0.0137949', '# - symmetrization time = 0.00394702', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,089', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0161221', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.52468 MB', '# - factor time = 0.151953', '# - factor nonzeros = 940,585', '# - factor memory = 7.52468 MB', '# - total flops = 2.24119e+08, min = 137603, max = 5.97816e+07', '# - flop rate = 1.47492 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.32465e+08', '# --------------------------------------------', '# total = 2.32465e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.0008e-11\trel.res = 6.53181e-16\tbw.error = 1.63926e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0140429', '# - total flops = 5.89424e+06, min = 13488, max = 780496', '# - flop rate = 0.419732 GFlop/s', '# - bytes moved = 18.6028 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.32472 GByte/s', '# - solve arithmetic intensity = 0.316846 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.260s; elapsed, 0.311s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.250s; elapsed, 0.313s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.260s; elapsed, 0.311s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.280s; elapsed, 0.312s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.300s; elapsed, 0.312s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.270s; elapsed, 0.311s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.250s; elapsed, 0.310s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.280s; elapsed, 0.313s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.260s; elapsed, 0.312s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.280s; elapsed, 0.311s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.280s; elapsed, 0.311s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.280s; elapsed, 0.312s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.300s; elapsed, 0.311sindividual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.290s; elapsed, 0.311s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.250s; elapsed, 0.312s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.310s; elapsed, 0.317s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.280s; elapsed, 0.312s, #calls: 1', 'TOTAL : CPU, 0.280s; elapsed, 0.312s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.270s; elapsed, 0.311s, #calls: 1', 'TOTAL : CPU, 0.270s; elapsed, 0.311s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.280s; elapsed, 0.312s, #calls: 1', 'TOTAL : CPU, 0.280s; elapsed, 0.312s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.260s; elapsed, 0.311s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.311s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.250s; elapsed, 0.310s, #calls: 1', 'TOTAL : CPU, 0.250s; elapsed, 0.310s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.260s; elapsed, 0.311s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.311s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.250s; elapsed, 0.313s, #calls: 1', 'TOTAL : CPU, 0.250s; elapsed, 0.313s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.260s; elapsed, 0.312s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.312s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.250s; elapsed, 0.312s, #calls: 1', 'TOTAL : CPU, 0.250s; elapsed, 0.312s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.300s; elapsed, 0.311s, #calls: 1', 'TOTAL : CPU, 0.300s; elapsed, 0.311s', 'Exiting profilerExiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.300s; elapsed, 0.312s, #calls: 1', '', 'TOTAL : CPU, 0.300s; elapsed, 0.312s', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.290s; elapsed, 0.311s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.311s', 'Exiting profilerExiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.280s; elapsed, 0.311s, #calls: 1', 'TOTAL : CPU, 0.280s; elapsed, 0.311s', '', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.280s; elapsed, 0.313s, #calls: 1', 'TOTAL : CPU, 0.280s; elapsed, 0.313s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.280s; elapsed, 0.311s, #calls: 1', 'TOTAL : CPU, 0.280s; elapsed, 0.311s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.400s; elapsed, 0.401s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.031s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.056s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.310s; elapsed, 0.317s, #calls: 1', 'TOTAL : CPU, 0.790s; elapsed, 0.805s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 915.6875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.540s; elapsed, 0.276s', 'individual call time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.059s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.027s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,089', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************# - nd time = ', '0.1058', '# - matching time = 0.0145051', '# - symmetrization time = 0.00408411', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,089', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.016897', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.52468 MB', '# - factor time = 0.120029', '# - factor nonzeros = 940,585', '# - factor memory = 7.52468 MB', '# - total flops = 2.24121e+08, min = 137603, max = 5.97816e+07', '# - flop rate = 1.86722 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.32465e+08', '# --------------------------------------------', '# total = 2.32465e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.0169e-11\trel.res = 6.63682e-16\tbw.error = 1.63926e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.011342', '# - total flops = 5.89424e+06, min = 13488, max = 780496', '# - flop rate = 0.51968 GFlop/s', '# - bytes moved = 18.6044 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.6403 GByte/s', '# - solve arithmetic intensity = 0.31682 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.540s; elapsed, 0.285s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.540s; elapsed, 0.282s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.530s; elapsed, 0.284s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.540s; elapsed, 0.283s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.530s; elapsed, 0.283s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.560s; elapsed, 0.282s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.530s; elapsed, 0.284s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.560s; elapsed, 0.284s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.550s; elapsed, 0.284s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.560s; elapsed, 0.283s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.530s; elapsed, 0.283s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.540s; elapsed, 0.284s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.550s; elapsed, 0.283s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.540s; elapsed, 0.285s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.560s; elapsed, 0.284s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.580s; elapsed, 0.289s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.540s; elapsed, 0.285s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.285s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.530s; elapsed, 0.284s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.284s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.560s; elapsed, 0.282s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.282s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.540s; elapsed, 0.285s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.285s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.550s; elapsed, 0.283s, #calls: 1', 'TOTAL : CPU, 0.550s; elapsed, 0.283s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.530s; elapsed, 0.283s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.283s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.540s; elapsed, 0.284s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.284s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.540s; elapsed, 0.283s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.283s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.540s; elapsed, 0.282s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.282s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.560s; elapsed, 0.284s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.284s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.530s; elapsed, 0.283s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.283s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.550s; elapsed, 0.284s, #calls: 1', 'TOTAL : CPU, 0.550s; elapsed, 0.284s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.560s; elapsed, 0.284s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.284s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.530s; elapsed, 0.284s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.284s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.560s; elapsed, 0.283s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.283s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.540s; elapsed, 0.276s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.027s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.059s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.580s; elapsed, 0.289s, #calls: 1', 'TOTAL : CPU, 1.300s; elapsed, 0.651s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 14151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.330s; elapsed, 0.328s', 'individual call time for EIGEN_LDLT: CPU, 0.040s; elapsed, 0.047s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.025s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.330s; elapsed, 0.328s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.025s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.040s; elapsed, 0.047s, #calls: 1', 'TOTAL : CPU, 0.400s; elapsed, 0.401s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 14151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.490s; elapsed, 0.246s', 'individual call time for EIGEN_LDLT: CPU, 0.090s; elapsed, 0.047s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.023s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.490s; elapsed, 0.246s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.090s; elapsed, 0.047s, #calls: 1', 'TOTAL : CPU, 0.620s; elapsed, 0.316s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 14151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.730s; elapsed, 0.194s', 'individual call time for EIGEN_LDLT: CPU, 0.210s; elapsed, 0.050s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.021s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.730s; elapsed, 0.194s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.210s; elapsed, 0.050s, #calls: 1', 'TOTAL : CPU, 1.020s; elapsed, 0.264s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 14151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.450s; elapsed, 0.193s', 'individual call time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.052s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.020s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.450s; elapsed, 0.193s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.020s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.052s, #calls: 1', 'TOTAL : CPU, 2.020s; elapsed, 0.264s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 14151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.200s; elapsed, 0.218s', 'individual call time for EIGEN_LDLT: CPU, 1.090s; elapsed, 0.068s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.390s; elapsed, 0.024s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.200s; elapsed, 0.218s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.390s; elapsed, 0.024s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.090s; elapsed, 0.068s, #calls: 1', 'TOTAL : CPU, 4.680s; elapsed, 0.310s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 7075.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.320s; elapsed, 0.321s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.047s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.025s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************# - nd time = ', '0.076623', '# - matching time = 0.0108061', '# - symmetrization time = 0.00413513', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.014225', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.70359 MB', '# - factor time = 0.186975', '# - factor nonzeros = 962,949', '# - factor memory = 7.70359 MB', '# - total flops = 2.49112e+08, min = 3.46562e+07, max = 2.14456e+08', '# - flop rate = 1.33233 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.63271e+08', '# --------------------------------------------', '# total = 2.63271e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.1079e-11\trel.res = 7.23077e-16\tbw.error = 1.25632e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.011647', '# - total flops = 2.47602e+06, min = 838849, max = 1.63717e+06', '# - flop rate = 0.212589 GFlop/s', '# - bytes moved = 21.8361 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.87483 GByte/s', '# - solve arithmetic intensity = 0.113391 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.300s; elapsed, 0.311s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.320s; elapsed, 0.318s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.300s; elapsed, 0.311s, #calls: 1', 'TOTAL : CPU, 0.300s; elapsed, 0.311s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.320s; elapsed, 0.321s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.025s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.047s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.320s; elapsed, 0.318s, #calls: 1', 'TOTAL : CPU, 0.710s; elapsed, 0.712s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 7075.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.500s; elapsed, 0.254s', 'individual call time for EIGEN_LDLT: CPU, 0.090s; elapsed, 0.049s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.023s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0840979', '# - matching time = 0.0117409', '# - symmetrization time = 0.00390196', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0169139', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.70359 MB', '# - factor time = 0.11983', '# - factor nonzeros = 962,949', '# - factor memory = 7.70359 MB', '# - total flops = 2.49139e+08, min = 3.46562e+07, max = 2.14483e+08', '# - flop rate = 2.07911 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.63271e+08', '# --------------------------------------------', '# total = 2.63271e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.04075e-11\trel.res = 6.79254e-16\tbw.error = 1.20867e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.012692', '# - total flops = 2.47602e+06, min = 838849, max = 1.63717e+06', '# - flop rate = 0.195085 GFlop/s', '# - bytes moved = 21.8492 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.7215 GByte/s', '# - solve arithmetic intensity = 0.113323 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.510s; elapsed, 0.259s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.540s; elapsed, 0.266s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.510s; elapsed, 0.259s, #calls: 1', 'TOTAL : CPU, 0.510s; elapsed, 0.259s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.500s; elapsed, 0.254s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.090s; elapsed, 0.049s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.540s; elapsed, 0.266s, #calls: 1', 'TOTAL : CPU, 1.180s; elapsed, 0.592s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 7075.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.200s; elapsed, 0.312s', 'individual call time for EIGEN_LDLT: CPU, 0.220s; elapsed, 0.055s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.021s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# - nd time = # (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.0953259', '# - matching time = 0.012867', '# - symmetrization time = 0.00445414', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0206342', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.70359 MB', '# - factor time = 0.0949342', '# - factor nonzeros = 962,949', '# - factor memory = 7.70359 MB', '# - total flops = 2.49144e+08, min = 3.46562e+07, max = 2.14488e+08', '# - flop rate = 2.62439 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.63271e+08', '# --------------------------------------------', '# total = 2.63271e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.02827e-11\trel.res = 6.71106e-16\tbw.error = 1.20855e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0111341', '# - total flops = 2.47602e+06, min = 838849, max = 1.63717e+06', '# - flop rate = 0.22238 GFlop/s', '# - bytes moved = 21.8668 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.96394 GByte/s', '# - solve arithmetic intensity = 0.113232 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.990s; elapsed, 0.250s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.030s; elapsed, 0.257s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.990s; elapsed, 0.250s, #calls: 1', 'TOTAL : CPU, 0.990s; elapsed, 0.250s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.200s; elapsed, 0.312s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.220s; elapsed, 0.055s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.030s; elapsed, 0.257s, #calls: 1', 'TOTAL : CPU, 2.540s; elapsed, 0.645s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 7075.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.670s; elapsed, 0.222s', 'individual call time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.051s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.020s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 68', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.08389', '# - matching time = 0.012392', '# - symmetrization time = 0.00386405', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0180838', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.70359 MB', '# - factor time = 0.100795', '# - factor nonzeros = 962,949', '# - factor memory = 7.70359 MB', '# - total flops = 2.49146e+08, min = 3.46578e+07, max = 2.14488e+08', '# - flop rate = 2.47181 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.63271e+08', '# --------------------------------------------', '# total = 2.63271e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.00272e-11\trel.res = 6.54433e-16\tbw.error = 1.2116e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0104201', '# - total flops = 2.47602e+06, min = 838849, max = 1.63717e+06', '# - flop rate = 0.23762 GFlop/s', '# - bytes moved = 21.8836 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.10014 GByte/s', '# - solve arithmetic intensity = 0.113145 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.900s; elapsed, 0.241s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.940s; elapsed, 0.243s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.900s; elapsed, 0.241s, #calls: 1', 'TOTAL : CPU, 1.900s; elapsed, 0.241s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.670s; elapsed, 0.222s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.020s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.051s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.940s; elapsed, 0.243s, #calls: 1', 'TOTAL : CPU, 4.190s; elapsed, 0.537s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 7075.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.750s; elapsed, 0.253s', 'individual call time for EIGEN_LDLT: CPU, 1.570s; elapsed, 0.098s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.410s; elapsed, 0.026s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.095108', '# - matching time = 0.0149369', '# - symmetrization time = 0.00495195', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0240209', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.70359 MB', '# - factor time = 0.107529', '# - factor nonzeros = 962,949', '# - factor memory = 7.70359 MB', '# - total flops = 2.49152e+08, min = 3.46594e+07, max = 2.14492e+08', '# - flop rate = 2.31706 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.63271e+08', '# --------------------------------------------', '# total = 2.63271e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.00817e-11\trel.res = 6.57988e-16\tbw.error = 1.2116e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.011657', '# - total flops = 2.47602e+06, min = 838849, max = 1.63717e+06', '# - flop rate = 0.212406 GFlop/s', '# - bytes moved = 21.8961 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.87837 GByte/s', '# - solve arithmetic intensity = 0.11308 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.140s; elapsed, 0.269s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.380s; elapsed, 0.274s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.140s; elapsed, 0.269s, #calls: 1', 'TOTAL : CPU, 4.140s; elapsed, 0.269s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.750s; elapsed, 0.253s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.410s; elapsed, 0.026s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.570s; elapsed, 0.098s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.380s; elapsed, 0.274s, #calls: 1', 'TOTAL : CPU, 10.110s; elapsed, 0.650s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 3537.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.370s; elapsed, 0.370s', 'individual call time for EIGEN_LDLT: CPU, 0.060s; elapsed, 0.054s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.029s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 4CPP -2', 'CPP -2', 'CPP -2', ' MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0932651', '# - matching time = 0.0127902', '# - symmetrization time = 0.00439405', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0129809', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.71692 MB', '# - factor time = 0.146585', '# - factor nonzeros = 964,615', '# - factor memory = 7.71692 MB', '# - total flops = 2.5035e+08, min = 2.90159e+07, max = 1.34565e+08', '# - flop rate = 1.70788 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.63777e+08', '# --------------------------------------------', '# total = 2.63777e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.13122e-11\trel.res = 7.383e-16\tbw.error = 2.51177e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0131218', '# - total flops = 2.86909e+06, min = 493941, max = 897810', '# - flop rate = 0.21865 GFlop/s', '# - bytes moved = 20.6003 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.56992 GByte/s', '# - solve arithmetic intensity = 0.139274 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.270s; elapsed, 0.287s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.290s; elapsed, 0.289s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.280s; elapsed, 0.290s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.280s; elapsed, 0.293s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.280s; elapsed, 0.290s, #calls: 1', 'TOTAL : CPU, 0.280s; elapsed, 0.290s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.290s; elapsed, 0.289s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.289s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.270s; elapsed, 0.287s, #calls: 1', 'TOTAL : CPU, 0.270s; elapsed, 0.287s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.370s; elapsed, 0.370s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.029s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.060s; elapsed, 0.054s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.280s; elapsed, 0.293s, #calls: 1', 'TOTAL : CPU, 0.740s; elapsed, 0.746s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 3537.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.530s; elapsed, 0.276s', 'individual call time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.054s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.023s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.094069', '# - matching time = 0.0131409', '# - symmetrization time = 0.00549698', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0154889', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.71692 MB', '# - factor time = 0.097012', '# - factor nonzeros = 964,615', '# - factor memory = 7.71692 MB', '# - total flops = 2.50361e+08, min = 2.90159e+07, max = 1.34577e+08', '# - flop rate = 2.58073 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.63777e+08', '# --------------------------------------------', '# total = 2.63777e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.13019e-11\trel.res = 7.37623e-16\tbw.error = 2.51177e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.011322', '# - total flops = 2.86909e+06, min = 493941, max = 897810', '# - flop rate = 0.253408 GFlop/s', '# - bytes moved = 20.6104 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.82038 GByte/s', '# - solve arithmetic intensity = 0.139206 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.480s; elapsed, 0.245s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.480s; elapsed, 0.247s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.490s; elapsed, 0.249s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.500s; elapsed, 0.252s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.480s; elapsed, 0.247s, #calls: 1', 'TOTAL : CPU, 0.480s; elapsed, 0.247s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.480s; elapsed, 0.245s, #calls: 1', 'TOTAL : CPU, 0.480s; elapsed, 0.245s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.490s; elapsed, 0.249s, #calls: 1', 'TOTAL : CPU, 0.490s; elapsed, 0.249s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.530s; elapsed, 0.276s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.054s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.500s; elapsed, 0.252s, #calls: 1', 'TOTAL : CPU, 1.180s; elapsed, 0.605s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 3537.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.860s; elapsed, 0.227s', 'individual call time for EIGEN_LDLT: CPU, 0.290s; elapsed, 0.073s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.023s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.095171', '# - matching time = 0.013761', '# - symmetrization time = 0.005826', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0149691', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.71692 MB', '# - factor time = 0.133696', '# - factor nonzeros = 964,615', '# - factor memory = 7.71692 MB', '# - total flops = 2.50364e+08, min = 2.90159e+07, max = 1.34579e+08', '# - flop rate = 1.87263 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.63777e+08', '# --------------------------------------------', '# total = 2.63777e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.12775e-11\trel.res = 7.3603e-16\tbw.error = 2.51177e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0121679', '# - total flops = 2.86909e+06, min = 493941, max = 897810', '# - flop rate = 0.235791 GFlop/s', '# - bytes moved = 20.623 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.69486 GByte/s', '# - solve arithmetic intensity = 0.139121 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.110s; elapsed, 0.283s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.120s; elapsed, 0.286s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.130s; elapsed, 0.285s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.160s; elapsed, 0.289s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.110s; elapsed, 0.283s, #calls: 1', 'TOTAL : CPU, 1.110s; elapsed, 0.283s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.120s; elapsed, 0.286s, #calls: 1', 'TOTAL : CPU, 1.120s; elapsed, 0.286s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.130s; elapsed, 0.285s, #calls: 1', 'TOTAL : CPU, 1.130s; elapsed, 0.285s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.860s; elapsed, 0.227s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.290s; elapsed, 0.073s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.160s; elapsed, 0.289s, #calls: 1', 'TOTAL : CPU, 2.400s; elapsed, 0.612s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 3537.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.930s; elapsed, 0.256s', 'individual call time for EIGEN_LDLT: CPU, 0.470s; elapsed, 0.060s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.021s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threadsCPP -2', 'CPP -2', 'CPP -2', '', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.098124', '# - matching time = 0.014226', '# - symmetrization time = 0.0075109', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0177991', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.71692 MB', '# - factor time = 0.122007', '# - factor nonzeros = 964,615', '# - factor memory = 7.71692 MB', '# - total flops = 2.50371e+08, min = 2.90175e+07, max = 1.34581e+08', '# - flop rate = 2.0521 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.63777e+08', '# --------------------------------------------', '# total = 2.63777e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.13334e-11\trel.res = 7.3968e-16\tbw.error = 2.51177e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0131922', '# - total flops = 2.86909e+06, min = 493941, max = 897810', '# - flop rate = 0.217484 GFlop/s', '# - bytes moved = 20.6319 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.56395 GByte/s', '# - solve arithmetic intensity = 0.139061 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.240s; elapsed, 0.286s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.230s; elapsed, 0.283sindividual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.220s; elapsed, 0.284s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.290s; elapsed, 0.289s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.240s; elapsed, 0.286s, #calls: 1', 'TOTAL : CPU, 2.240s; elapsed, 0.286s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.230s; elapsed, 0.283s, #calls: 1', 'TOTAL : CPU, 2.230s; elapsed, 0.283s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.220s; elapsed, 0.284s, #calls: 1', 'TOTAL : CPU, 2.220s; elapsed, 0.284s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.930s; elapsed, 0.256s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.470s; elapsed, 0.060s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.290s; elapsed, 0.289s, #calls: 1', 'TOTAL : CPU, 4.870s; elapsed, 0.626s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 1768.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.390s; elapsed, 0.382s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.054s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.029s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.092463', '# - matching time = 0.0130241', '# - symmetrization time = 0.00567818', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0138922', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.6429 MB', '# - factor time = 0.122361', '# - factor nonzeros = 955,363', '# - factor memory = 7.6429 MB', '# - total flops = 2.43618e+08, min = 181487, max = 9.78935e+07', '# - flop rate = 1.99098 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.55532e+08', '# --------------------------------------------', '# total = 2.55532e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.95445e-12\trel.res = 6.49683e-16\tbw.error = 8.94762e-16', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0157862', '# - total flops = 3.54215e+06, min = 18825, max = 800361', '# - flop rate = 0.224383 GFlop/s', '# - bytes moved = 19.924 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.26212 GByte/s', '# - solve arithmetic intensity = 0.177783 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.260s; elapsed, 0.268s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.250s; elapsed, 0.270s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.260s; elapsed, 0.270s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.230s; elapsed, 0.270s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.250s; elapsed, 0.271s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.240s; elapsed, 0.271s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.240s; elapsed, 0.269s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.280s; elapsed, 0.274s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.240s; elapsed, 0.271s, #calls: 1', 'TOTAL : CPU, 0.240s; elapsed, 0.271s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.260s; elapsed, 0.268s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.268s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.250s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 0.250s; elapsed, 0.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.250s; elapsed, 0.271s, #calls: 1', 'TOTAL : CPU, 0.250s; elapsed, 0.271s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.240s; elapsed, 0.269s, #calls: 1', 'TOTAL : CPU, 0.240s; elapsed, 0.269s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.230s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 0.230s; elapsed, 0.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.260s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.270s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.390s; elapsed, 0.382s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.029s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.054s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.280s; elapsed, 0.274s, #calls: 1', 'TOTAL : CPU, 0.750s; elapsed, 0.739s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 1768.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.570s; elapsed, 0.293s', 'individual call time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.054s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.026s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.095938', '# - matching time = 0.0134561', '# - symmetrization time = 0.011538', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0161731', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.6429 MB', '# - factor time = 0.1486', '# - factor nonzeros = 955,363', '# - factor memory = 7.6429 MB', '# - total flops = 2.43625e+08, min = 181487, max = 9.79012e+07', '# - flop rate = 1.63947 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.55532e+08', '# --------------------------------------------', '# total = 2.55532e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.96659e-12\trel.res = 6.50475e-16\tbw.error = 8.85161e-16', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.012501', '# - total flops = 3.54215e+06, min = 18825, max = 800361', '# - flop rate = 0.283349 GFlop/s', '# - bytes moved = 19.9308 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.59433 GByte/s', '# - solve arithmetic intensity = 0.177722 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.590s; elapsed, 0.309s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.610s; elapsed, 0.309s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.600s; elapsed, 0.310s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.600s; elapsed, 0.310s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.610s; elapsed, 0.310s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.610s; elapsed, 0.310s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.600s; elapsed, 0.309s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.630s; elapsed, 0.315s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.610s; elapsed, 0.309s, #calls: 1', 'TOTAL : CPU, 0.610s; elapsed, 0.309s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.600s; elapsed, 0.310s, #calls: 1', 'TOTAL : CPU, 0.600s; elapsed, 0.310s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.610s; elapsed, 0.310s, #calls: 1', 'TOTAL : CPU, 0.610s; elapsed, 0.310s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.610s; elapsed, 0.310s, #calls: 1', 'TOTAL : CPU, 0.610s; elapsed, 0.310s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.600s; elapsed, 0.310s, #calls: 1', 'TOTAL : CPU, 0.600s; elapsed, 0.310s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.600s; elapsed, 0.309s, #calls: 1', 'TOTAL : CPU, 0.600s; elapsed, 0.309s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.590s; elapsed, 0.309s, #calls: 1', 'TOTAL : CPU, 0.590s; elapsed, 0.309s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.570s; elapsed, 0.293s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.026s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.054s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.630s; elapsed, 0.315s, #calls: 1', 'TOTAL : CPU, 1.360s; elapsed, 0.687s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 1768.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.940s; elapsed, 0.247s', 'individual call time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.059s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.023s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.# - nd time = ', '# ******************************************************************', '0.096673', '# - matching time = 0.0141649', '# - symmetrization time = 0.00667596', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0191431', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.6429 MB', '# - factor time = 0.11052', '# - factor nonzeros = 955,363', '# - factor memory = 7.6429 MB', '# - total flops = 2.43632e+08, min = 183182, max = 9.79029e+07', '# - flop rate = 2.20442 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.55532e+08', '# --------------------------------------------', '# total = 2.55532e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.72427e-12\trel.res = 6.3466e-16\tbw.error = 8.82623e-16', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.014473', '# - total flops = 3.54215e+06, min = 18825, max = 800361', '# - flop rate = 0.244742 GFlop/s', '# - bytes moved = 19.9395 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.37771 GByte/s', '# - solve arithmetic intensity = 0.177644 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.070s; elapsed, 0.277s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.050s; elapsed, 0.274s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.080s; elapsed, 0.279sindividual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.060s; elapsed, 0.270s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.070s; elapsed, 0.277s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.070s; elapsed, 0.275s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.070s; elapsed, 0.275s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.110s; elapsed, 0.281s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.080s; elapsed, 0.279s, #calls: 1', 'TOTAL : CPU, 1.080s; elapsed, 0.279s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.070s; elapsed, 0.277s, #calls: 1', 'TOTAL : CPU, 1.070s; elapsed, 0.277s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.070s; elapsed, 0.277s, #calls: 1', 'TOTAL : CPU, 1.070s; elapsed, 0.277s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.070s; elapsed, 0.275s, #calls: 1', 'TOTAL : CPU, 1.070s; elapsed, 0.275s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.050s; elapsed, 0.274s, #calls: 1', 'TOTAL : CPU, 1.050s; elapsed, 0.274s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.060s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 1.060s; elapsed, 0.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.070s; elapsed, 0.275s, #calls: 1', 'TOTAL : CPU, 1.070s; elapsed, 0.275s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.940s; elapsed, 0.247s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.059s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.110s; elapsed, 0.281s, #calls: 1', 'TOTAL : CPU, 2.370s; elapsed, 0.610s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 884.4375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.390s; elapsed, 0.394s', 'individual call time for EIGEN_LDLT: CPU, 0.060s; elapsed, 0.056s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.029s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.093646', '# - matching time = 0.0131981', '# - symmetrization time = 0.00600481', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0146019', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.6429 MB', '# - factor time = 0.164273', '# - factor nonzeros = 955,363', '# - factor memory = 7.6429 MB', '# - total flops = 2.43618e+08, min = 178295, max = 4.76965e+07', '# - flop rate = 1.483 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.52727e+08', '# --------------------------------------------', '# total = 2.52727e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.03051e-11\trel.res = 6.72566e-16\tbw.error = 9.34049e-16', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.020925', '# - total flops = 5.0589e+06, min = 14405, max = 774758', '# - flop rate = 0.241763 GFlop/s', '# - bytes moved = 18.8309 MB, min = 0 MB, max = 0 MB', '# - byte rate = 0.899922 GByte/s', '# - solve arithmetic intensity = 0.268649 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.250s; elapsed, 0.321s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.300s; elapsed, 0.322s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.290s; elapsed, 0.322sindividual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.300s; elapsed, 0.325s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.320s; elapsed, 0.323s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.290s; elapsed, 0.323s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.290s; elapsed, 0.321s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.270s; elapsed, 0.323s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.330s; elapsed, 0.323s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.270s; elapsed, 0.323s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.280s; elapsed, 0.322s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.300s; elapsed, 0.322s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.300s; elapsed, 0.322sindividual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.290s; elapsed, 0.323sindividual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.270s; elapsed, 0.324s', '', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.320s; elapsed, 0.329s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.290s; elapsed, 0.323s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.323s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.300s; elapsed, 0.325s, #calls: 1', 'TOTAL : CPU, 0.300s; elapsed, 0.325s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.280s; elapsed, 0.322s, #calls: 1', 'TOTAL : CPU, 0.280s; elapsed, 0.322s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.290s; elapsed, 0.321s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.321s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.270s; elapsed, 0.323s, #calls: 1', 'TOTAL : CPU, 0.270s; elapsed, 0.323s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.300s; elapsed, 0.322s, #calls: 1', 'TOTAL : CPU, 0.300s; elapsed, 0.322s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.250s; elapsed, 0.321s, #calls: 1', 'TOTAL : CPU, 0.250s; elapsed, 0.321s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.290s; elapsed, 0.323s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.323s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.270s; elapsed, 0.324s, #calls: 1', 'TOTAL : CPU, 0.270s; elapsed, 0.324s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.300s; elapsed, 0.322s, #calls: 1', 'TOTAL : CPU, 0.300s; elapsed, 0.322s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.330s; elapsed, 0.323s, #calls: 1', 'TOTAL : CPU, 0.330s; elapsed, 0.323s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.320s; elapsed, 0.323s, #calls: 1', 'TOTAL : CPU, 0.320s; elapsed, 0.323s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.290s; elapsed, 0.322s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.322s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.300s; elapsed, 0.322s, #calls: 1', 'TOTAL : CPU, 0.300s; elapsed, 0.322s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.270s; elapsed, 0.323s, #calls: 1', 'TOTAL : CPU, 0.270s; elapsed, 0.323s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.390s; elapsed, 0.394s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.029s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.060s; elapsed, 0.056s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.320s; elapsed, 0.329s, #calls: 1', 'TOTAL : CPU, 0.800s; elapsed, 0.808s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 884.4375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.520s; elapsed, 0.266s', 'individual call time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.057s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.025s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 72,305', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.098695', '# - matching time = 0.0140989', '# - symmetrization time = 0.00617003', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.017652', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.6429 MB', '# - factor time = 0.109833', '# - factor nonzeros = 955,363', '# - factor memory = 7.6429 MB', '# - total flops = 2.43622e+08, min = 178295, max = 4.76965e+07', '# - flop rate = 2.21812 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.52727e+08', '# --------------------------------------------', '# total = 2.52727e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.0207e-11\trel.res = 6.66167e-16\tbw.error = 9.34544e-16', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0140102', '# - total flops = 5.0589e+06, min = 14405, max = 774758', '# - flop rate = 0.361087 GFlop/s', '# - bytes moved = 18.8371 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.34453 GByte/s', '# - solve arithmetic intensity = 0.26856 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.510s; elapsed, 0.270s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.520s; elapsed, 0.270s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.520s; elapsed, 0.271s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.530s; elapsed, 0.269s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.510s; elapsed, 0.270sindividual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.530s; elapsed, 0.272s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.520s; elapsed, 0.270s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.510s; elapsed, 0.269s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.490s; elapsed, 0.270s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.510s; elapsed, 0.270s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.520s; elapsed, 0.270s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.510s; elapsed, 0.268s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.500s; elapsed, 0.272s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.520s; elapsed, 0.270s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.530s; elapsed, 0.270s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.550s; elapsed, 0.276s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.500s; elapsed, 0.272s, #calls: 1', 'TOTAL : CPU, 0.500s; elapsed, 0.272s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.510s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 0.510s; elapsed, 0.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.520s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.520s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.530s; elapsed, 0.272s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.272s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.530s; elapsed, 0.269s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.269s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.520s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.510s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 0.510s; elapsed, 0.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.520s; elapsed, 0.271s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.271s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.510s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 0.510s; elapsed, 0.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.510s; elapsed, 0.269s, #calls: 1', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.490s; elapsed, 0.270s, #calls: 1TOTAL : CPU, 0.510s; elapsed, 0.269s', '', 'TOTAL : CPU, 0.490s; elapsed, 0.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.520s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.510s; elapsed, 0.268s, #calls: 1', 'TOTAL : CPU, 0.510s; elapsed, 0.268s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.530s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.270s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.520s; elapsed, 0.266s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.025s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.057s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.550s; elapsed, 0.276s, #calls: 1', 'TOTAL : CPU, 1.230s; elapsed, 0.624s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 14151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.950s; elapsed, 0.956s', 'individual call time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.227s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.064s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.950s; elapsed, 0.956s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.064s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.227s, #calls: 1', 'TOTAL : CPU, 1.240s; elapsed, 1.248s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 14151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.240s; elapsed, 0.627s', 'individual call time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.219s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.054s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.240s; elapsed, 0.627s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.054s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.219s, #calls: 1', 'TOTAL : CPU, 1.740s; elapsed, 0.899s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 14151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 2.580s; elapsed, 0.660s', 'individual call time for EIGEN_LDLT: CPU, 0.790s; elapsed, 0.221s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.046s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 2.580s; elapsed, 0.660s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.046s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.790s; elapsed, 0.221s, #calls: 1', 'TOTAL : CPU, 3.520s; elapsed, 0.927s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 14151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.660s; elapsed, 0.477s', 'individual call time for EIGEN_LDLT: CPU, 1.520s; elapsed, 0.247s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.041s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.660s; elapsed, 0.477s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.041s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.520s; elapsed, 0.247s, #calls: 1', 'TOTAL : CPU, 5.430s; elapsed, 0.765s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 14151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.810s; elapsed, 0.514s', 'individual call time for EIGEN_LDLT: CPU, 2.990s; elapsed, 0.230s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.038s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.810s; elapsed, 0.514s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.038s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.990s; elapsed, 0.230s, #calls: 1', 'TOTAL : CPU, 11.250s; elapsed, 0.782s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 7075.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.000s; elapsed, 1.007s', 'individual call time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.239s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.069s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0930569', '# - matching time = 0.017782', '# - symmetrization time = 0.00673199', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0224431', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.2436 MB', '# - factor time = 0.617503', '# - factor nonzeros = 2,280,451', '# - factor memory = 18.2436 MB', '# - total flops = 1.23667e+09, min = 2.27838e+08, max = 1.00883e+09', '# - flop rate = 2.00269 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29515e+09', '# --------------------------------------------', '# total = 1.29515e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.61784e-11\trel.res = 9.99011e-16\tbw.error = 1.86482e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.021126', '# - total flops = 5.54918e+06, min = 2.00566e+06, max = 3.54351e+06', '# - flop rate = 0.26267 GFlop/s', '# - bytes moved = 44.0333 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.08431 GByte/s', '# - solve arithmetic intensity = 0.126022 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.780s; elapsed, 0.788s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.800s; elapsed, 0.796s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.780s; elapsed, 0.788s, #calls: 1', 'TOTAL : CPU, 0.780s; elapsed, 0.788s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.000s; elapsed, 1.007s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.069s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.239s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.800s; elapsed, 0.796s, #calls: 1', 'TOTAL : CPU, 2.110s; elapsed, 2.111s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 7075.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.210s; elapsed, 0.615s', 'individual call time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.235s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.055s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0875828', '# - matching time = 0.0173571', '# - symmetrization time = 0.00715208', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0279641', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.2436 MB', '# - factor time = 0.328372', '# - factor nonzeros = 2,280,451', '# - factor memory = 18.2436 MB', '# - total flops = 1.23673e+09, min = 2.27838e+08, max = 1.00889e+09', '# - flop rate = 3.76625 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29515e+09', '# --------------------------------------------', '# total = 1.29515e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.56822e-11\trel.res = 9.68375e-16\tbw.error = 1.86748e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0155661', '# - total flops = 5.55004e+06, min = 2.00566e+06, max = 3.54438e+06', '# - flop rate = 0.356546 GFlop/s', '# - bytes moved = 44.0936 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.83267 GByte/s', '# - solve arithmetic intensity = 0.12587 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.990s; elapsed, 0.498s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.010s; elapsed, 0.507s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.990s; elapsed, 0.498s, #calls: 1', 'TOTAL : CPU, 0.990s; elapsed, 0.498s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.210s; elapsed, 0.615s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.055s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.235s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.010s; elapsed, 0.507s, #calls: 1', 'TOTAL : CPU, 2.740s; elapsed, 1.412s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 7075.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.600s; elapsed, 0.417s', 'individual call time for EIGEN_LDLT: CPU, 0.780s; elapsed, 0.237s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.042s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.088232', '# - matching time = 0.017029', '# - symmetrization time = 0.00733018', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.025816', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.2436 MB', '# - factor time = 0.265829', '# - factor nonzeros = 2,280,451', '# - factor memory = 18.2436 MB', '# - total flops = 1.23683e+09, min = 2.27838e+08, max = 1.00899e+09', '# - flop rate = 4.65272 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29515e+09', '# --------------------------------------------', '# total = 1.29515e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.56097e-11\trel.res = 9.63895e-16\tbw.error = 1.88062e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0244491', '# - total flops = 5.55089e+06, min = 2.00566e+06, max = 3.54523e+06', '# - flop rate = 0.227038 GFlop/s', '# - bytes moved = 44.1217 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.80463 GByte/s', '# - solve arithmetic intensity = 0.125809 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.760s; elapsed, 0.444s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.800s; elapsed, 0.453s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.760s; elapsed, 0.444s, #calls: 1', 'TOTAL : CPU, 1.760s; elapsed, 0.444s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.600s; elapsed, 0.417s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.130s; elapsed, 0.042s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.780s; elapsed, 0.237s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.800s; elapsed, 0.453s, #calls: 1', 'TOTAL : CPU, 4.310s; elapsed, 1.149s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 7075.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.890s; elapsed, 0.508s', 'individual call time for EIGEN_LDLT: CPU, 1.550s; elapsed, 0.240s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.041s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0958142', '# - matching time = 0.018568', '# - symmetrization time = 0.00725913', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.028338', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.2436 MB', '# - factor time = 0.293609', '# - factor nonzeros = 2,280,451', '# - factor memory = 18.2436 MB', '# - total flops = 1.23683e+09, min = 2.2784e+08, max = 1.00899e+09', '# - flop rate = 4.21252 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29515e+09', '# --------------------------------------------', '# total = 1.29515e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.54618e-11\trel.res = 9.54764e-16\tbw.error = 1.72307e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0208991', '# - total flops = 5.55089e+06, min = 2.00566e+06, max = 3.54523e+06', '# - flop rate = 0.265605 GFlop/s', '# - bytes moved = 44.1649 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.11325 GByte/s', '# - solve arithmetic intensity = 0.125686 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.810s; elapsed, 0.480s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.900s; elapsed, 0.489s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.810s; elapsed, 0.480s, #calls: 1', 'TOTAL : CPU, 3.810s; elapsed, 0.480s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.890s; elapsed, 0.508s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.041s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.550s; elapsed, 0.240s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.900s; elapsed, 0.489s, #calls: 1', 'TOTAL : CPU, 9.590s; elapsed, 1.278s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 7075.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.930s; elapsed, 0.464s', 'individual call time for EIGEN_LDLT: CPU, 2.990s; elapsed, 0.260s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.480s; elapsed, 0.040s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# - nd time = # ******************************************************************', '0.105526', '# - matching time = 0.0289898', '# - symmetrization time = 0.00821805', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.031229', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.2436 MB', '# - factor time = 0.397043', '# - factor nonzeros = 2,280,451', '# - factor memory = 18.2436 MB', '# - total flops = 1.23684e+09, min = 2.27842e+08, max = 1.009e+09', '# - flop rate = 3.11512 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29515e+09', '# --------------------------------------------', '# total = 1.29515e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.53985e-11\trel.res = 9.50857e-16\tbw.error = 1.71176e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0257299', '# - total flops = 5.55089e+06, min = 2.00566e+06, max = 3.54523e+06', '# - flop rate = 0.215737 GFlop/s', '# - bytes moved = 44.1973 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.71774 GByte/s', '# - solve arithmetic intensity = 0.125593 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.560s; elapsed, 0.613s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.920s; elapsed, 0.622s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.560s; elapsed, 0.613s, #calls: 1', 'TOTAL : CPU, 9.560s; elapsed, 0.613s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.930s; elapsed, 0.464s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.480s; elapsed, 0.040s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.990s; elapsed, 0.260s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.920s; elapsed, 0.622s, #calls: 1', 'TOTAL : CPU, 20.320s; elapsed, 1.386s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 3537.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.000s; elapsed, 0.988s', 'individual call time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.235s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.067s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0927379', '# - matching time = 0.017843', '# - symmetrization time = 0.00919604', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.019186', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.337 MB', '# - factor time = 0.370947', '# - factor nonzeros = 2,292,125', '# - factor memory = 18.337 MB', '# - total flops = 1.23992e+09, min = 1.94869e+08, max = 4.82492e+08', '# - flop rate = 3.34259 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29573e+09', '# --------------------------------------------', '# total = 1.29573e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.58604e-11\trel.res = 9.79375e-16\tbw.error = 1.66213e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0216169', '# - total flops = 7.0497e+06, min = 1.35975e+06, max = 2.14762e+06', '# - flop rate = 0.326119 GFlop/s', '# - bytes moved = 39.6406 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.83378 GByte/s', '# - solve arithmetic intensity = 0.17784 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.530s; elapsed, 0.536s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.540s; elapsed, 0.541s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.530s; elapsed, 0.541s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.550s; elapsed, 0.546s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.530s; elapsed, 0.536s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.536s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.530s; elapsed, 0.541s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.541s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.540s; elapsed, 0.541s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.541s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.000s; elapsed, 0.988s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.067s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.235s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.550s; elapsed, 0.546s, #calls: 1', 'TOTAL : CPU, 1.850s; elapsed, 1.836s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 3537.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.450s; elapsed, 0.739s', 'individual call time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.247s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.056s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', 'CPP -2', '2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.107401', '# - matching time = 0.0203741', '# - symmetrization time = 0.010222', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0232809', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.337 MB', '# - factor time = 0.323268', '# - factor nonzeros = 2,292,125', '# - factor memory = 18.337 MB', '# - total flops = 1.23994e+09, min = 1.94869e+08, max = 4.82507e+08', '# - flop rate = 3.83563 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29573e+09', '# --------------------------------------------', '# total = 1.29573e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.57167e-11\trel.res = 9.70501e-16\tbw.error = 1.65132e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.018183', '# - total flops = 7.0497e+06, min = 1.35975e+06, max = 2.14762e+06', '# - flop rate = 0.387708 GFlop/s', '# - bytes moved = 39.6513 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.18068 GByte/s', '# - solve arithmetic intensity = 0.177792 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.020s; elapsed, 0.516s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.010s; elapsed, 0.516s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.020s; elapsed, 0.511s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.040s; elapsed, 0.521s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.020s; elapsed, 0.516s, #calls: 1', 'TOTAL : CPU, 1.020s; elapsed, 0.516s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.010s; elapsed, 0.516s, #calls: 1', 'TOTAL : CPU, 1.010s; elapsed, 0.516s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.020s; elapsed, 0.511s, #calls: 1', 'TOTAL : CPU, 1.020s; elapsed, 0.511s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.450s; elapsed, 0.739s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.056s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.247s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.040s; elapsed, 0.521s, #calls: 1', 'TOTAL : CPU, 3.030s; elapsed, 1.563s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 3537.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.880s; elapsed, 0.490s', 'individual call time for EIGEN_LDLT: CPU, 0.830s; elapsed, 0.273s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.048s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', 'CPP -2', '4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.107556', '# - matching time = 0.0212789', '# - symmetrization time = 0.0104649', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.025902', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.337 MB', '# - factor time = 0.324438', '# - factor nonzeros = 2,292,125', '# - factor memory = 18.337 MB', '# - total flops = 1.23994e+09, min = 1.94869e+08, max = 4.82507e+08', '# - flop rate = 3.8218 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29573e+09', '# --------------------------------------------', '# total = 1.29573e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.56816e-11\trel.res = 9.68339e-16\tbw.error = 1.64794e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0202739', '# - total flops = 7.0497e+06, min = 1.35975e+06, max = 2.14762e+06', '# - flop rate = 0.347722 GFlop/s', '# - bytes moved = 39.6649 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.95645 GByte/s', '# - solve arithmetic intensity = 0.177731 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.080s; elapsed, 0.525s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.060s; elapsed, 0.520s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.070s; elapsed, 0.525s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.120s; elapsed, 0.530s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.080s; elapsed, 0.525s, #calls: 1', 'TOTAL : CPU, 2.080s; elapsed, 0.525s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.060s; elapsed, 0.520s, #calls: 1', 'TOTAL : CPU, 2.060s; elapsed, 0.520s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.070s; elapsed, 0.525s, #calls: 1', 'TOTAL : CPU, 2.070s; elapsed, 0.525s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.880s; elapsed, 0.490s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.048s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.830s; elapsed, 0.273s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.120s; elapsed, 0.530s, #calls: 1', 'TOTAL : CPU, 4.990s; elapsed, 1.340s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 3537.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.490s; elapsed, 0.458s', 'individual call time for EIGEN_LDLT: CPU, 1.500s; elapsed, 0.232s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.041s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.107845', '# - matching time = 0.034759', '# - symmetrization time = 0.0142128', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0269399', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.337 MB', '# - factor time = 0.354244', '# - factor nonzeros = 2,292,125', '# - factor memory = 18.337 MB', '# - total flops = 1.23995e+09, min = 1.9487e+08, max = 4.82511e+08', '# - flop rate = 3.50026 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29573e+09', '# --------------------------------------------', '# total = 1.29573e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.5656e-11\trel.res = 9.66757e-16\tbw.error = 1.65316e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0232182', '# - total flops = 7.0497e+06, min = 1.35975e+06, max = 2.14762e+06', '# - flop rate = 0.303629 GFlop/s', '# - bytes moved = 39.6928 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.70956 GByte/s', '# - solve arithmetic intensity = 0.177606 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.550s; elapsed, 0.578s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.550s; elapsed, 0.579s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.510s; elapsed, 0.573s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.640s; elapsed, 0.583s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.550s; elapsed, 0.579s, #calls: 1', 'TOTAL : CPU, 4.550s; elapsed, 0.579s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.550s; elapsed, 0.578s, #calls: 1', 'TOTAL : CPU, 4.550s; elapsed, 0.578s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.510s; elapsed, 0.573s, #calls: 1', 'TOTAL : CPU, 4.510s; elapsed, 0.573s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.490s; elapsed, 0.458s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.041s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.500s; elapsed, 0.232s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.640s; elapsed, 0.583s, #calls: 1', 'TOTAL : CPU, 9.870s; elapsed, 1.314s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 1768.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.100s; elapsed, 1.098s', 'individual call time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.242s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.068s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 66', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.0942969', '# - matching time = 0.0180972', '# - symmetrization time = 0.0117099', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.02071', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.337 MB', '# - factor time = 0.410468', '# - factor nonzeros = 2,292,125', '# - factor memory = 18.337 MB', '# - total flops = 1.23992e+09, min = 695096, max = 3.55062e+08', '# - flop rate = 3.02075 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29007e+09', '# --------------------------------------------', '# total = 1.29007e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.44884e-11\trel.res = 8.94653e-16\tbw.error = 1.6174e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0240459', '# - total flops = 9.317e+06, min = 26908, max = 2.00677e+06', '# - flop rate = 0.387466 GFlop/s', '# - bytes moved = 38.3707 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.59572 GByte/s', '# - solve arithmetic intensity = 0.242816 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.550s; elapsed, 0.589s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.570s; elapsed, 0.588s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.560s; elapsed, 0.588s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.570s; elapsed, 0.589s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.580s; elapsed, 0.590s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.570s; elapsed, 0.584s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.580s; elapsed, 0.588s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.590s; elapsed, 0.595s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.570s; elapsed, 0.588s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.588s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.560s; elapsed, 0.588s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.588s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.550s; elapsed, 0.589s, #calls: 1', 'TOTAL : CPU, 0.550s; elapsed, 0.589s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.570s; elapsed, 0.589s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.589s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.580s; elapsed, 0.590s, #calls: 1', 'TOTAL : CPU, 0.580s; elapsed, 0.590s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.580s; elapsed, 0.588s, #calls: 1', 'TOTAL : CPU, 0.580s; elapsed, 0.588s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.570s; elapsed, 0.584s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.584s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.100s; elapsed, 1.098s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.068s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.242s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.590s; elapsed, 0.595s, #calls: 1', 'TOTAL : CPU, 2.000s; elapsed, 2.002s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 1768.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.400s; elapsed, 0.719s', 'individual call time for EIGEN_LDLT: CPU, 0.440s; elapsed, 0.256s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.063s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.112944', '# - matching time = 0.021127', '# - symmetrization time = 0.0117459', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0252371', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.337 MB', '# - factor time = 0.391614', '# - factor nonzeros = 2,292,125', '# - factor memory = 18.337 MB', '# - total flops = 1.23993e+09, min = 695096, max = 3.55071e+08', '# - flop rate = 3.16621 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29007e+09', '# --------------------------------------------', '# total = 1.29007e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.4423e-11\trel.res = 8.90617e-16\tbw.error = 1.6174e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0187171', '# - total flops = 9.317e+06, min = 26908, max = 2.00677e+06', '# - flop rate = 0.497781 GFlop/s', '# - bytes moved = 38.3791 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.05049 GByte/s', '# - solve arithmetic intensity = 0.242762 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.160s; elapsed, 0.590s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.160s; elapsed, 0.594s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.110s; elapsed, 0.595s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.150s; elapsed, 0.596s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.050s; elapsed, 0.594s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.030s; elapsed, 0.594s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.170s; elapsed, 0.594s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.190s; elapsed, 0.600s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.160s; elapsed, 0.590s, #calls: 1', 'TOTAL : CPU, 1.160s; elapsed, 0.590s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.150s; elapsed, 0.596s, #calls: 1', 'TOTAL : CPU, 1.150s; elapsed, 0.596s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.160s; elapsed, 0.594s, #calls: 1', 'TOTAL : CPU, 1.160s; elapsed, 0.594s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.110s; elapsed, 0.595s, #calls: 1', 'TOTAL : CPU, 1.110s; elapsed, 0.595s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.170s; elapsed, 0.594s, #calls: 1', 'TOTAL : CPU, 1.170s; elapsed, 0.594s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.050s; elapsed, 0.594s, #calls: 1', 'TOTAL : CPU, 1.050s; elapsed, 0.594s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.030s; elapsed, 0.594s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.594s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.400s; elapsed, 0.719s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.063s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.440s; elapsed, 0.256s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.190s; elapsed, 0.600s, #calls: 1', 'TOTAL : CPU, 3.140s; elapsed, 1.638s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 1768.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.930s; elapsed, 0.500s', 'individual call time for EIGEN_LDLT: CPU, 0.800s; elapsed, 0.253s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.048s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threadsCPP -2', '', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.110517', '# - matching time = 0.0215931', '# - symmetrization time = 0.0132511', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0292461', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.337 MB', '# - factor time = 0.359902', '# - factor nonzeros = 2,292,125', '# - factor memory = 18.337 MB', '# - total flops = 1.23994e+09, min = 696756, max = 3.55075e+08', '# - flop rate = 3.44521 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29007e+09', '# --------------------------------------------', '# total = 1.29007e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.43497e-11\trel.res = 8.86089e-16\tbw.error = 1.61805e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0219939', '# - total flops = 9.317e+06, min = 26908, max = 2.00677e+06', '# - flop rate = 0.423618 GFlop/s', '# - bytes moved = 38.3901 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.74549 GByte/s', '# - solve arithmetic intensity = 0.242693 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.250s; elapsed, 0.566s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.010s; elapsed, 0.570s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.950s; elapsed, 0.570s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.250s; elapsed, 0.571sindividual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.210s; elapsed, 0.570s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.970s; elapsed, 0.570s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.240s; elapsed, 0.571s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.280s; elapsed, 0.576s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.210s; elapsed, 0.570s, #calls: 1', 'TOTAL : CPU, 2.210s; elapsed, 0.570s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.250s; elapsed, 0.566s, #calls: 1', 'TOTAL : CPU, 2.250s; elapsed, 0.566s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.240s; elapsed, 0.571s, #calls: 1', 'TOTAL : CPU, 2.240s; elapsed, 0.571s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.010s; elapsed, 0.570s, #calls: 1', 'TOTAL : CPU, 2.010s; elapsed, 0.570s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.950s; elapsed, 0.570s, #calls: 1', 'TOTAL : CPU, 1.950s; elapsed, 0.570s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.970s; elapsed, 0.570s, #calls: 1', 'TOTAL : CPU, 1.970s; elapsed, 0.570s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.250s; elapsed, 0.571s, #calls: 1', 'TOTAL : CPU, 2.250s; elapsed, 0.571s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.930s; elapsed, 0.500s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.048s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.800s; elapsed, 0.253s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.280s; elapsed, 0.576s, #calls: 1', 'TOTAL : CPU, 5.160s; elapsed, 1.377s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 884.4375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.110s; elapsed, 1.106s', 'individual call time for EIGEN_LDLT: CPU, 0.260s; elapsed, 0.259s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.078s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.109694', '# - matching time = 0.021348', '# - symmetrization time = 0.00631881', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.021131', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.337 MB', '# - factor time = 0.413345', '# - factor nonzeros = 2,292,125', '# - factor memory = 18.337 MB', '# - total flops = 1.23992e+09, min = 695096, max = 3.15217e+08', '# - flop rate = 2.99973 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.27889e+09', '# --------------------------------------------', '# total = 1.27889e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.32291e-11\trel.res = 8.16895e-16\tbw.error = 1.60964e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0228071', '# - total flops = 1.46854e+07, min = 22488, max = 1.96095e+06', '# - flop rate = 0.643895 GFlop/s', '# - bytes moved = 36.3296 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.59291 GByte/s', '# - solve arithmetic intensity = 0.404226 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.550s; elapsed, 0.604s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.580s; elapsed, 0.603s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.580s; elapsed, 0.602s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.570s; elapsed, 0.602s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.530s; elapsed, 0.603s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.570s; elapsed, 0.602s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.580s; elapsed, 0.603s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.570s; elapsed, 0.603s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.600s; elapsed, 0.604s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.530s; elapsed, 0.603s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.590s; elapsed, 0.602s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.590s; elapsed, 0.603s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.560s; elapsed, 0.602s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.530s; elapsed, 0.603s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.550s; elapsed, 0.604s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.610s; elapsed, 0.608s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.600s; elapsed, 0.604s, #calls: 1', 'TOTAL : CPU, 0.600s; elapsed, 0.604s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.570s; elapsed, 0.603s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.603s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.590s; elapsed, 0.602s, #calls: 1', 'TOTAL : CPU, 0.590s; elapsed, 0.602s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.530s; elapsed, 0.603s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.603s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.550s; elapsed, 0.604s, #calls: 1', 'TOTAL : CPU, 0.550s; elapsed, 0.604s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.570s; elapsed, 0.602s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.602s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.590s; elapsed, 0.603s, #calls: 1', 'TOTAL : CPU, 0.590s; elapsed, 0.603s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.580s; elapsed, 0.603s, #calls: 1', 'TOTAL : CPU, 0.580s; elapsed, 0.603s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.530s; elapsed, 0.603s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.603s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.580s; elapsed, 0.602s, #calls: 1', 'TOTAL : CPU, 0.580s; elapsed, 0.602s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.560s; elapsed, 0.602s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.602s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.570s; elapsed, 0.602s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.602s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.580s; elapsed, 0.603s, #calls: 1', 'TOTAL : CPU, 0.580s; elapsed, 0.603s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.530s; elapsed, 0.603s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.603s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.550s; elapsed, 0.604s, #calls: 1', 'TOTAL : CPU, 0.550s; elapsed, 0.604s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.110s; elapsed, 1.106s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.078s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.260s; elapsed, 0.259s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.610s; elapsed, 0.608s, #calls: 1', 'TOTAL : CPU, 2.060s; elapsed, 2.052s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 884.4375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.460s; elapsed, 0.738s', 'individual call time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.249s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.057s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.110598', '# - matching time = 0.0213292', '# - symmetrization time = 0.00794005', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0253968', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.337 MB', '# - factor time = 0.371187', '# - factor nonzeros = 2,292,125', '# - factor memory = 18.337 MB', '# - total flops = 1.23993e+09, min = 695096, max = 3.15217e+08', '# - flop rate = 3.34044 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.27889e+09', '# --------------------------------------------', '# total = 1.27889e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.36283e-11\trel.res = 8.41546e-16\tbw.error = 1.58062e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.01807', '# - total flops = 1.46854e+07, min = 22488, max = 1.96095e+06', '# - flop rate = 0.812695 GFlop/s', '# - bytes moved = 36.3336 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.01071 GByte/s', '# - solve arithmetic intensity = 0.404182 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.040s; elapsed, 0.567s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 1.120s; elapsed, 0.566s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.060s; elapsed, 0.566s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.080s; elapsed, 0.566s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.020s; elapsed, 0.566s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.020s; elapsed, 0.566s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.010s; elapsed, 0.567s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 1.100s; elapsed, 0.566s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 1.100s; elapsed, 0.567s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 1.110s; elapsed, 0.564s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 1.110s; elapsed, 0.566s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 1.120s; elapsed, 0.566s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.050s; elapsed, 0.567s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 1.000s; elapsed, 0.566s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 1.090s; elapsed, 0.566s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.130s; elapsed, 0.573s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.040s; elapsed, 0.567s, #calls: 1', 'TOTAL : CPU, 1.040s; elapsed, 0.567s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 1.120s; elapsed, 0.566s, #calls: 1', 'TOTAL : CPU, 1.120s; elapsed, 0.566s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.020s; elapsed, 0.566s, #calls: 1', 'TOTAL : CPU, 1.020s; elapsed, 0.566s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 1.000s; elapsed, 0.566s, #calls: 1', 'TOTAL : CPU, 1.000s; elapsed, 0.566s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 1.100s; elapsed, 0.566s, #calls: 1', 'TOTAL : CPU, 1.100s; elapsed, 0.566s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.080s; elapsed, 0.566s, #calls: 1', 'TOTAL : CPU, 1.080s; elapsed, 0.566s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 1.100s; elapsed, 0.567s, #calls: 1', 'TOTAL : CPU, 1.100s; elapsed, 0.567s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 1.110s; elapsed, 0.564s, #calls: 1', 'TOTAL : CPU, 1.110s; elapsed, 0.564s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.010s; elapsed, 0.567s, #calls: 1', 'TOTAL : CPU, 1.010s; elapsed, 0.567s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 1.110s; elapsed, 0.566s, #calls: 1', 'TOTAL : CPU, 1.110s; elapsed, 0.566s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.020s; elapsed, 0.566s, #calls: 1', 'TOTAL : CPU, 1.020s; elapsed, 0.566s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.060s; elapsed, 0.566s, #calls: 1', 'TOTAL : CPU, 1.060s; elapsed, 0.566s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 1.120s; elapsed, 0.566s, #calls: 1', 'TOTAL : CPU, 1.120s; elapsed, 0.566s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.050s; elapsed, 0.567s, #calls: 1', 'TOTAL : CPU, 1.050s; elapsed, 0.567s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 1.090s; elapsed, 0.566s, #calls: 1', 'TOTAL : CPU, 1.090s; elapsed, 0.566s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.460s; elapsed, 0.738s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.057s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.249s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.130s; elapsed, 0.573s, #calls: 1', 'TOTAL : CPU, 3.120s; elapsed, 1.617s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 13651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.950s; elapsed, 0.945s', 'individual call time for EIGEN_LDLT: CPU, 0.220s; elapsed, 0.225s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.066s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.950s; elapsed, 0.945s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.066s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.220s; elapsed, 0.225s, #calls: 1', 'TOTAL : CPU, 1.240s; elapsed, 1.237s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 13651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.250s; elapsed, 0.637s', 'individual call time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.220s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.056s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.250s; elapsed, 0.637s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.056s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.220s, #calls: 1', 'TOTAL : CPU, 1.760s; elapsed, 0.913s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 13651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.750s; elapsed, 0.454s', 'individual call time for EIGEN_LDLT: CPU, 0.790s; elapsed, 0.234s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.048s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.750s; elapsed, 0.454s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.048s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.790s; elapsed, 0.234s, #calls: 1', 'TOTAL : CPU, 2.700s; elapsed, 0.736s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 13651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.430s; elapsed, 0.451s', 'individual call time for EIGEN_LDLT: CPU, 1.500s; elapsed, 0.224s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.041s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.430s; elapsed, 0.451s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.041s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.500s; elapsed, 0.224s, #calls: 1', 'TOTAL : CPU, 5.180s; elapsed, 0.716s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 13651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.060s; elapsed, 0.469s', 'individual call time for EIGEN_LDLT: CPU, 3.050s; elapsed, 0.249s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.480s; elapsed, 0.040s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.060s; elapsed, 0.469s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.480s; elapsed, 0.040s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.050s; elapsed, 0.249s, #calls: 1', 'TOTAL : CPU, 10.590s; elapsed, 0.757s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 6825.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.990s; elapsed, 0.992s', 'individual call time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.233s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.069s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,057', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.091037', '# - matching time = 0.017674', '# - symmetrization time = 0.00653481', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,057', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.022315', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.3681 MB', '# - factor time = 0.59684', '# - factor nonzeros = 2,171,017', '# - factor memory = 17.3681 MB', '# - total flops = 1.1267e+09, min = 2.48891e+08, max = 8.77805e+08', '# - flop rate = 1.88777 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.18662e+09', '# --------------------------------------------', '# total = 1.18662e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.66736e-11\trel.res = 1.02959e-15\tbw.error = 2.06992e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.017998', '# - total flops = 5.32738e+06, min = 2.07883e+06, max = 3.24855e+06', '# - flop rate = 0.295999 GFlop/s', '# - bytes moved = 42.9647 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.38719 GByte/s', '# - solve arithmetic intensity = 0.123994 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.750s; elapsed, 0.762s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.770s; elapsed, 0.771s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.750s; elapsed, 0.762s, #calls: 1', 'TOTAL : CPU, 0.750s; elapsed, 0.762s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.990s; elapsed, 0.992s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.069s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.233s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.770s; elapsed, 0.771s, #calls: 1', 'TOTAL : CPU, 2.060s; elapsed, 2.065s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 6825.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.270s; elapsed, 0.648s', 'individual call time for EIGEN_LDLT: CPU, 0.400s; elapsed, 0.220s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.056s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,057', '# - number of levels = 68', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.085459', '# - matching time = 0.017695', '# - symmetrization time = 0.00746584', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,057', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0300748', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.3681 MB', '# - factor time = 0.345265', '# - factor nonzeros = 2,171,017', '# - factor memory = 17.3681 MB', '# - total flops = 1.12676e+09, min = 2.48891e+08, max = 8.77865e+08', '# - flop rate = 3.26345 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.18662e+09', '# --------------------------------------------', '# total = 1.18662e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.67788e-11\trel.res = 1.03609e-15\tbw.error = 2.06057e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0157819', '# - total flops = 5.32999e+06, min = 2.07883e+06, max = 3.25116e+06', '# - flop rate = 0.337728 GFlop/s', '# - bytes moved = 43.0716 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.72918 GByte/s', '# - solve arithmetic intensity = 0.123747 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.030s; elapsed, 0.515s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.040s; elapsed, 0.525s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.030s; elapsed, 0.515s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.515s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.270s; elapsed, 0.648s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.056s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.400s; elapsed, 0.220s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.040s; elapsed, 0.525s, #calls: 1', 'TOTAL : CPU, 2.810s; elapsed, 1.449s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 6825.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.780s; elapsed, 0.462s', 'individual call time for EIGEN_LDLT: CPU, 0.790s; elapsed, 0.235s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.050s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,057', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0854671', '# - matching time = 0.0176389', '# - symmetrization time = 0.00707698', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,057', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0251122', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.3681 MB', '# - factor time = 0.315094', '# - factor nonzeros = 2,171,017', '# - factor memory = 17.3681 MB', '# - total flops = 1.12676e+09, min = 2.48891e+08, max = 8.77866e+08', '# - flop rate = 3.57594 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.18662e+09', '# --------------------------------------------', '# total = 1.18662e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.66738e-11\trel.res = 1.02961e-15\tbw.error = 2.06014e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.018105', '# - total flops = 5.32999e+06, min = 2.07883e+06, max = 3.25116e+06', '# - flop rate = 0.294393 GFlop/s', '# - bytes moved = 43.0824 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.37958 GByte/s', '# - solve arithmetic intensity = 0.123716 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.910s; elapsed, 0.481s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.950s; elapsed, 0.490s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.910s; elapsed, 0.481s, #calls: 1', 'TOTAL : CPU, 1.910s; elapsed, 0.481s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.780s; elapsed, 0.462s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.050s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.790s; elapsed, 0.235s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.950s; elapsed, 0.490s, #calls: 1', 'TOTAL : CPU, 4.690s; elapsed, 1.237s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 6825.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.470s; elapsed, 0.456s', 'individual call time for EIGEN_LDLT: CPU, 1.490s; elapsed, 0.230s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.042s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,057', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0910499', '# - matching time = 0.0190332', '# - symmetrization time = 0.00712323', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,057', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0296152', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.3681 MB', '# - factor time = 0.33401', '# - factor nonzeros = 2,171,017', '# - factor memory = 17.3681 MB', '# - total flops = 1.1268e+09, min = 2.48892e+08, max = 8.77908e+08', '# - flop rate = 3.37355 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.18662e+09', '# --------------------------------------------', '# total = 1.18662e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.71226e-11\trel.res = 1.05732e-15\tbw.error = 2.05844e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.017072', '# - total flops = 5.32999e+06, min = 2.07883e+06, max = 3.25116e+06', '# - flop rate = 0.312207 GFlop/s', '# - bytes moved = 43.1071 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.52502 GByte/s', '# - solve arithmetic intensity = 0.123645 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.040s; elapsed, 0.512s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.160s; elapsed, 0.521s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.040s; elapsed, 0.512s, #calls: 1', 'TOTAL : CPU, 4.040s; elapsed, 0.512s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.470s; elapsed, 0.456s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.042s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.490s; elapsed, 0.230s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.160s; elapsed, 0.521s, #calls: 1', 'TOTAL : CPU, 9.390s; elapsed, 1.249s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 6825.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.690s; elapsed, 0.507s', 'individual call time for EIGEN_LDLT: CPU, 3.050s; elapsed, 0.230s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.038s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,057', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.10355', '# - matching time = 0.0289049', '# - symmetrization time = 0.00817704', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,057', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.037725', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.3681 MB', '# - factor time = 0.402765', '# - factor nonzeros = 2,171,017', '# - factor memory = 17.3681 MB', '# - total flops = 1.12684e+09, min = 2.48893e+08, max = 8.77944e+08', '# - flop rate = 2.79775 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.18662e+09', '# --------------------------------------------', '# total = 1.18662e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.65852e-11\trel.res = 1.02414e-15\tbw.error = 2.05914e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.022898', '# - total flops = 5.32999e+06, min = 2.07883e+06, max = 3.25116e+06', '# - flop rate = 0.232771 GFlop/s', '# - bytes moved = 43.1462 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.88428 GByte/s', '# - solve arithmetic intensity = 0.123533 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.240s; elapsed, 0.620s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.080s; elapsed, 0.631s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.240s; elapsed, 0.620s, #calls: 1', 'TOTAL : CPU, 9.240s; elapsed, 0.620s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.690s; elapsed, 0.507s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.038s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.050s; elapsed, 0.230s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.080s; elapsed, 0.631s, #calls: 1', 'TOTAL : CPU, 21.270s; elapsed, 1.406s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 3412.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.070s; elapsed, 1.072s', 'individual call time for EIGEN_LDLT: CPU, 0.250s; elapsed, 0.248s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.077s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,063', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.101225', '# - matching time = 0.0195701', '# - symmetrization time = 0.00991893', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.021662', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.6027 MB', '# - factor time = 0.369496', '# - factor nonzeros = 2,200,341', '# - factor memory = 17.6027 MB', '# - total flops = 1.17047e+09, min = 1.89709e+08, max = 4.51898e+08', '# - flop rate = 3.16775 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.22692e+09', '# --------------------------------------------', '# total = 1.22692e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.60273e-11\trel.res = 9.89682e-16\tbw.error = 2.4436e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0188692', '# - total flops = 6.73405e+06, min = 1.27554e+06, max = 2.06214e+06', '# - flop rate = 0.356881 GFlop/s', '# - bytes moved = 39.2925 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.08236 GByte/s', '# - solve arithmetic intensity = 0.171383 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.540s; elapsed, 0.551s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.550s; elapsed, 0.550s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.520s; elapsed, 0.546s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.550s; elapsed, 0.555s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.540s; elapsed, 0.551s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.551s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.520s; elapsed, 0.546s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.546s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.550s; elapsed, 0.550s, #calls: 1', 'TOTAL : CPU, 0.550s; elapsed, 0.550s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.070s; elapsed, 1.072s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.077s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.250s; elapsed, 0.248s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.550s; elapsed, 0.555s, #calls: 1', 'TOTAL : CPU, 1.950s; elapsed, 1.952s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 3412.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.440s; elapsed, 0.730s', 'individual call time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.240s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.057s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,063', '# - number of levels = 67', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.104636', '# - matching time = 0.0202429', '# - symmetrization time = 0.0104849', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0245519', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.6027 MB', '# - factor time = 0.331503', '# - factor nonzeros = 2,200,341', '# - factor memory = 17.6027 MB', '# - total flops = 1.17048e+09, min = 1.89709e+08, max = 4.51908e+08', '# - flop rate = 3.53084 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.22692e+09', '# --------------------------------------------', '# total = 1.22692e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.55832e-11\trel.res = 9.62261e-16\tbw.error = 2.45106e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.018975', '# - total flops = 6.73405e+06, min = 1.27554e+06, max = 2.06214e+06', '# - flop rate = 0.35489 GFlop/s', '# - bytes moved = 39.3012 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.07121 GByte/s', '# - solve arithmetic intensity = 0.171345 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.030s; elapsed, 0.526s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.030s; elapsed, 0.521s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.030s; elapsed, 0.525s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.060s; elapsed, 0.530s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.030s; elapsed, 0.526s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.526s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.030s; elapsed, 0.521s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.521s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.030s; elapsed, 0.525s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.525s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.440s; elapsed, 0.730s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.057s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.240s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.060s; elapsed, 0.530s, #calls: 1', 'TOTAL : CPU, 3.030s; elapsed, 1.558s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 3412.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 2.500s; elapsed, 0.642s', 'individual call time for EIGEN_LDLT: CPU, 0.780s; elapsed, 0.233s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.048s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,063', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.103655', '# - matching time = 0.022033', '# - symmetrization time = 0.0128319', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0294049', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.6027 MB', '# - factor time = 0.330995', '# - factor nonzeros = 2,200,341', '# - factor memory = 17.6027 MB', '# - total flops = 1.17048e+09, min = 1.89709e+08, max = 4.51908e+08', '# - flop rate = 3.53626 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.22692e+09', '# --------------------------------------------', '# total = 1.22692e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.5551e-11\trel.res = 9.60268e-16\tbw.error = 2.45161e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0187099', '# - total flops = 6.73405e+06, min = 1.27554e+06, max = 2.06214e+06', '# - flop rate = 0.359919 GFlop/s', '# - bytes moved = 39.3122 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.10115 GByte/s', '# - solve arithmetic intensity = 0.171297 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.090s; elapsed, 0.532s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.100s; elapsed, 0.531s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.080s; elapsed, 0.527s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.150s; elapsed, 0.536s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.090s; elapsed, 0.532s, #calls: 1', 'TOTAL : CPU, 2.090s; elapsed, 0.532s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.080s; elapsed, 0.527s, #calls: 1', 'TOTAL : CPU, 2.080s; elapsed, 0.527s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.100s; elapsed, 0.531s, #calls: 1', 'TOTAL : CPU, 2.100s; elapsed, 0.531s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 2.500s; elapsed, 0.642s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.048s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.780s; elapsed, 0.233s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.150s; elapsed, 0.536s, #calls: 1', 'TOTAL : CPU, 5.590s; elapsed, 1.458s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 3412.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.840s; elapsed, 0.504s', 'individual call time for EIGEN_LDLT: CPU, 1.570s; elapsed, 0.267s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.042s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', 'CPP -2', 'CPP -2', '8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,063', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.105834', '# - matching time = 0.0216041', '# - symmetrization time = 0.0103359', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.029871', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.6027 MB', '# - factor time = 0.370354', '# - factor nonzeros = 2,200,341', '# - factor memory = 17.6027 MB', '# - total flops = 1.17049e+09, min = 1.8971e+08, max = 4.51911e+08', '# - flop rate = 3.16046 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.22692e+09', '# --------------------------------------------', '# total = 1.22692e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.57108e-11\trel.res = 9.70137e-16\tbw.error = 2.45357e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.025686', '# - total flops = 6.73405e+06, min = 1.27554e+06, max = 2.06214e+06', '# - flop rate = 0.262168 GFlop/s', '# - bytes moved = 39.335 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.53138 GByte/s', '# - solve arithmetic intensity = 0.171197 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.560s; elapsed, 0.582s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.570s; elapsed, 0.580s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.550s; elapsed, 0.575s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.660s; elapsed, 0.584s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.560s; elapsed, 0.582s, #calls: 1', 'TOTAL : CPU, 4.560s; elapsed, 0.582s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.570s; elapsed, 0.580s, #calls: 1', 'TOTAL : CPU, 4.570s; elapsed, 0.580s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.550s; elapsed, 0.575s, #calls: 1', 'TOTAL : CPU, 4.550s; elapsed, 0.575s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.840s; elapsed, 0.504s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.042s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.570s; elapsed, 0.267s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.660s; elapsed, 0.584s, #calls: 1', 'TOTAL : CPU, 10.320s; elapsed, 1.397s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 1706.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.970s; elapsed, 0.975s', 'individual call time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.225s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.066s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', '1 OpenMP thread', 'CPP -2CPP -2CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '', '', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,063', '# - number of levels = 66', '# - nd time = 0.0857451', '# - matching time = 0.0164521', '# - symmetrization time = 0.00949192', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.019804', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.6027 MB', '# - factor time = 0.356274', '# - factor nonzeros = 2,200,341', '# - factor memory = 17.6027 MB', '# - total flops = 1.17047e+09, min = 908142, max = 2.96546e+08', '# - flop rate = 3.28532 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.22123e+09', '# --------------------------------------------', '# total = 1.22123e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.41958e-11\trel.res = 8.7659e-16\tbw.error = 2.56115e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.021194', '# - total flops = 8.86617e+06, min = 29121, max = 2.0318e+06', '# - flop rate = 0.418335 GFlop/s', '# - bytes moved = 37.7608 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.78168 GByte/s', '# - solve arithmetic intensity = 0.234798 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.490s; elapsed, 0.515s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.480s; elapsed, 0.516s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.510s; elapsed, 0.515s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.480s; elapsed, 0.517s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.500s; elapsed, 0.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.490s; elapsed, 0.515s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.500s; elapsed, 0.516s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.520s; elapsed, 0.522s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.500s; elapsed, 0.513s, #calls: 1', 'TOTAL : CPU, 0.500s; elapsed, 0.513s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.510s; elapsed, 0.515s, #calls: 1', 'TOTAL : CPU, 0.510s; elapsed, 0.515s', 'Exiting profiler', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.480s; elapsed, 0.517s, #calls: 1time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.490s; elapsed, 0.515s, #calls: 1', 'TOTAL : CPU, 0.490s; elapsed, 0.515s', '', 'TOTAL : CPU, 0.480s; elapsed, 0.517s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.480s; elapsed, 0.516s, #calls: 1', 'TOTAL : CPU, 0.480s; elapsed, 0.516s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.490s; elapsed, 0.515s, #calls: 1', 'TOTAL : CPU, 0.490s; elapsed, 0.515s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.500s; elapsed, 0.516s, #calls: 1', 'TOTAL : CPU, 0.500s; elapsed, 0.516s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.970s; elapsed, 0.975s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.066s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.225s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.520s; elapsed, 0.522s, #calls: 1', 'TOTAL : CPU, 1.780s; elapsed, 1.787s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 1706.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.300s; elapsed, 0.665s', 'individual call time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.246s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.062s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,063', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.109016', '# - matching time = 0.0209942', '# - symmetrization time = 0.022599', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0266368', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.6027 MB', '# - factor time = 0.376132', '# - factor nonzeros = 2,200,341', '# - factor memory = 17.6027 MB', '# - total flops = 1.17048e+09, min = 908142, max = 2.96551e+08', '# - flop rate = 3.11188 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.22123e+09', '# --------------------------------------------', '# total = 1.22123e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.41384e-11\trel.res = 8.73046e-16\tbw.error = 2.56115e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0201659', '# - total flops = 8.86617e+06, min = 29121, max = 2.0318e+06', '# - flop rate = 0.439661 GFlop/s', '# - bytes moved = 37.7669 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.87281 GByte/s', '# - solve arithmetic intensity = 0.23476 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.070s; elapsed, 0.591s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.160s; elapsed, 0.584s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.040s; elapsed, 0.590s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.150s; elapsed, 0.590s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.160s; elapsed, 0.589s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.120s; elapsed, 0.589s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.130s; elapsed, 0.590s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.190s; elapsed, 0.594s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.070s; elapsed, 0.591s, #calls: 1', 'TOTAL : CPU, 1.070s; elapsed, 0.591s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.160s; elapsed, 0.584s, #calls: 1', 'TOTAL : CPU, 1.160s; elapsed, 0.584s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.040s; elapsed, 0.590s, #calls: 1', 'TOTAL : CPU, 1.040s; elapsed, 0.590s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.120s; elapsed, 0.589s, #calls: 1', 'TOTAL : CPU, 1.120s; elapsed, 0.589s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.160s; elapsed, 0.589s, #calls: 1', 'TOTAL : CPU, 1.160s; elapsed, 0.589s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.150s; elapsed, 0.590s, #calls: 1', 'TOTAL : CPU, 1.150s; elapsed, 0.590s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.130s; elapsed, 0.590s, #calls: 1', 'TOTAL : CPU, 1.130s; elapsed, 0.590s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.300s; elapsed, 0.665s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.062s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.246s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.190s; elapsed, 0.594s, #calls: 1', 'TOTAL : CPU, 3.030s; elapsed, 1.567s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 1706.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.940s; elapsed, 0.504s', 'individual call time for EIGEN_LDLT: CPU, 0.800s; elapsed, 0.244s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.052s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,063', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.105714', '# - matching time = 0.0211971', '# - symmetrization time = 0.012078', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0268068', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.6027 MB', '# - factor time = 0.355683', '# - factor nonzeros = 2,200,341', '# - factor memory = 17.6027 MB', '# - total flops = 1.17048e+09, min = 909542, max = 2.96554e+08', '# - flop rate = 3.29081 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.22123e+09', '# --------------------------------------------', '# total = 1.22123e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.38528e-11\trel.res = 8.55407e-16\tbw.error = 2.55981e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.026298', '# - total flops = 8.86617e+06, min = 29121, max = 2.0318e+06', '# - flop rate = 0.337142 GFlop/s', '# - bytes moved = 37.7751 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.43642 GByte/s', '# - solve arithmetic intensity = 0.234709 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.900s; elapsed, 0.564s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.160s; elapsed, 0.561s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.990s; elapsed, 0.561s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.170s; elapsed, 0.560s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.940s; elapsed, 0.563sindividual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.180s; elapsed, 0.565s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.210s; elapsed, 0.558s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.250s; elapsed, 0.568s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.160s; elapsed, 0.561s, #calls: 1', 'TOTAL : CPU, 2.160s; elapsed, 0.561s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.900s; elapsed, 0.564s, #calls: 1', 'TOTAL : CPU, 1.900s; elapsed, 0.564s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.990s; elapsed, 0.561s, #calls: 1', 'TOTAL : CPU, 1.990s; elapsed, 0.561s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.170s; elapsed, 0.560s, #calls: 1', 'TOTAL : CPU, 2.170s; elapsed, 0.560s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.940s; elapsed, 0.563s, #calls: 1', 'TOTAL : CPU, 1.940s; elapsed, 0.563s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.210s; elapsed, 0.558s, #calls: 1', 'TOTAL : CPU, 2.210s; elapsed, 0.558s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.180s; elapsed, 0.565s, #calls: 1', 'TOTAL : CPU, 2.180s; elapsed, 0.565s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.940s; elapsed, 0.504s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.052s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.800s; elapsed, 0.244s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.250s; elapsed, 0.568s, #calls: 1', 'TOTAL : CPU, 5.170s; elapsed, 1.369s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 853.1875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.120s; elapsed, 1.128s', 'individual call time for EIGEN_LDLT: CPU, 0.280s; elapsed, 0.279s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.075s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,063', '# - number of levels = 65', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.093853', '# - matching time = 0.0185828', '# - symmetrization time = 0.010432', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0210679', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.6382 MB', '# - factor time = 0.378064', '# - factor nonzeros = 2,204,769', '# - factor memory = 17.6382 MB', '# - total flops = 1.17308e+09, min = 594238, max = 2.95348e+08', '# - flop rate = 3.10285 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.21323e+09', '# --------------------------------------------', '# total = 1.21323e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.285e-11\trel.res = 7.93488e-16\tbw.error = 1.53534e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.020797', '# - total flops = 1.19466e+07, min = 20012, max = 1.96858e+06', '# - flop rate = 0.574437 GFlop/s', '# - bytes moved = 36.41 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.75073 GByte/s', '# - solve arithmetic intensity = 0.328113 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.540s; elapsed, 0.552s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.520s; elapsed, 0.552s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.530s; elapsed, 0.552s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.520s; elapsed, 0.552s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.520s; elapsed, 0.554s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.510s; elapsed, 0.553s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.480s; elapsed, 0.552s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.540s; elapsed, 0.553s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.480s; elapsed, 0.552s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.500s; elapsed, 0.547s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.540s; elapsed, 0.552s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.480s; elapsed, 0.552s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.510s; elapsed, 0.552s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.540s; elapsed, 0.552s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.500s; elapsed, 0.553s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.550s; elapsed, 0.557s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.480s; elapsed, 0.552s, #calls: 1', 'TOTAL : CPU, 0.480s; elapsed, 0.552s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.540s; elapsed, 0.552s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.552s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.540s; elapsed, 0.552s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.552s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.520s; elapsed, 0.552s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.552s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.520s; elapsed, 0.552s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.552s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.520s; elapsed, 0.554s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.554s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.540s; elapsed, 0.553s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.553s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.480s; elapsed, 0.552s, #calls: 1', 'TOTAL : CPU, 0.480s; elapsed, 0.552s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.530s; elapsed, 0.552s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.552s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.510s; elapsed, 0.553s, #calls: 1', 'TOTAL : CPU, 0.510s; elapsed, 0.553s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.540s; elapsed, 0.552s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.552s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.510s; elapsed, 0.552s, #calls: 1', 'TOTAL : CPU, 0.510s; elapsed, 0.552s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.500s; elapsed, 0.547s, #calls: 1', 'TOTAL : CPU, 0.500s; elapsed, 0.547s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.500s; elapsed, 0.553s, #calls: 1', 'TOTAL : CPU, 0.500s; elapsed, 0.553s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.480s; elapsed, 0.552s, #calls: 1', 'TOTAL : CPU, 0.480s; elapsed, 0.552s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.120s; elapsed, 1.128s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.075s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.280s; elapsed, 0.279s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.550s; elapsed, 0.557s, #calls: 1', 'TOTAL : CPU, 2.030s; elapsed, 2.040s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 853.1875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.350s; elapsed, 0.687s', 'individual call time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.257s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.062s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 130,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,063', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.112594', '# - matching time = 0.02191', '# - symmetrization time = 0.010946', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,063', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0268061', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.6382 MB', '# - factor time = 0.322364', '# - factor nonzeros = 2,204,769', '# - factor memory = 17.6382 MB', '# - total flops = 1.17308e+09, min = 594238, max = 2.95348e+08', '# - flop rate = 3.639 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.21323e+09', '# --------------------------------------------', '# total = 1.21323e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.2812e-11\trel.res = 7.91139e-16\tbw.error = 1.53398e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.020294', '# - total flops = 1.19466e+07, min = 20012, max = 1.96858e+06', '# - flop rate = 0.588677 GFlop/s', '# - bytes moved = 36.4164 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.79445 GByte/s', '# - solve arithmetic intensity = 0.328055 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.030s; elapsed, 0.529s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.980s; elapsed, 0.529s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.980s; elapsed, 0.527s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.940s; elapsed, 0.529s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 1.030s; elapsed, 0.530s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.960s; elapsed, 0.529s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.990s; elapsed, 0.529s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.980s; elapsed, 0.528s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.980s; elapsed, 0.529s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 1.000s; elapsed, 0.531s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 1.030s; elapsed, 0.531s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 1.030s; elapsed, 0.524s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.960s; elapsed, 0.528s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 1.020s; elapsed, 0.529s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 1.030s; elapsed, 0.530s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.050s; elapsed, 0.534s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.030s; elapsed, 0.529s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.529s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 1.000s; elapsed, 0.531s, #calls: 1', 'TOTAL : CPU, 1.000s; elapsed, 0.531s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 1.030s; elapsed, 0.530s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.530s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.960s; elapsed, 0.529s, #calls: 1', 'TOTAL : CPU, 0.960s; elapsed, 0.529s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.980s; elapsed, 0.529s, #calls: 1', 'TOTAL : CPU, 0.980s; elapsed, 0.529s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.980s; elapsed, 0.529s, #calls: 1', 'TOTAL : CPU, 0.980s; elapsed, 0.529s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.980s; elapsed, 0.528s, #calls: 1', 'TOTAL : CPU, 0.980s; elapsed, 0.528s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 1.030s; elapsed, 0.531s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.531s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.940s; elapsed, 0.529s, #calls: 1', 'TOTAL : CPU, 0.940s; elapsed, 0.529s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 1.030s; elapsed, 0.524s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.524s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 1.020s; elapsed, 0.529s, #calls: 1', 'TOTAL : CPU, 1.020s; elapsed, 0.529s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.960s; elapsed, 0.528s, #calls: 1', 'TOTAL : CPU, 0.960s; elapsed, 0.528s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.980s; elapsed, 0.527s, #calls: 1', 'TOTAL : CPU, 0.980s; elapsed, 0.527s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 1.030s; elapsed, 0.530s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.530s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.990s; elapsed, 0.529s, #calls: 1', 'TOTAL : CPU, 0.990s; elapsed, 0.529s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.350s; elapsed, 0.687s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.062s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.257s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.050s; elapsed, 0.534s, #calls: 1', 'TOTAL : CPU, 2.950s; elapsed, 1.540s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 13651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.340s; elapsed, 0.333s', 'individual call time for EIGEN_LDLT: CPU, 0.040s; elapsed, 0.047s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.025s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.340s; elapsed, 0.333s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.025s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.040s; elapsed, 0.047s, #calls: 1', 'TOTAL : CPU, 0.410s; elapsed, 0.404s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 13651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.480s; elapsed, 0.245s', 'individual call time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.048s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.022s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.480s; elapsed, 0.245s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.022s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.048s, #calls: 1', 'TOTAL : CPU, 0.630s; elapsed, 0.315s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 13651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.810s; elapsed, 0.212s', 'individual call time for EIGEN_LDLT: CPU, 0.190s; elapsed, 0.049s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.021s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.810s; elapsed, 0.212s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.190s; elapsed, 0.049s, #calls: 1', 'TOTAL : CPU, 1.090s; elapsed, 0.282s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 13651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.550s; elapsed, 0.207s', 'individual call time for EIGEN_LDLT: CPU, 0.580s; elapsed, 0.074s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.021s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.550s; elapsed, 0.207s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.580s; elapsed, 0.074s, #calls: 1', 'TOTAL : CPU, 2.300s; elapsed, 0.301s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 13651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 2.990s; elapsed, 0.204s', 'individual call time for EIGEN_LDLT: CPU, 0.810s; elapsed, 0.050s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.019s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 2.990s; elapsed, 0.204s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.300s; elapsed, 0.019s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.810s; elapsed, 0.050s, #calls: 1', 'TOTAL : CPU, 4.100s; elapsed, 0.274s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 6825.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.360s; elapsed, 0.364s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.049s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.028s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0804002', '# - matching time = 0.011189', '# - symmetrization time = 0.00332999', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.014451', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.19591 MB', '# - factor time = 0.179167', '# - factor nonzeros = 899,489', '# - factor memory = 7.19591 MB', '# - total flops = 2.17387e+08, min = 3.70215e+07, max = 1.80366e+08', '# - flop rate = 1.21332 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.31866e+08', '# --------------------------------------------', '# total = 2.31866e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.24011e-11\trel.res = 8.09365e-16\tbw.error = 4.06316e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0134962', '# - total flops = 2.3471e+06, min = 856409, max = 1.49069e+06', '# - flop rate = 0.173909 GFlop/s', '# - bytes moved = 21.3289 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.58037 GByte/s', '# - solve arithmetic intensity = 0.110043 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.300s; elapsed, 0.309s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.300s; elapsed, 0.314s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.300s; elapsed, 0.309s, #calls: 1', 'TOTAL : CPU, 0.300s; elapsed, 0.309s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.360s; elapsed, 0.364s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.028s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.049s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.300s; elapsed, 0.314s, #calls: 1', 'TOTAL : CPU, 0.740s; elapsed, 0.754s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 6825.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.480s; elapsed, 0.246s', 'individual call time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.049s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.023s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.081439', '# - matching time = 0.011272', '# - symmetrization time = 0.0036931', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.015671', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.19591 MB', '# - factor time = 0.110079', '# - factor nonzeros = 899,489', '# - factor memory = 7.19591 MB', '# - total flops = 2.17416e+08, min = 3.70215e+07, max = 1.80395e+08', '# - flop rate = 1.97509 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.31866e+08', '# --------------------------------------------', '# total = 2.31866e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.18614e-11\trel.res = 7.74142e-16\tbw.error = 3.26284e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.00935507', '# - total flops = 2.34774e+06, min = 856409, max = 1.49133e+06', '# - flop rate = 0.250959 GFlop/s', '# - bytes moved = 21.3528 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.28248 GByte/s', '# - solve arithmetic intensity = 0.10995 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.460s; elapsed, 0.240s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.490s; elapsed, 0.243s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.460s; elapsed, 0.240s, #calls: 1', 'TOTAL : CPU, 0.460s; elapsed, 0.240s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.480s; elapsed, 0.246s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.049s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.490s; elapsed, 0.243s, #calls: 1', 'TOTAL : CPU, 1.110s; elapsed, 0.562s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 6825.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.070s; elapsed, 0.281s', 'individual call time for EIGEN_LDLT: CPU, 0.250s; elapsed, 0.062s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.020s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3CPP -2', '', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0914109', '# - matching time = 0.012269', '# - symmetrization time = 0.00415492', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0172369', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.19591 MB', '# - factor time = 0.120228', '# - factor nonzeros = 899,489', '# - factor memory = 7.19591 MB', '# - total flops = 2.17437e+08, min = 3.70215e+07, max = 1.80416e+08', '# - flop rate = 1.80854 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.31866e+08', '# --------------------------------------------', '# total = 2.31866e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.18802e-11\trel.res = 7.75366e-16\tbw.error = 3.26284e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0108619', '# - total flops = 2.34774e+06, min = 856409, max = 1.49133e+06', '# - flop rate = 0.216145 GFlop/s', '# - bytes moved = 21.364 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.96688 GByte/s', '# - solve arithmetic intensity = 0.109892 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.030s; elapsed, 0.266s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.070s; elapsed, 0.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.030s; elapsed, 0.266s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.266s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.070s; elapsed, 0.281s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.020s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.250s; elapsed, 0.062s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.070s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 2.470s; elapsed, 0.632s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 6825.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.740s; elapsed, 0.228s', 'individual call time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.050s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.020s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.091553', '# - matching time = 0.0128579', '# - symmetrization time = 0.00370002', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0169401', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.19591 MB', '# - factor time = 0.0946472', '# - factor nonzeros = 899,489', '# - factor memory = 7.19591 MB', '# - total flops = 2.17439e+08, min = 3.70231e+07, max = 1.80416e+08', '# - flop rate = 2.29737 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.31866e+08', '# --------------------------------------------', '# total = 2.31866e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.16124e-11\trel.res = 7.57889e-16\tbw.error = 3.26284e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.010365', '# - total flops = 2.34774e+06, min = 856409, max = 1.49133e+06', '# - flop rate = 0.226506 GFlop/s', '# - bytes moved = 21.3673 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.06148 GByte/s', '# - solve arithmetic intensity = 0.109876 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.880s; elapsed, 0.239s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.940s; elapsed, 0.243s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.880s; elapsed, 0.239s, #calls: 1', 'TOTAL : CPU, 1.880s; elapsed, 0.239s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.740s; elapsed, 0.228s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.020s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.050s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.940s; elapsed, 0.243s, #calls: 1', 'TOTAL : CPU, 4.250s; elapsed, 0.542s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 6825.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.850s; elapsed, 0.258s', 'individual call time for EIGEN_LDLT: CPU, 0.890s; elapsed, 0.056s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.340s; elapsed, 0.021s', 'CPP -2', '# Initializing STRUMPACK', '# using 16CPP -2', ' OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.101624', '# - matching time = 0.0140259', '# - symmetrization time = 0.0042479', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.021292', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.19591 MB', '# - factor time = 0.099273', '# - factor nonzeros = 899,489', '# - factor memory = 7.19591 MB', '# - total flops = 2.17444e+08, min = 3.70246e+07, max = 1.8042e+08', '# - flop rate = 2.19037 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.31866e+08', '# --------------------------------------------', '# total = 2.31866e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.16336e-11\trel.res = 7.59275e-16\tbw.error = 3.26284e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.011379', '# - total flops = 2.34774e+06, min = 856409, max = 1.49133e+06', '# - flop rate = 0.206322 GFlop/s', '# - bytes moved = 21.3718 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.87818 GByte/s', '# - solve arithmetic intensity = 0.109852 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.120s; elapsed, 0.266s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.310s; elapsed, 0.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.120s; elapsed, 0.266s, #calls: 1', 'TOTAL : CPU, 4.120s; elapsed, 0.266s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.850s; elapsed, 0.258s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.340s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.890s; elapsed, 0.056s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.310s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 9.390s; elapsed, 0.605s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 3412.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.380s; elapsed, 0.389s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.053s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.030s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.081346', '# - matching time = 0.0118451', '# - symmetrization time = 0.00834489', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0127261', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.05593 MB', '# - factor time = 0.117368', '# - factor nonzeros = 881,991', '# - factor memory = 7.05593 MB', '# - total flops = 2.06216e+08, min = 3.70215e+07, max = 8.34426e+07', '# - flop rate = 1.75701 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.19939e+08', '# --------------------------------------------', '# total = 2.19939e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.13219e-11\trel.res = 7.38927e-16\tbw.error = 1.08941e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.012624', '# - total flops = 2.70089e+06, min = 511448, max = 839344', '# - flop rate = 0.213948 GFlop/s', '# - bytes moved = 19.8444 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.57196 GByte/s', '# - solve arithmetic intensity = 0.136103 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.230s; elapsed, 0.250s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.250s; elapsed, 0.258s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.240s; elapsed, 0.258s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.260s; elapsed, 0.261s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.240s; elapsed, 0.258s, #calls: 1', 'TOTAL : CPU, 0.240s; elapsed, 0.258s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.250s; elapsed, 0.258s, #calls: 1', 'TOTAL : CPU, 0.250s; elapsed, 0.258s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.230s; elapsed, 0.250s, #calls: 1', 'TOTAL : CPU, 0.230s; elapsed, 0.250s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.380s; elapsed, 0.389s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.040s; elapsed, 0.030s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.053s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.260s; elapsed, 0.261s, #calls: 1', 'TOTAL : CPU, 0.730s; elapsed, 0.733s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 3412.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.570s; elapsed, 0.287s', 'individual call time for EIGEN_LDLT: CPU, 0.120s; elapsed, 0.059s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.023s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0920041', '# - matching time = 0.0144989', '# - symmetrization time = 0.0052309', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.016155', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.05593 MB', '# - factor time = 0.108325', '# - factor nonzeros = 881,991', '# - factor memory = 7.05593 MB', '# - total flops = 2.06223e+08, min = 3.70215e+07, max = 8.34491e+07', '# - flop rate = 1.90374 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.19939e+08', '# --------------------------------------------', '# total = 2.19939e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.1215e-11\trel.res = 7.31952e-16\tbw.error = 1.10174e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0117881', '# - total flops = 2.70089e+06, min = 511448, max = 839344', '# - flop rate = 0.229119 GFlop/s', '# - bytes moved = 19.8514 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.68402 GByte/s', '# - solve arithmetic intensity = 0.136055 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.510s; elapsed, 0.257s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.510s; elapsed, 0.259s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.500s; elapsed, 0.257s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.520s; elapsed, 0.260s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.510s; elapsed, 0.257s, #calls: 1', 'TOTAL : CPU, 0.510s; elapsed, 0.257s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.510s; elapsed, 0.259s, #calls: 1', 'TOTAL : CPU, 0.510s; elapsed, 0.259s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.500s; elapsed, 0.257s, #calls: 1', 'TOTAL : CPU, 0.500s; elapsed, 0.257s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.570s; elapsed, 0.287s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.120s; elapsed, 0.059s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.520s; elapsed, 0.260s, #calls: 1', 'TOTAL : CPU, 1.260s; elapsed, 0.630s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 3412.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.880s; elapsed, 0.232s', 'individual call time for EIGEN_LDLT: CPU, 0.210s; elapsed, 0.051s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.021s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threadsCPP -2', 'CPP -2', 'CPP -2', '', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,# - nd time = ', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.0828328', '# - matching time = 0.0120008', '# - symmetrization time = 0.00544786', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0141001', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.05593 MB', '# - factor time = 0.094934', '# - factor nonzeros = 881,991', '# - factor memory = 7.05593 MB', '# - total flops = 2.06223e+08, min = 3.70215e+07, max = 8.34491e+07', '# - flop rate = 2.17227 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.19939e+08', '# --------------------------------------------', '# total = 2.19939e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.12111e-11\trel.res = 7.317e-16\tbw.error = 1.10174e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.011755', '# - total flops = 2.70089e+06, min = 511448, max = 839344', '# - flop rate = 0.229765 GFlop/s', '# - bytes moved = 19.8548 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.68905 GByte/s', '# - solve arithmetic intensity = 0.136032 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.900s; elapsed, 0.228s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.900s; elapsed, 0.231s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.910s; elapsed, 0.231s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.930s; elapsed, 0.232s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.910s; elapsed, 0.231s, #calls: 1', 'TOTAL : CPU, 0.910s; elapsed, 0.231s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.900s; elapsed, 0.228s, #calls: 1', 'TOTAL : CPU, 0.900s; elapsed, 0.228s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.900s; elapsed, 0.231s, #calls: 1', 'TOTAL : CPU, 0.900s; elapsed, 0.231s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.880s; elapsed, 0.232s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.210s; elapsed, 0.051s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.930s; elapsed, 0.232s, #calls: 1', 'TOTAL : CPU, 2.110s; elapsed, 0.537s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 3412.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.680s; elapsed, 0.223s', 'individual call time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.050s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.020s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.091676', '# - matching time = 0.0160999', '# - symmetrization time = 0.005445', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.016166', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.05593 MB', '# - factor time = 0.120134', '# - factor nonzeros = 881,991', '# - factor memory = 7.05593 MB', '# - total flops = 2.06229e+08, min = 3.70231e+07, max = 8.34514e+07', '# - flop rate = 1.71666 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.19939e+08', '# --------------------------------------------', '# total = 2.19939e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.13871e-11\trel.res = 7.43184e-16\tbw.error = 1.10247e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.013006', '# - total flops = 2.70089e+06, min = 511448, max = 839344', '# - flop rate = 0.207665 GFlop/s', '# - bytes moved = 19.8621 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.52715 GByte/s', '# - solve arithmetic intensity = 0.135982 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.110s; elapsed, 0.273s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.120s; elapsed, 0.272s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.130s; elapsed, 0.270s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.200s; elapsed, 0.274s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.110s; elapsed, 0.273s, #calls: 1', 'TOTAL : CPU, 2.110s; elapsed, 0.273s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.120s; elapsed, 0.272s, #calls: 1', 'TOTAL : CPU, 2.120s; elapsed, 0.272s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.130s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 2.130s; elapsed, 0.270s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.680s; elapsed, 0.223s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.020s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.050s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.200s; elapsed, 0.274s, #calls: 1', 'TOTAL : CPU, 4.450s; elapsed, 0.568s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 1706.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.420s; elapsed, 0.420s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.056s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.030s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0940499', '# - matching time = 0.012903', '# - symmetrization time = 0.00532103', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0144031', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.05593 MB', '# - factor time = 0.11751', '# - factor nonzeros = 881,991', '# - factor memory = 7.05593 MB', '# - total flops = 2.06216e+08, min = 225576, max = 6.3946e+07', '# - flop rate = 1.75488 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.18436e+08', '# --------------------------------------------', '# total = 2.18436e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.06023e-11\trel.res = 6.91968e-16\tbw.error = 1.08348e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0140209', '# - total flops = 3.41102e+06, min = 19352, max = 819391', '# - flop rate = 0.243281 GFlop/s', '# - bytes moved = 19.4123 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.38452 GByte/s', '# - solve arithmetic intensity = 0.175715 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.260s; elapsed, 0.266s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.260s; elapsed, 0.268s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.220s; elapsed, 0.266s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.240s; elapsed, 0.265s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.240s; elapsed, 0.263s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.240s; elapsed, 0.266s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.220s; elapsed, 0.268s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.270s; elapsed, 0.269s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.260s; elapsed, 0.268s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.268s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.260s; elapsed, 0.266s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.266s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.220s; elapsed, 0.266s, #calls: 1', 'TOTAL : CPU, 0.220s; elapsed, 0.266s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.240s; elapsed, 0.265s, #calls: 1', 'TOTAL : CPU, 0.240s; elapsed, 0.265s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.240s; elapsed, 0.263s, #calls: 1', 'TOTAL : CPU, 0.240s; elapsed, 0.263s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.240s; elapsed, 0.266s, #calls: 1', 'TOTAL : CPU, 0.240s; elapsed, 0.266s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.220s; elapsed, 0.268s, #calls: 1', 'TOTAL : CPU, 0.220s; elapsed, 0.268s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.420s; elapsed, 0.420s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.030s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.056s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.270s; elapsed, 0.269s, #calls: 1', 'TOTAL : CPU, 0.770s; elapsed, 0.774s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 1706.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.570s; elapsed, 0.289s', 'individual call time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.056s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.024s', 'CPP -2', '# Initializing STRUMPACK', '# using 2CPP -2', 'CPP -2', ' OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0959358', '# - matching time = 0.0137439', '# - symmetrization time = 0.00611806', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0161719', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.05593 MB', '# - factor time = 0.155462', '# - factor nonzeros = 881,991', '# - factor memory = 7.05593 MB', '# - total flops = 2.0622e+08, min = 225576, max = 6.39502e+07', '# - flop rate = 1.3265 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.18436e+08', '# --------------------------------------------', '# total = 2.18436e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.05418e-11\trel.res = 6.88014e-16\tbw.error = 1.08348e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0141349', '# - total flops = 3.41102e+06, min = 19352, max = 819391', '# - flop rate = 0.241319 GFlop/s', '# - bytes moved = 19.4183 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.37378 GByte/s', '# - solve arithmetic intensity = 0.175661 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.620s; elapsed, 0.315s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.620s; elapsed, 0.315s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.600s; elapsed, 0.314s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.630s; elapsed, 0.318s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.610s; elapsed, 0.316s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.630s; elapsed, 0.316s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.610s; elapsed, 0.316s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.640s; elapsed, 0.321s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.630s; elapsed, 0.316s, #calls: 1', 'TOTAL : CPU, 0.630s; elapsed, 0.316s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.620s; elapsed, 0.315s, #calls: 1', 'TOTAL : CPU, 0.620s; elapsed, 0.315s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.630s; elapsed, 0.318s, #calls: 1', 'TOTAL : CPU, 0.630s; elapsed, 0.318s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.600s; elapsed, 0.314s, #calls: 1', 'TOTAL : CPU, 0.600s; elapsed, 0.314s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.610s; elapsed, 0.316s, #calls: 1', 'TOTAL : CPU, 0.610s; elapsed, 0.316s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.610s; elapsed, 0.316s, #calls: 1', 'TOTAL : CPU, 0.610s; elapsed, 0.316s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.620s; elapsed, 0.315s, #calls: 1', 'TOTAL : CPU, 0.620s; elapsed, 0.315s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.570s; elapsed, 0.289s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.024s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.056s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.640s; elapsed, 0.321s, #calls: 1', 'TOTAL : CPU, 1.370s; elapsed, 0.691s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 1706.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.950s; elapsed, 0.250s', 'individual call time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.059s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.023s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', 'CPP -2', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0988181', '# - matching time = 0.0189059', '# - symmetrization time = 0.00658202', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0182159', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.05593 MB', '# - factor time = 0.136388', '# - factor nonzeros = 881,991', '# - factor memory = 7.05593 MB', '# - total flops = 2.06227e+08, min = 227131, max = 6.39525e+07', '# - flop rate = 1.51206 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.18436e+08', '# --------------------------------------------', '# total = 2.18436e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.07523e-11\trel.res = 7.01753e-16\tbw.error = 1.08377e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.014482', '# - total flops = 3.41102e+06, min = 19352, max = 819391', '# - flop rate = 0.235535 GFlop/s', '# - bytes moved = 19.4256 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.34136 GByte/s', '# - solve arithmetic intensity = 0.175594 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.130s; elapsed, 0.309s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.220s; elapsed, 0.310sindividual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.230s; elapsed, 0.313s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.220s; elapsed, 0.310s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.180s; elapsed, 0.308s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.200s; elapsed, 0.310s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.150s; elapsed, 0.311s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.250s; elapsed, 0.315s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.230s; elapsed, 0.313s, #calls: 1', 'TOTAL : CPU, 1.230s; elapsed, 0.313s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.130s; elapsed, 0.309s, #calls: 1', 'TOTAL : CPU, 1.130s; elapsed, 0.309s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.220s; elapsed, 0.310s, #calls: 1', 'TOTAL : CPU, 1.220s; elapsed, 0.310s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.220s; elapsed, 0.310s, #calls: 1', 'TOTAL : CPU, 1.220s; elapsed, 0.310s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.180s; elapsed, 0.308s, #calls: 1', 'TOTAL : CPU, 1.180s; elapsed, 0.308s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.200s; elapsed, 0.310s, #calls: 1', 'TOTAL : CPU, 1.200s; elapsed, 0.310s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.150s; elapsed, 0.311s, #calls: 1', 'TOTAL : CPU, 1.150s; elapsed, 0.311s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.950s; elapsed, 0.250s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.059s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.250s; elapsed, 0.315s, #calls: 1', 'TOTAL : CPU, 2.530s; elapsed, 0.648s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 853.1875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.410s; elapsed, 0.412s', 'individual call time for EIGEN_LDLT: CPU, 0.070s; elapsed, 0.056s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.029s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.091644', '# - matching time = 0.013088', '# - symmetrization time = 0.00437808', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0152209', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.05593 MB', '# - factor time = 0.149521', '# - factor nonzeros = 881,991', '# - factor memory = 7.05593 MB', '# - total flops = 2.06216e+08, min = 172869, max = 4.26352e+07', '# - flop rate = 1.37918 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.15638e+08', '# --------------------------------------------', '# total = 2.15638e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.75506e-12\trel.res = 6.36669e-16\tbw.error = 1.09812e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0146589', '# - total flops = 4.7536e+06, min = 13772, max = 794351', '# - flop rate = 0.32428 GFlop/s', '# - bytes moved = 18.5117 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.26283 GByte/s', '# - solve arithmetic intensity = 0.256789 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.270s; elapsed, 0.299s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.290s; elapsed, 0.302s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.250s; elapsed, 0.303s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.270s; elapsed, 0.298s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.240s; elapsed, 0.301s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.290s; elapsed, 0.299s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.280s; elapsed, 0.300s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.260s; elapsed, 0.300s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.270s; elapsed, 0.300s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.260s; elapsed, 0.300s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.240s; elapsed, 0.299s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.250s; elapsed, 0.300s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.240s; elapsed, 0.302s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.270s; elapsed, 0.298s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.260s; elapsed, 0.299s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.290s; elapsed, 0.303s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.270s; elapsed, 0.299s, #calls: 1', 'TOTAL : CPU, 0.270s; elapsed, 0.299s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.250s; elapsed, 0.303s, #calls: 1', 'TOTAL : CPU, 0.250s; elapsed, 0.303s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.260s; elapsed, 0.300s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.300s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.240s; elapsed, 0.299s, #calls: 1', 'TOTAL : CPU, 0.240s; elapsed, 0.299s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.290s; elapsed, 0.302s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.302s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.270s; elapsed, 0.298s, #calls: 1', 'TOTAL : CPU, 0.270s; elapsed, 0.298s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.260s; elapsed, 0.300s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.300s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.290s; elapsed, 0.299s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.299s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.240s; elapsed, 0.301s, #calls: 1', 'TOTAL : CPU, 0.240s; elapsed, 0.301s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.240s; elapsed, 0.302s, #calls: 1', 'TOTAL : CPU, 0.240s; elapsed, 0.302s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.280s; elapsed, 0.300s, #calls: 1', 'TOTAL : CPU, 0.280s; elapsed, 0.300s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.260s; elapsed, 0.299s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.299s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.270s; elapsed, 0.298s, #calls: 1Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.250s; elapsed, 0.300s, #calls: 1', 'TOTAL : CPU, 0.250s; elapsed, 0.300s', '', 'TOTAL : CPU, 0.270s; elapsed, 0.298s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.270s; elapsed, 0.300s, #calls: 1', 'TOTAL : CPU, 0.270s; elapsed, 0.300s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.410s; elapsed, 0.412s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.029s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.070s; elapsed, 0.056s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.290s; elapsed, 0.303s, #calls: 1', 'TOTAL : CPU, 0.800s; elapsed, 0.800s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '13651 853.1875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.560s; elapsed, 0.284s', 'individual call time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.051s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.024s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 13,651', '# - number of nonzeros = 71,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0937591', '# - matching time = 0.0133018', '# - symmetrization time = 0.00445914', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.016326', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.05593 MB', '# - factor time = 0.123868', '# - factor nonzeros = 881,991', '# - factor memory = 7.05593 MB', '# - total flops = 2.06219e+08, min = 172869, max = 4.26352e+07', '# - flop rate = 1.66483 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.15638e+08', '# --------------------------------------------', '# total = 2.15638e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 9.7554e-12\trel.res = 6.36691e-16\tbw.error = 1.09812e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0104358', '# - total flops = 4.7536e+06, min = 13772, max = 794351', '# - flop rate = 0.455508 GFlop/s', '# - bytes moved = 18.5136 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.77404 GByte/s', '# - solve arithmetic intensity = 0.256763 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.530s; elapsed, 0.272s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.510s; elapsed, 0.272s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.520s; elapsed, 0.272s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.510s; elapsed, 0.272s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.510s; elapsed, 0.272s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.540s; elapsed, 0.273s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.520s; elapsed, 0.272s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.520s; elapsed, 0.271s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.520s; elapsed, 0.268s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.510s; elapsed, 0.271s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.510s; elapsed, 0.271s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.510s; elapsed, 0.274s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.510s; elapsed, 0.274s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.530s; elapsed, 0.271s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.530s; elapsed, 0.271s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.540s; elapsed, 0.277s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.510s; elapsed, 0.274s, #calls: 1', 'TOTAL : CPU, 0.510s; elapsed, 0.274s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.520s; elapsed, 0.272s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.272s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.510s; elapsed, 0.272s, #calls: 1', 'TOTAL : CPU, 0.510s; elapsed, 0.272s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.530s; elapsed, 0.272s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.272s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.530s; elapsed, 0.271s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.271s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.510s; elapsed, 0.272s, #calls: 1', 'TOTAL : CPU, 0.510s; elapsed, 0.272s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.520s; elapsed, 0.271s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.271s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.520s; elapsed, 0.268s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.268s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.530s; elapsed, 0.271s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.271s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.520s; elapsed, 0.272s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.272s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.510s; elapsed, 0.272s, #calls: 1', 'TOTAL : CPU, 0.510s; elapsed, 0.272s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.510s; elapsed, 0.271s, #calls: 1Exiting profiler', '', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.510s; elapsed, 0.271s, #calls: 1', 'TOTAL : CPU, 0.510s; elapsed, 0.271s', 'TOTAL : CPU, 0.510s; elapsed, 0.271s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.540s; elapsed, 0.273s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.273s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.510s; elapsed, 0.274s, #calls: 1', 'TOTAL : CPU, 0.510s; elapsed, 0.274s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.560s; elapsed, 0.284s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.024s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.051s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.540s; elapsed, 0.277s, #calls: 1', 'TOTAL : CPU, 1.250s; elapsed, 0.636s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 15151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.930s; elapsed, 0.929s', 'individual call time for EIGEN_LDLT: CPU, 0.220s; elapsed, 0.225s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.066s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.930s; elapsed, 0.929s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.066s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.220s; elapsed, 0.225s, #calls: 1', 'TOTAL : CPU, 1.220s; elapsed, 1.220s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 15151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.160s; elapsed, 0.596s', 'individual call time for EIGEN_LDLT: CPU, 0.420s; elapsed, 0.234s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.051s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.160s; elapsed, 0.596s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.051s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.420s; elapsed, 0.234s, #calls: 1', 'TOTAL : CPU, 1.670s; elapsed, 0.880s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 15151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 2.020s; elapsed, 0.522s', 'individual call time for EIGEN_LDLT: CPU, 0.800s; elapsed, 0.233s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.045s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 2.020s; elapsed, 0.522s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.045s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.800s; elapsed, 0.233s, #calls: 1', 'TOTAL : CPU, 2.970s; elapsed, 0.800s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 15151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.280s; elapsed, 0.431s', 'individual call time for EIGEN_LDLT: CPU, 1.530s; elapsed, 0.245s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.042s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.280s; elapsed, 0.431s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.042s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.530s; elapsed, 0.245s, #calls: 1', 'TOTAL : CPU, 5.060s; elapsed, 0.717s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 15151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.730s; elapsed, 0.446s', 'individual call time for EIGEN_LDLT: CPU, 2.960s; elapsed, 0.244s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.040s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.730s; elapsed, 0.446s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.470s; elapsed, 0.040s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.960s; elapsed, 0.244s, #calls: 1', 'TOTAL : CPU, 10.160s; elapsed, 0.730s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 7575.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.030s; elapsed, 1.030s', 'individual call time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.241s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.076s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 15,151', '# - number of nonzeros = 132,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0920248', '# - matching time = 0.016768', '# - symmetrization time = 0.00681901', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0215611', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.3274 MB', '# - factor time = 0.581437', '# - factor nonzeros = 2,290,919', '# - factor memory = 18.3274 MB', '# - total flops = 1.18081e+09, min = 1.91931e+08, max = 9.88882e+08', '# - flop rate = 2.03085 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.23655e+09', '# --------------------------------------------', '# total = 1.23655e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.02213e-11\trel.res = 1.24866e-15\tbw.error = 2.16324e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0172751', '# - total flops = 5.5745e+06, min = 1.86431e+06, max = 3.71019e+06', '# - flop rate = 0.32269 GFlop/s', '# - bytes moved = 44.3976 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.57004 GByte/s', '# - solve arithmetic intensity = 0.125559 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.740s; elapsed, 0.745s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.750s; elapsed, 0.751s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.740s; elapsed, 0.745s, #calls: 1', 'TOTAL : CPU, 0.740s; elapsed, 0.745s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.030s; elapsed, 1.030s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.076s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.241s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.750s; elapsed, 0.751s, #calls: 1', 'TOTAL : CPU, 2.100s; elapsed, 2.098s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 7575.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.210s; elapsed, 0.616s', 'individual call time for EIGEN_LDLT: CPU, 0.400s; elapsed, 0.221s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.052s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 15,151', '# - number of nonzeros = 132,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.093374', '# - matching time = 0.0180361', '# - symmetrization time = 0.00760913', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0305541', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.3274 MB', '# - factor time = 0.389111', '# - factor nonzeros = 2,290,919', '# - factor memory = 18.3274 MB', '# - total flops = 1.18089e+09, min = 1.91931e+08, max = 9.88957e+08', '# - flop rate = 3.03483 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.23655e+09', '# --------------------------------------------', '# total = 1.23655e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.90247e-11\trel.res = 1.17477e-15\tbw.error = 2.15866e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0166059', '# - total flops = 5.57537e+06, min = 1.86431e+06, max = 3.71106e+06', '# - flop rate = 0.335748 GFlop/s', '# - bytes moved = 44.459 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.67731 GByte/s', '# - solve arithmetic intensity = 0.125405 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.140s; elapsed, 0.570s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.160s; elapsed, 0.579s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.140s; elapsed, 0.570s, #calls: 1', 'TOTAL : CPU, 1.140s; elapsed, 0.570s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.210s; elapsed, 0.616s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.052s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.400s; elapsed, 0.221s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.160s; elapsed, 0.579s, #calls: 1', 'TOTAL : CPU, 2.870s; elapsed, 1.468s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 7575.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 2.060s; elapsed, 0.533s', 'individual call time for EIGEN_LDLT: CPU, 0.810s; elapsed, 0.253s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.045s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -24 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 15,151', '# - number of nonzeros = 132,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 68', '# - nd time = 0.112134', '# - matching time = 0.020875', '# - symmetrization time = 0.00843382', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0272381', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.3274 MB', '# - factor time = 0.360395', '# - factor nonzeros = 2,290,919', '# - factor memory = 18.3274 MB', '# - total flops = 1.18098e+09, min = 1.91931e+08, max = 9.89054e+08', '# - flop rate = 3.27692 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.23655e+09', '# --------------------------------------------', '# total = 1.23655e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.91725e-11\trel.res = 1.1839e-15\tbw.error = 2.17012e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0217271', '# - total flops = 5.57619e+06, min = 1.86431e+06, max = 3.71188e+06', '# - flop rate = 0.256647 GFlop/s', '# - bytes moved = 44.4884 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.0476 GByte/s', '# - solve arithmetic intensity = 0.12534 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.260s; elapsed, 0.570s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.310s; elapsed, 0.579s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.260s; elapsed, 0.570s, #calls: 1', 'TOTAL : CPU, 2.260s; elapsed, 0.570s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 2.060s; elapsed, 0.533s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.045s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.810s; elapsed, 0.253s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.310s; elapsed, 0.579s, #calls: 1', 'TOTAL : CPU, 5.330s; elapsed, 1.410s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 7575.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.330s; elapsed, 0.436s', 'individual call time for EIGEN_LDLT: CPU, 1.500s; elapsed, 0.257s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.040s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2CPP -2', ' MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 15,151', '# - number of nonzeros = 132,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.113523', '# - matching time = 0.0207288', '# - symmetrization time = 0.00852203', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0304511', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.3274 MB', '# - factor time = 0.307879', '# - factor nonzeros = 2,290,919', '# - factor memory = 18.3274 MB', '# - total flops = 1.18099e+09, min = 1.91933e+08, max = 9.89054e+08', '# - flop rate = 3.83588 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.23655e+09', '# --------------------------------------------', '# total = 1.23655e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.86499e-11\trel.res = 1.15163e-15\tbw.error = 2.17127e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0173819', '# - total flops = 5.57619e+06, min = 1.86431e+06, max = 3.71188e+06', '# - flop rate = 0.320804 GFlop/s', '# - bytes moved = 44.5335 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.56206 GByte/s', '# - solve arithmetic intensity = 0.125213 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.070s; elapsed, 0.514s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.160s; elapsed, 0.521s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.070s; elapsed, 0.514s, #calls: 1', 'TOTAL : CPU, 4.070s; elapsed, 0.514s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.330s; elapsed, 0.436s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.040s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.500s; elapsed, 0.257s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.160s; elapsed, 0.521s, #calls: 1', 'TOTAL : CPU, 9.240s; elapsed, 1.254s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 7575.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.470s; elapsed, 0.430s', 'individual call time for EIGEN_LDLT: CPU, 2.960s; elapsed, 0.244s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.039s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 15,151', '# - number of nonzeros = 132,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.114849', '# - matching time = 0.022846', '# - symmetrization time = 0.00854087', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0333521', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.3274 MB', '# - factor time = 0.373845', '# - factor nonzeros = 2,290,919', '# - factor memory = 18.3274 MB', '# - total flops = 1.18099e+09, min = 1.91934e+08, max = 9.89058e+08', '# - flop rate = 3.15904 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.23655e+09', '# --------------------------------------------', '# total = 1.23655e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.85795e-11\trel.res = 1.14728e-15\tbw.error = 2.18273e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.026763', '# - total flops = 5.57619e+06, min = 1.86431e+06, max = 3.71188e+06', '# - flop rate = 0.208355 GFlop/s', '# - bytes moved = 44.5713 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.66541 GByte/s', '# - solve arithmetic intensity = 0.125107 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.360s; elapsed, 0.601s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.760s; elapsed, 0.612s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.360s; elapsed, 0.601s, #calls: 1', 'TOTAL : CPU, 9.360s; elapsed, 0.601s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.470s; elapsed, 0.430s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.039s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.960s; elapsed, 0.244s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 9.760s; elapsed, 0.612s, #calls: 1', 'TOTAL : CPU, 19.640s; elapsed, 1.325s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 3787.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.060s; elapsed, 1.061s', 'individual call time for EIGEN_LDLT: CPU, 0.250s; elapsed, 0.246s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.076s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 15,151', '# - number of nonzeros = 132,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,083', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.110708', '# - matching time = 0.020179', '# - symmetrization time = 0.0106511', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,083', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.019567', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.9811 MB', '# - factor time = 0.404013', '# - factor nonzeros = 2,247,641', '# - factor memory = 17.9811 MB', '# - total flops = 1.13189e+09, min = 1.91919e+08, max = 3.50417e+08', '# - flop rate = 2.80163 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.18443e+09', '# --------------------------------------------', '# total = 1.18443e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.73115e-11\trel.res = 1.06898e-15\tbw.error = 5.45588e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0203171', '# - total flops = 7.13702e+06, min = 1.5544e+06, max = 2.07878e+06', '# - flop rate = 0.351282 GFlop/s', '# - bytes moved = 38.9492 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.91707 GByte/s', '# - solve arithmetic intensity = 0.183239 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.570s; elapsed, 0.590s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.570s; elapsed, 0.595s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.590s; elapsed, 0.594s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.600s; elapsed, 0.599s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.570s; elapsed, 0.595s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.595s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.590s; elapsed, 0.594s, #calls: 1', 'TOTAL : CPU, 0.590s; elapsed, 0.594s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.570s; elapsed, 0.590s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.590s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.060s; elapsed, 1.061s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.076s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.250s; elapsed, 0.246s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.600s; elapsed, 0.599s, #calls: 1', 'TOTAL : CPU, 1.990s; elapsed, 1.982s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 3787.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.400s; elapsed, 0.715s', 'individual call time for EIGEN_LDLT: CPU, 0.440s; elapsed, 0.259s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.063s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 15,151', '# - number of nonzeros = 132,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,083', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.112983', '# - matching time = 0.0206501', '# - symmetrization time = 0.009969', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,083', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0223191', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.9811 MB', '# - factor time = 0.31894', '# - factor nonzeros = 2,247,641', '# - factor memory = 17.9811 MB', '# - total flops = 1.1319e+09, min = 1.91919e+08, max = 3.50421e+08', '# - flop rate = 3.54893 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.18443e+09', '# --------------------------------------------', '# total = 1.18443e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.72399e-11\trel.res = 1.06456e-15\tbw.error = 5.45588e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0186892', '# - total flops = 7.13702e+06, min = 1.5544e+06, max = 2.07878e+06', '# - flop rate = 0.38188 GFlop/s', '# - bytes moved = 38.9548 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.08435 GByte/s', '# - solve arithmetic intensity = 0.183213 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.020s; elapsed, 0.517s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.010s; elapsed, 0.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.010s; elapsed, 0.517s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.040s; elapsed, 0.521s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.020s; elapsed, 0.517s, #calls: 1', 'TOTAL : CPU, 1.020s; elapsed, 0.517s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.010s; elapsed, 0.517s, #calls: 1', 'TOTAL : CPU, 1.010s; elapsed, 0.517s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.010s; elapsed, 0.513s, #calls: 1', 'TOTAL : CPU, 1.010s; elapsed, 0.513s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.400s; elapsed, 0.715s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.063s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.440s; elapsed, 0.259s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.040s; elapsed, 0.521s, #calls: 1', 'TOTAL : CPU, 3.000s; elapsed, 1.558s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 3787.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.780s; elapsed, 0.461s', 'individual call time for EIGEN_LDLT: CPU, 0.810s; elapsed, 0.242s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.046s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 15,151', '# - number of nonzeros = 132,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,083', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.112945', '# - matching time = 0.0210841', '# - symmetrization time = 0.010988', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,083', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0249031', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.9811 MB', '# - factor time = 0.379833', '# - factor nonzeros = 2,247,641', '# - factor memory = 17.9811 MB', '# - total flops = 1.1319e+09, min = 1.91919e+08, max = 3.50421e+08', '# - flop rate = 2.97999 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.18443e+09', '# --------------------------------------------', '# total = 1.18443e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.72103e-11\trel.res = 1.06273e-15\tbw.error = 5.45588e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.02789', '# - total flops = 7.13702e+06, min = 1.5544e+06, max = 2.07878e+06', '# - flop rate = 0.255899 GFlop/s', '# - bytes moved = 38.962 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.39699 GByte/s', '# - solve arithmetic intensity = 0.183179 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.320s; elapsed, 0.590s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.320s; elapsed, 0.587s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.340s; elapsed, 0.590s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.380s; elapsed, 0.595s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.320s; elapsed, 0.590s, #calls: 1', 'TOTAL : CPU, 2.320s; elapsed, 0.590s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.340s; elapsed, 0.590s, #calls: 1', 'TOTAL : CPU, 2.340s; elapsed, 0.590s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.320s; elapsed, 0.587s, #calls: 1', 'TOTAL : CPU, 2.320s; elapsed, 0.587s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.780s; elapsed, 0.461s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.046s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.810s; elapsed, 0.242s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.380s; elapsed, 0.595s, #calls: 1', 'TOTAL : CPU, 5.120s; elapsed, 1.343s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 3787.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.390s; elapsed, 0.448s', 'individual call time for EIGEN_LDLT: CPU, 1.520s; elapsed, 0.261s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.044s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 15,151', '# - number of nonzeros = 132,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,083', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.121256', '# - matching time = 0.0220299', '# - symmetrization time = 0.0113342', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,083', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0256839', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.9811 MB', '# - factor time = 0.385643', '# - factor nonzeros = 2,247,641', '# - factor memory = 17.9811 MB', '# - total flops = 1.1319e+09, min = 1.91921e+08, max = 3.50423e+08', '# - flop rate = 2.93511 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.18443e+09', '# --------------------------------------------', '# total = 1.18443e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.62174e-11\trel.res = 1.00142e-15\tbw.error = 5.45377e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0292959', '# - total flops = 7.13702e+06, min = 1.5544e+06, max = 2.07878e+06', '# - flop rate = 0.243618 GFlop/s', '# - bytes moved = 38.9648 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.33004 GByte/s', '# - solve arithmetic intensity = 0.183166 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.790s; elapsed, 0.607s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.820s; elapsed, 0.612s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.820s; elapsed, 0.612s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.920s; elapsed, 0.616s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.790s; elapsed, 0.607s, #calls: 1', 'TOTAL : CPU, 4.790s; elapsed, 0.607s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.820s; elapsed, 0.612s, #calls: 1', 'TOTAL : CPU, 4.820s; elapsed, 0.612s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.820s; elapsed, 0.612s, #calls: 1', 'TOTAL : CPU, 4.820s; elapsed, 0.612s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.390s; elapsed, 0.448s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.270s; elapsed, 0.044s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.520s; elapsed, 0.261s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.920s; elapsed, 0.616s, #calls: 1', 'TOTAL : CPU, 10.100s; elapsed, 1.370s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 1893.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.050s; elapsed, 1.050s', 'individual call time for EIGEN_LDLT: CPU, 0.260s; elapsed, 0.261s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.075s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 15,151', '# - number of nonzeros = 132,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,083', '# - number of levels = 66', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.0925648', '# - matching time = 0.0172641', '# - symmetrization time = 0.0064199', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,083', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.017647', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.9811 MB', '# - factor time = 0.388098', '# - factor nonzeros = 2,247,641', '# - factor memory = 17.9811 MB', '# - total flops = 1.13189e+09, min = 635738, max = 3.35592e+08', '# - flop rate = 2.91652 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.17894e+09', '# --------------------------------------------', '# total = 1.17894e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.45241e-11\trel.res = 8.96863e-16\tbw.error = 9.53513e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.013973', '# - total flops = 9.61597e+06, min = 27075, max = 1.84878e+06', '# - flop rate = 0.688183 GFlop/s', '# - bytes moved = 37.6714 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.69601 GByte/s', '# - solve arithmetic intensity = 0.255259 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.540s; elapsed, 0.542s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.520s; elapsed, 0.542s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.510s; elapsed, 0.540s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.520s; elapsed, 0.543s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.530s; elapsed, 0.544s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.540s; elapsed, 0.542s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.540s; elapsed, 0.543s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.540s; elapsed, 0.547s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.520s; elapsed, 0.542s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.542s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.540s; elapsed, 0.542s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.542s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.510s; elapsed, 0.540s, #calls: 1', 'TOTAL : CPU, 0.510s; elapsed, 0.540s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.520s; elapsed, 0.543s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.543s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.540s; elapsed, 0.542s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.542s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.530s; elapsed, 0.544s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.544s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.540s; elapsed, 0.543s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.543s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.050s; elapsed, 1.050s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.075s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.260s; elapsed, 0.261s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.540s; elapsed, 0.547s, #calls: 1', 'TOTAL : CPU, 1.930s; elapsed, 1.933s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 1893.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.290s; elapsed, 0.655s', 'individual call time for EIGEN_LDLT: CPU, 0.440s; elapsed, 0.252s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.058s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 15,151', '# - number of nonzeros = 132,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,083', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.113463', '# - matching time = 0.0208652', '# - symmetrization time = 0.00681305', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,083', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0229719', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.9811 MB', '# - factor time = 0.4217', '# - factor nonzeros = 2,247,641', '# - factor memory = 17.9811 MB', '# - total flops = 1.1319e+09, min = 635738, max = 3.35592e+08', '# - flop rate = 2.68413 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.17894e+09', '# --------------------------------------------', '# total = 1.17894e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.4629e-11\trel.res = 9.03339e-16\tbw.error = 1.06162e-14', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0175869', '# - total flops = 9.61597e+06, min = 27075, max = 1.84878e+06', '# - flop rate = 0.546768 GFlop/s', '# - bytes moved = 37.6753 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.14223 GByte/s', '# - solve arithmetic intensity = 0.255233 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.060s; elapsed, 0.614s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.200s; elapsed, 0.612s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.210s; elapsed, 0.614s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.190s; elapsed, 0.613s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.050s; elapsed, 0.614s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.190s; elapsed, 0.614s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.220s; elapsed, 0.612s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.220s; elapsed, 0.619s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.060s; elapsed, 0.614s, #calls: 1', 'TOTAL : CPU, 1.060s; elapsed, 0.614s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.050s; elapsed, 0.614s, #calls: 1', 'TOTAL : CPU, 1.050s; elapsed, 0.614s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.190s; elapsed, 0.613s, #calls: 1', 'TOTAL : CPU, 1.190s; elapsed, 0.613s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.200s; elapsed, 0.612s, #calls: 1', 'TOTAL : CPU, 1.200s; elapsed, 0.612s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.210s; elapsed, 0.614s, #calls: 1', 'TOTAL : CPU, 1.210s; elapsed, 0.614s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.190s; elapsed, 0.614s, #calls: 1', 'TOTAL : CPU, 1.190s; elapsed, 0.614s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.220s; elapsed, 0.612s, #calls: 1', 'TOTAL : CPU, 1.220s; elapsed, 0.612s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.290s; elapsed, 0.655s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.058s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.440s; elapsed, 0.252s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.220s; elapsed, 0.619s, #calls: 1', 'TOTAL : CPU, 3.050s; elapsed, 1.584s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 1893.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.650s; elapsed, 0.427s', 'individual call time for EIGEN_LDLT: CPU, 0.780s; elapsed, 0.237s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.046s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 15,151', '# - number of nonzeros = 132,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,083', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.116333', '# - matching time = 0.021883', '# - symmetrization time = 0.00738192', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,083', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.021419', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.9811 MB', '# - factor time = 0.372121', '# - factor nonzeros = 2,247,641', '# - factor memory = 17.9811 MB', '# - total flops = 1.1319e+09, min = 637608, max = 3.35592e+08', '# - flop rate = 3.04176 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.17894e+09', '# --------------------------------------------', '# total = 1.17894e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.45342e-11\trel.res = 8.97483e-16\tbw.error = 1.06162e-14', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.018209', '# - total flops = 9.61597e+06, min = 27075, max = 1.84878e+06', '# - flop rate = 0.52809 GFlop/s', '# - bytes moved = 37.678 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.0692 GByte/s', '# - solve arithmetic intensity = 0.255215 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.260s; elapsed, 0.569s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.260s; elapsed, 0.568s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.250s; elapsed, 0.568s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.930s; elapsed, 0.569s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.240s; elapsed, 0.569s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.970s; elapsed, 0.570s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.210s; elapsed, 0.569s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.270s; elapsed, 0.574s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.260s; elapsed, 0.569s, #calls: 1', 'TOTAL : CPU, 2.260s; elapsed, 0.569s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.250s; elapsed, 0.568s, #calls: 1', 'TOTAL : CPU, 2.250s; elapsed, 0.568s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.930s; elapsed, 0.569s, #calls: 1', 'TOTAL : CPU, 1.930s; elapsed, 0.569s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.260s; elapsed, 0.568s, #calls: 1', 'TOTAL : CPU, 2.260s; elapsed, 0.568s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.970s; elapsed, 0.570s, #calls: 1', 'TOTAL : CPU, 1.970s; elapsed, 0.570s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.240s; elapsed, 0.569s, #calls: 1', 'TOTAL : CPU, 2.240s; elapsed, 0.569s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.210s; elapsed, 0.569s, #calls: 1', 'TOTAL : CPU, 2.210s; elapsed, 0.569s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.650s; elapsed, 0.427s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.046s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.780s; elapsed, 0.237s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.270s; elapsed, 0.574s, #calls: 1', 'TOTAL : CPU, 4.850s; elapsed, 1.283s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 946.9375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.090s; elapsed, 1.085s', 'individual call time for EIGEN_LDLT: CPU, 0.250s; elapsed, 0.256s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.076s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 15,151', '# - number of nonzeros = 132,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,083', '# - number of levels = 65', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.111805', '# - matching time = 0.020946', '# - symmetrization time = 0.00759792', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,083', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.021102', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.9811 MB', '# - factor time = 0.455091', '# - factor nonzeros = 2,247,641', '# - factor memory = 17.9811 MB', '# - total flops = 1.13189e+09, min = 555340, max = 3.3205e+08', '# - flop rate = 2.48718 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.16787e+09', '# --------------------------------------------', '# total = 1.16787e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.27478e-11\trel.res = 7.87172e-16\tbw.error = 3.35503e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0177581', '# - total flops = 1.52775e+07, min = 21844, max = 1.77972e+06', '# - flop rate = 0.860311 GFlop/s', '# - bytes moved = 36.297 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.04397 GByte/s', '# - solve arithmetic intensity = 0.420903 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.590s; elapsed, 0.638s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.600s; elapsed, 0.641s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.600s; elapsed, 0.641s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.620s; elapsed, 0.642s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.630s; elapsed, 0.641s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.620s; elapsed, 0.642s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.560s; elapsed, 0.643s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.580s; elapsed, 0.641s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.580s; elapsed, 0.640s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.600s; elapsed, 0.642s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.610s; elapsed, 0.643s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.630s; elapsed, 0.643s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.610s; elapsed, 0.642s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.630s; elapsed, 0.642s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.610s; elapsed, 0.641s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.640s; elapsed, 0.647s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.590s; elapsed, 0.638s, #calls: 1', 'TOTAL : CPU, 0.590s; elapsed, 0.638s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.620s; elapsed, 0.642s, #calls: 1', 'TOTAL : CPU, 0.620s; elapsed, 0.642s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.560s; elapsed, 0.643s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.643s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.600s; elapsed, 0.641s, #calls: 1', 'TOTAL : CPU, 0.600s; elapsed, 0.641s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.630s; elapsed, 0.641s, #calls: 1', 'TOTAL : CPU, 0.630s; elapsed, 0.641s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.620s; elapsed, 0.642s, #calls: 1', 'TOTAL : CPU, 0.620s; elapsed, 0.642s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.600s; elapsed, 0.642s, #calls: 1', 'TOTAL : CPU, 0.600s; elapsed, 0.642s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.580s; elapsed, 0.640s, #calls: 1', 'TOTAL : CPU, 0.580s; elapsed, 0.640s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.630s; elapsed, 0.642s, #calls: 1', 'TOTAL : CPU, 0.630s; elapsed, 0.642s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.600s; elapsed, 0.641s, #calls: 1', 'TOTAL : CPU, 0.600s; elapsed, 0.641s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.610s; elapsed, 0.642s, #calls: 1', 'TOTAL : CPU, 0.610s; elapsed, 0.642s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.630s; elapsed, 0.643s, #calls: 1', 'TOTAL : CPU, 0.630s; elapsed, 0.643s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.610s; elapsed, 0.641s, #calls: 1', 'TOTAL : CPU, 0.610s; elapsed, 0.641s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.580s; elapsed, 0.641s, #calls: 1', 'TOTAL : CPU, 0.580s; elapsed, 0.641s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.610s; elapsed, 0.643s, #calls: 1', 'TOTAL : CPU, 0.610s; elapsed, 0.643s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.090s; elapsed, 1.085s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.076s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.250s; elapsed, 0.256s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.640s; elapsed, 0.647s, #calls: 1', 'TOTAL : CPU, 2.060s; elapsed, 2.064s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 946.9375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.270s; elapsed, 0.644s', 'individual call time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.259s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.060s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 15,151', '# - number of nonzeros = 132,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,083', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.126856', '# - matching time = 0.0227029', '# - symmetrization time = 0.00802994', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,083', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.025517', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.9811 MB', '# - factor time = 0.351168', '# - factor nonzeros = 2,247,641', '# - factor memory = 17.9811 MB', '# - total flops = 1.1319e+09, min = 555340, max = 3.3205e+08', '# - flop rate = 3.22323 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.16787e+09', '# --------------------------------------------', '# total = 1.16787e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.29676e-11\trel.res = 8.0075e-16\tbw.error = 3.35503e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.017313', '# - total flops = 1.52775e+07, min = 21844, max = 1.77972e+06', '# - flop rate = 0.882431 GFlop/s', '# - bytes moved = 36.3006 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.09672 GByte/s', '# - solve arithmetic intensity = 0.420861 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 1.100s; elapsed, 0.562s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.090s; elapsed, 0.563s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 1.110s; elapsed, 0.563s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 1.080s; elapsed, 0.560s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.100s; elapsed, 0.564s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.090s; elapsed, 0.563s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.110s; elapsed, 0.564s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.050s; elapsed, 0.562s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.030s; elapsed, 0.563s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.000s; elapsed, 0.562s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 1.010s; elapsed, 0.562s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 1.120s; elapsed, 0.565s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 1.070s; elapsed, 0.562s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 1.110s; elapsed, 0.563s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 1.100s; elapsed, 0.561s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.120s; elapsed, 0.568s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.100s; elapsed, 0.564s, #calls: 1', 'TOTAL : CPU, 1.100s; elapsed, 0.564s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.110s; elapsed, 0.564s, #calls: 1', 'TOTAL : CPU, 1.110s; elapsed, 0.564s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 1.110s; elapsed, 0.563s, #calls: 1', 'TOTAL : CPU, 1.110s; elapsed, 0.563s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 1.110s; elapsed, 0.563s, #calls: 1', 'TOTAL : CPU, 1.110s; elapsed, 0.563s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.090s; elapsed, 0.563s, #calls: 1', 'TOTAL : CPU, 1.090s; elapsed, 0.563s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 1.080s; elapsed, 0.560s, #calls: 1', 'TOTAL : CPU, 1.080s; elapsed, 0.560s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 1.010s; elapsed, 0.562s, #calls: 1', 'TOTAL : CPU, 1.010s; elapsed, 0.562s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 1.100s; elapsed, 0.562s, #calls: 1', 'TOTAL : CPU, 1.100s; elapsed, 0.562s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.000s; elapsed, 0.562s, #calls: 1', 'TOTAL : CPU, 1.000s; elapsed, 0.562sExiting profiler', '', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.030s; elapsed, 0.563s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.563s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 1.070s; elapsed, 0.562s, #calls: 1', 'TOTAL : CPU, 1.070s; elapsed, 0.562s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 1.120s; elapsed, 0.565s, #calls: 1', 'TOTAL : CPU, 1.120s; elapsed, 0.565s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.090s; elapsed, 0.563s, #calls: 1', 'TOTAL : CPU, 1.090s; elapsed, 0.563s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 1.100s; elapsed, 0.561s, #calls: 1', 'TOTAL : CPU, 1.100s; elapsed, 0.561s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.050s; elapsed, 0.562s, #calls: 1', 'TOTAL : CPU, 1.050s; elapsed, 0.562s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.270s; elapsed, 0.644s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.060s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.259s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.120s; elapsed, 0.568s, #calls: 1', 'TOTAL : CPU, 2.930s; elapsed, 1.532s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 14651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.930s; elapsed, 0.933s', 'individual call time for EIGEN_LDLT: CPU, 0.220s; elapsed, 0.219s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.065s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.930s; elapsed, 0.933s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.065s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.220s; elapsed, 0.219s, #calls: 1', 'TOTAL : CPU, 1.210s; elapsed, 1.217s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 14651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.200s; elapsed, 0.615s', 'individual call time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.217s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.051s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.200s; elapsed, 0.615s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.051s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.217s, #calls: 1', 'TOTAL : CPU, 1.700s; elapsed, 0.883s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 14651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 2.400s; elapsed, 0.618s', 'individual call time for EIGEN_LDLT: CPU, 0.790s; elapsed, 0.242s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.048s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 2.400s; elapsed, 0.618s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.048s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.790s; elapsed, 0.242s, #calls: 1', 'TOTAL : CPU, 3.350s; elapsed, 0.909s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 14651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.290s; elapsed, 0.432s', 'individual call time for EIGEN_LDLT: CPU, 1.510s; elapsed, 0.244s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.040s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.290s; elapsed, 0.432s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.040s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.510s; elapsed, 0.244s, #calls: 1', 'TOTAL : CPU, 5.050s; elapsed, 0.716s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 14651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.870s; elapsed, 0.457s', 'individual call time for EIGEN_LDLT: CPU, 3.030s; elapsed, 0.244s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.041s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.870s; elapsed, 0.457s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.500s; elapsed, 0.041s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.030s; elapsed, 0.244s, #calls: 1', 'TOTAL : CPU, 10.400s; elapsed, 0.742s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 7325.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.980s; elapsed, 0.977s', 'individual call time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.236s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.069s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.# - nd time = ', '# ******************************************************************', '0.096272', '# - matching time = 0.01771', '# - symmetrization time = 0.00689602', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.022279', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.613525', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.08505e+09, min = 2.09373e+08, max = 8.75678e+08', '# - flop rate = 1.76855 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.14259e+09', '# --------------------------------------------', '# total = 1.14259e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.62462e-11\trel.res = 1.0032e-15\tbw.error = 1.73909e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.018492', '# - total flops = 5.3468e+06, min = 1.93797e+06, max = 3.40882e+06', '# - flop rate = 0.289141 GFlop/s', '# - bytes moved = 43.3862 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.34622 GByte/s', '# - solve arithmetic intensity = 0.123237 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.780s; elapsed, 0.784s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.790s; elapsed, 0.792s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.780s; elapsed, 0.784s, #calls: 1', 'TOTAL : CPU, 0.780s; elapsed, 0.784s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.980s; elapsed, 0.977s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.069s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.236s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.790s; elapsed, 0.792s, #calls: 1', 'TOTAL : CPU, 2.070s; elapsed, 2.074s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 7325.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.200s; elapsed, 0.610s', 'individual call time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.222s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.051s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0909331', '# - matching time = 0.0176201', '# - symmetrization time = 0.00702095', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0257821', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.336538', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.0851e+09, min = 2.09373e+08, max = 8.75729e+08', '# - flop rate = 3.22431 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.14259e+09', '# --------------------------------------------', '# total = 1.14259e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.55563e-11\trel.res = 9.60601e-16\tbw.error = 1.7363e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.015094', '# - total flops = 5.34832e+06, min = 1.93797e+06, max = 3.41034e+06', '# - flop rate = 0.354333 GFlop/s', '# - bytes moved = 43.4535 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.87885 GByte/s', '# - solve arithmetic intensity = 0.123081 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.000s; elapsed, 0.506s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.020s; elapsed, 0.515s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.000s; elapsed, 0.506s, #calls: 1', 'TOTAL : CPU, 1.000s; elapsed, 0.506s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.200s; elapsed, 0.610s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.051s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.222s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.020s; elapsed, 0.515s, #calls: 1', 'TOTAL : CPU, 2.720s; elapsed, 1.398s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 7325.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 2.450s; elapsed, 0.628s', 'individual call time for EIGEN_LDLT: CPU, 0.820s; elapsed, 0.255s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.051s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using CPP -2', '2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.107496', '# - matching time = 0.019933', '# - symmetrization time = 0.00830007', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.029654', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.321788', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.08511e+09, min = 2.09373e+08, max = 8.75736e+08', '# - flop rate = 3.37212 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.14259e+09', '# --------------------------------------------', '# total = 1.14259e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.55568e-11\trel.res = 9.60632e-16\tbw.error = 1.72833e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0178552', '# - total flops = 5.34832e+06, min = 1.93797e+06, max = 3.41034e+06', '# - flop rate = 0.299539 GFlop/s', '# - bytes moved = 43.4677 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.43446 GByte/s', '# - solve arithmetic intensity = 0.123041 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.060s; elapsed, 0.521s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.120s; elapsed, 0.530s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.060s; elapsed, 0.521s, #calls: 1', 'TOTAL : CPU, 2.060s; elapsed, 0.521s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 2.450s; elapsed, 0.628s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.051s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.820s; elapsed, 0.255s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.120s; elapsed, 0.530s, #calls: 1', 'TOTAL : CPU, 5.570s; elapsed, 1.464s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 7325.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.280s; elapsed, 0.431s', 'individual call time for EIGEN_LDLT: CPU, 1.510s; elapsed, 0.247s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.042s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.108554', '# - matching time = 0.0214782', '# - symmetrization time = 0.00855708', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0295579', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.269104', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.08515e+09, min = 2.09375e+08, max = 8.75776e+08', '# - flop rate = 4.03246 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.14259e+09', '# --------------------------------------------', '# total = 1.14259e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.56172e-11\trel.res = 9.6436e-16\tbw.error = 1.79793e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0191679', '# - total flops = 5.34832e+06, min = 1.93797e+06, max = 3.41034e+06', '# - flop rate = 0.279025 GFlop/s', '# - bytes moved = 43.4973 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.26928 GByte/s', '# - solve arithmetic intensity = 0.122957 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.710s; elapsed, 0.470s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.840s; elapsed, 0.480s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.710s; elapsed, 0.470s, #calls: 1', 'TOTAL : CPU, 3.710s; elapsed, 0.470s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.280s; elapsed, 0.431s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.260s; elapsed, 0.042s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.510s; elapsed, 0.247s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.840s; elapsed, 0.480s, #calls: 1', 'TOTAL : CPU, 8.890s; elapsed, 1.201s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 7325.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.110s; elapsed, 0.472s', 'individual call time for EIGEN_LDLT: CPU, 3.040s; elapsed, 0.258s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.490s; elapsed, 0.042s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.109485', '# - matching time = 0.0237281', '# - symmetrization time = 0.00819612', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.030988', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.320158', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.08516e+09, min = 2.09376e+08, max = 8.75779e+08', '# - flop rate = 3.38944 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.14259e+09', '# --------------------------------------------', '# total = 1.14259e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.55885e-11\trel.res = 9.62587e-16\tbw.error = 1.78159e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0278749', '# - total flops = 5.34832e+06, min = 1.93797e+06, max = 3.41034e+06', '# - flop rate = 0.191868 GFlop/s', '# - bytes moved = 43.5424 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.56206 GByte/s', '# - solve arithmetic intensity = 0.12283 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.240s; elapsed, 0.536s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.690s; elapsed, 0.544s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 8.240s; elapsed, 0.536s, #calls: 1', 'TOTAL : CPU, 8.240s; elapsed, 0.536s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.110s; elapsed, 0.472s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.490s; elapsed, 0.042s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.040s; elapsed, 0.258s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 8.690s; elapsed, 0.544s, #calls: 1', 'TOTAL : CPU, 19.330s; elapsed, 1.315s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 3662.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.070s; elapsed, 1.065s', 'individual call time for EIGEN_LDLT: CPU, 0.250s; elapsed, 0.249s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.077s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.108204', '# - matching time = 0.0199461', '# - symmetrization time = 0.0103719', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0202298', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.368365', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.08505e+09, min = 1.79002e+08, max = 4.05598e+08', '# - flop rate = 2.94559 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.1396e+09', '# --------------------------------------------', '# total = 1.1396e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.61383e-11\trel.res = 9.96538e-16\tbw.error = 1.81756e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0200212', '# - total flops = 6.70752e+06, min = 1.30999e+06, max = 1.98026e+06', '# - flop rate = 0.335021 GFlop/s', '# - bytes moved = 39.3021 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.96303 GByte/s', '# - solve arithmetic intensity = 0.170665 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.550s; elapsed, 0.556s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.520s; elapsed, 0.551s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.550s; elapsed, 0.556s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.560s; elapsed, 0.560s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.550s; elapsed, 0.556s, #calls: 1', 'TOTAL : CPU, 0.550s; elapsed, 0.556s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.520s; elapsed, 0.551s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.551s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.550s; elapsed, 0.556s, #calls: 1', 'TOTAL : CPU, 0.550s; elapsed, 0.556s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.070s; elapsed, 1.065s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.077s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.250s; elapsed, 0.249s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.560s; elapsed, 0.560s, #calls: 1', 'TOTAL : CPU, 1.960s; elapsed, 1.951s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 3662.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.440s; elapsed, 0.733s', 'individual call time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.241s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.054s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.110193', '# - matching time = 0.0204458', '# - symmetrization time = 0.010983', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0235419', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.402146', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.08506e+09, min = 1.79002e+08, max = 4.0561e+08', '# - flop rate = 2.69818 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.1396e+09', '# --------------------------------------------', '# total = 1.1396e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.60231e-11\trel.res = 9.89423e-16\tbw.error = 1.72892e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0177121', '# - total flops = 6.70752e+06, min = 1.30999e+06, max = 1.98026e+06', '# - flop rate = 0.378697 GFlop/s', '# - bytes moved = 39.3115 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.21947 GByte/s', '# - solve arithmetic intensity = 0.170625 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.170s; elapsed, 0.594s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.180s; elapsed, 0.598s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.180s; elapsed, 0.598s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.210s; elapsed, 0.603s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.170s; elapsed, 0.594s, #calls: 1', 'TOTAL : CPU, 1.170s; elapsed, 0.594s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.180s; elapsed, 0.598s, #calls: 1', 'TOTAL : CPU, 1.180s; elapsed, 0.598s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.180s; elapsed, 0.598s, #calls: 1', 'TOTAL : CPU, 1.180s; elapsed, 0.598s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.440s; elapsed, 0.733s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.054s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.241s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.210s; elapsed, 0.603s, #calls: 1', 'TOTAL : CPU, 3.180s; elapsed, 1.631s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 3662.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.640s; elapsed, 0.428s', 'individual call time for EIGEN_LDLT: CPU, 0.780s; elapsed, 0.229s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.050s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.110631', '# - matching time = 0.0209811', '# - symmetrization time = 0.0109611', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0239658', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.311702', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.08506e+09, min = 1.79002e+08, max = 4.05612e+08', '# - flop rate = 3.4811 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.1396e+09', '# --------------------------------------------', '# total = 1.1396e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.60077e-11\trel.res = 9.88473e-16\tbw.error = 1.73483e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.023442', '# - total flops = 6.70752e+06, min = 1.30999e+06, max = 1.98026e+06', '# - flop rate = 0.286132 GFlop/s', '# - bytes moved = 39.3235 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.67748 GByte/s', '# - solve arithmetic intensity = 0.170573 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.020s; elapsed, 0.510s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.020s; elapsed, 0.515s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.030s; elapsed, 0.514s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.080s; elapsed, 0.520s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.020s; elapsed, 0.515s, #calls: 1', 'TOTAL : CPU, 2.020s; elapsed, 0.515s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.030s; elapsed, 0.514s, #calls: 1', 'TOTAL : CPU, 2.030s; elapsed, 0.514s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.020s; elapsed, 0.510s, #calls: 1', 'TOTAL : CPU, 2.020s; elapsed, 0.510s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.640s; elapsed, 0.428s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.050s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.780s; elapsed, 0.229s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.080s; elapsed, 0.520s, #calls: 1', 'TOTAL : CPU, 4.680s; elapsed, 1.227s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 3662.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.440s; elapsed, 0.452s', 'individual call time for EIGEN_LDLT: CPU, 1.520s; elapsed, 0.256s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.042s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# - nd time = # (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.110176', '# - matching time = 0.021827', '# - symmetrization time = 0.01074', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.026546', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.334374', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.08507e+09, min = 1.79004e+08, max = 4.05614e+08', '# - flop rate = 3.24508 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.1396e+09', '# --------------------------------------------', '# total = 1.1396e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.58963e-11\trel.res = 9.81591e-16\tbw.error = 1.7112e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0298018', '# - total flops = 6.70752e+06, min = 1.30999e+06, max = 1.98026e+06', '# - flop rate = 0.225071 GFlop/s', '# - bytes moved = 39.3473 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.3203 GByte/s', '# - solve arithmetic intensity = 0.17047 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.300s; elapsed, 0.545s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.320s; elapsed, 0.550s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.320s; elapsed, 0.550s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.420s; elapsed, 0.554s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.300s; elapsed, 0.545s, #calls: 1', 'TOTAL : CPU, 4.300s; elapsed, 0.545s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.320s; elapsed, 0.550s, #calls: 1', 'TOTAL : CPU, 4.320s; elapsed, 0.550s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.320s; elapsed, 0.550s, #calls: 1', 'TOTAL : CPU, 4.320s; elapsed, 0.550s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.440s; elapsed, 0.452s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.042s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.520s; elapsed, 0.256s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.420s; elapsed, 0.554s, #calls: 1', 'TOTAL : CPU, 9.630s; elapsed, 1.304s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 1831.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.990s; elapsed, 1.004s', 'individual call time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.236s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.069s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.099427', '# - matching time = 0.0205979', '# - symmetrization time = 0.010222', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0199101', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4218 MB', '# - factor time = 0.38962', '# - factor nonzeros = 2,177,723', '# - factor memory = 17.4218 MB', '# - total flops = 1.08505e+09, min = 746150, max = 2.89001e+08', '# - flop rate = 2.7849 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.13413e+09', '# --------------------------------------------', '# total = 1.13413e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.45204e-11\trel.res = 8.96633e-16\tbw.error = 1.36052e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0186419', '# - total flops = 8.80018e+06, min = 28189, max = 1.89032e+06', '# - flop rate = 0.472063 GFlop/s', '# - bytes moved = 38.099 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.04372 GByte/s', '# - solve arithmetic intensity = 0.230982 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.520s; elapsed, 0.568s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.550s; elapsed, 0.569s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.560s; elapsed, 0.563s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.560s; elapsed, 0.569s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.540s; elapsed, 0.568s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.560s; elapsed, 0.569s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.530s; elapsed, 0.567s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.570s; elapsed, 0.574s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.560s; elapsed, 0.569s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.569s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.560s; elapsed, 0.569s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.569s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.560s; elapsed, 0.563s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.563s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.550s; elapsed, 0.569s, #calls: 1', 'TOTAL : CPU, 0.550s; elapsed, 0.569s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.520s; elapsed, 0.568s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.568s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.540s; elapsed, 0.568s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.568s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.530s; elapsed, 0.567s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.567s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.990s; elapsed, 1.004s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.069s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.236s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.570s; elapsed, 0.574s, #calls: 1', 'TOTAL : CPU, 1.870s; elapsed, 1.883s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 1831.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.480s; elapsed, 0.748s', 'individual call time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.250s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.056s', 'CPP -2', '# Initializing STRUMPACK', '# using 2CPP -2', 'CPP -2', 'CPP -2', ' OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.111174', '# - matching time = 0.0207701', '# - symmetrization time = 0.00908494', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0217929', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4218 MB', '# - factor time = 0.314253', '# - factor nonzeros = 2,177,723', '# - factor memory = 17.4218 MB', '# - total flops = 1.08506e+09, min = 746150, max = 2.8901e+08', '# - flop rate = 3.45283 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.13413e+09', '# --------------------------------------------', '# total = 1.13413e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.44024e-11\trel.res = 8.89345e-16\tbw.error = 1.35888e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0186572', '# - total flops = 8.80018e+06, min = 28189, max = 1.89032e+06', '# - flop rate = 0.471677 GFlop/s', '# - bytes moved = 38.107 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.04248 GByte/s', '# - solve arithmetic intensity = 0.230933 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.000s; elapsed, 0.508s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.000s; elapsed, 0.505s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.990s; elapsed, 0.508s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.990s; elapsed, 0.509s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.960s; elapsed, 0.509s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.990s; elapsed, 0.508s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.940s; elapsed, 0.507s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.020s; elapsed, 0.513s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.990s; elapsed, 0.509s, #calls: 1', 'TOTAL : CPU, 0.990s; elapsed, 0.509s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.000s; elapsed, 0.508s, #calls: 1', 'TOTAL : CPU, 1.000s; elapsed, 0.508s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.000s; elapsed, 0.505s, #calls: 1', 'TOTAL : CPU, 1.000s; elapsed, 0.505s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.990s; elapsed, 0.508s, #calls: 1', 'TOTAL : CPU, 0.990s; elapsed, 0.508s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.960s; elapsed, 0.509s, #calls: 1', 'TOTAL : CPU, 0.960s; elapsed, 0.509s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.940s; elapsed, 0.507s, #calls: 1', 'TOTAL : CPU, 0.940s; elapsed, 0.507s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.990s; elapsed, 0.508s, #calls: 1', 'TOTAL : CPU, 0.990s; elapsed, 0.508s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.480s; elapsed, 0.748s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.056s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.250s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.020s; elapsed, 0.513s, #calls: 1', 'TOTAL : CPU, 3.030s; elapsed, 1.567s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 1831.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.790s; elapsed, 0.467s', 'individual call time for EIGEN_LDLT: CPU, 0.820s; elapsed, 0.263s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.053s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# - nd time = # (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.113226', '# - matching time = 0.0242381', '# - symmetrization time = 0.0100532', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0278649', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4218 MB', '# - factor time = 0.368608', '# - factor nonzeros = 2,177,723', '# - factor memory = 17.4218 MB', '# - total flops = 1.08507e+09, min = 747880, max = 2.89012e+08', '# - flop rate = 2.94369 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.13413e+09', '# --------------------------------------------', '# total = 1.13413e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.43736e-11\trel.res = 8.8757e-16\tbw.error = 1.35593e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.021266', '# - total flops = 8.80018e+06, min = 28189, max = 1.89032e+06', '# - flop rate = 0.413815 GFlop/s', '# - bytes moved = 38.1167 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.79238 GByte/s', '# - solve arithmetic intensity = 0.230875 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.280s; elapsed, 0.577s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.190s; elapsed, 0.580s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.280s; elapsed, 0.581s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.290s; elapsed, 0.581s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.250s; elapsed, 0.582s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.000s; elapsed, 0.580s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.970s; elapsed, 0.580s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.320s; elapsed, 0.585s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.250s; elapsed, 0.582s, #calls: 1', 'TOTAL : CPU, 2.250s; elapsed, 0.582s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.290s; elapsed, 0.581s, #calls: 1', 'TOTAL : CPU, 2.290s; elapsed, 0.581s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.280s; elapsed, 0.581s, #calls: 1', 'TOTAL : CPU, 2.280s; elapsed, 0.581s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.190s; elapsed, 0.580s, #calls: 1', 'TOTAL : CPU, 2.190s; elapsed, 0.580s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.280s; elapsed, 0.577s, #calls: 1', 'TOTAL : CPU, 2.280s; elapsed, 0.577s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.000s; elapsed, 0.580s, #calls: 1', 'TOTAL : CPU, 2.000s; elapsed, 0.580s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.970s; elapsed, 0.580s, #calls: 1', 'TOTAL : CPU, 1.970s; elapsed, 0.580s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.790s; elapsed, 0.467s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.170s; elapsed, 0.053s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.820s; elapsed, 0.263s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.320s; elapsed, 0.585s, #calls: 1', 'TOTAL : CPU, 5.100s; elapsed, 1.367s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 915.6875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.090s; elapsed, 1.087s', 'individual call time for EIGEN_LDLT: CPU, 0.290s; elapsed, 0.293s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.079s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 65', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.0941191', '# - matching time = 0.018338', '# - symmetrization time = 0.0100801', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0203371', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.30213', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.08505e+09, min = 535689, max = 2.85362e+08', '# - flop rate = 3.59134 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.1232e+09', '# --------------------------------------------', '# total = 1.1232e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.30996e-11\trel.res = 8.08899e-16\tbw.error = 1.11099e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.023288', '# - total flops = 1.37536e+07, min = 20884, max = 1.84992e+06', '# - flop rate = 0.590585 GFlop/s', '# - bytes moved = 36.1237 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.55117 GByte/s', '# - solve arithmetic intensity = 0.380735 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.450s; elapsed, 0.477s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.420s; elapsed, 0.477s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.460s; elapsed, 0.478s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.440s; elapsed, 0.476s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.470s; elapsed, 0.477s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.420s; elapsed, 0.478s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.450s; elapsed, 0.477s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.420s; elapsed, 0.477s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.480s; elapsed, 0.477s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.410s; elapsed, 0.477s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.430s; elapsed, 0.478s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.440s; elapsed, 0.479s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.480s; elapsed, 0.477s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.430s; elapsed, 0.473s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.460s; elapsed, 0.477s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.480s; elapsed, 0.483s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.460s; elapsed, 0.478s, #calls: 1', 'TOTAL : CPU, 0.460s; elapsed, 0.478s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.470s; elapsed, 0.477s, #calls: 1', 'TOTAL : CPU, 0.470s; elapsed, 0.477s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.420s; elapsed, 0.477s, #calls: 1', 'TOTAL : CPU, 0.420s; elapsed, 0.477s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.450s; elapsed, 0.477s, #calls: 1', 'TOTAL : CPU, 0.450s; elapsed, 0.477s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.440s; elapsed, 0.476s, #calls: 1', 'TOTAL : CPU, 0.440s; elapsed, 0.476s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.480s; elapsed, 0.477s, #calls: 1', 'TOTAL : CPU, 0.480s; elapsed, 0.477s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.430s; elapsed, 0.478s, #calls: 1', 'TOTAL : CPU, 0.430s; elapsed, 0.478s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.440s; elapsed, 0.479s, #calls: 1', 'TOTAL : CPU, 0.440s; elapsed, 0.479s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.480s; elapsed, 0.477s, #calls: 1', 'TOTAL : CPU, 0.480s; elapsed, 0.477s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.420s; elapsed, 0.478s, #calls: 1', 'TOTAL : CPU, 0.420s; elapsed, 0.478s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.410s; elapsed, 0.477s, #calls: 1', 'TOTAL : CPU, 0.410s; elapsed, 0.477s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.460s; elapsed, 0.477s, #calls: 1', 'TOTAL : CPU, 0.460s; elapsed, 0.477s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.420s; elapsed, 0.477s, #calls: 1', 'TOTAL : CPU, 0.420s; elapsed, 0.477s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.450s; elapsed, 0.477s, #calls: 1', 'TOTAL : CPU, 0.450s; elapsed, 0.477s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.430s; elapsed, 0.473s, #calls: 1', 'TOTAL : CPU, 0.430s; elapsed, 0.473s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.090s; elapsed, 1.087s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.079s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.290s; elapsed, 0.293s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.480s; elapsed, 0.483s, #calls: 1', 'TOTAL : CPU, 1.930s; elapsed, 1.942s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 915.6875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.430s; elapsed, 0.728s', 'individual call time for EIGEN_LDLT: CPU, 0.450s; elapsed, 0.261s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.057s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', ' = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 131,959', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,077', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.115615', '# - matching time = 0.021735', '# - symmetrization time = 0.0101881', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,077', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0277429', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 17.4217 MB', '# - factor time = 0.33319', '# - factor nonzeros = 2,177,707', '# - factor memory = 17.4217 MB', '# - total flops = 1.08505e+09, min = 535689, max = 2.85362e+08', '# - flop rate = 3.25656 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.1232e+09', '# --------------------------------------------', '# total = 1.1232e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.30501e-11\trel.res = 8.0584e-16\tbw.error = 1.07391e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0191011', '# - total flops = 1.37536e+07, min = 20884, max = 1.84992e+06', '# - flop rate = 0.720039 GFlop/s', '# - bytes moved = 36.1289 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.89145 GByte/s', '# - solve arithmetic intensity = 0.38068 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.030s; elapsed, 0.540s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 1.030s; elapsed, 0.539s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.030s; elapsed, 0.540s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.040s; elapsed, 0.539s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.010s; elapsed, 0.538s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 1.050s; elapsed, 0.539s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.010s; elapsed, 0.540s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.980s; elapsed, 0.539s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 1.060s; elapsed, 0.539s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 1.040s; elapsed, 0.536s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.060s; elapsed, 0.540s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.990s; elapsed, 0.540s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 1.050s; elapsed, 0.541s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 1.050s; elapsed, 0.539s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 1.070s; elapsed, 0.539s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.080s; elapsed, 0.545s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.010s; elapsed, 0.540s, #calls: 1', 'TOTAL : CPU, 1.010s; elapsed, 0.540s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.030s; elapsed, 0.540s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.540s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 1.040s; elapsed, 0.536s, #calls: 1', 'TOTAL : CPU, 1.040s; elapsed, 0.536s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.060s; elapsed, 0.540s, #calls: 1', 'TOTAL : CPU, 1.060s; elapsed, 0.540s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 1.050s; elapsed, 0.539s, #calls: 1', 'TOTAL : CPU, 1.050s; elapsed, 0.539s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 1.070s; elapsed, 0.539s, #calls: 1', 'TOTAL : CPU, 1.070s; elapsed, 0.539s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.030s; elapsed, 0.540s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.540s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 1.060s; elapsed, 0.539s, #calls: 1', 'TOTAL : CPU, 1.060s; elapsed, 0.539s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.990s; elapsed, 0.540s, #calls: 1', 'TOTAL : CPU, 0.990s; elapsed, 0.540s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 1.050s; elapsed, 0.539s, #calls: 1', 'TOTAL : CPU, 1.050s; elapsed, 0.539s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.040s; elapsed, 0.539s, #calls: 1', 'TOTAL : CPU, 1.040s; elapsed, 0.539s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 1.030s; elapsed, 0.539s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.539s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.980s; elapsed, 0.539s, #calls: 1', 'TOTAL : CPU, 0.980s; elapsed, 0.539s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.010s; elapsed, 0.538s, #calls: 1', 'TOTAL : CPU, 1.010s; elapsed, 0.538s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 1.050s; elapsed, 0.541s, #calls: 1', 'TOTAL : CPU, 1.050s; elapsed, 0.541s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.430s; elapsed, 0.728s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.057s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.450s; elapsed, 0.261s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.080s; elapsed, 0.545s, #calls: 1', 'TOTAL : CPU, 3.060s; elapsed, 1.592s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 14651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.330s; elapsed, 0.327s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.047s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.025s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.330s; elapsed, 0.327s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.025s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.047s, #calls: 1', 'TOTAL : CPU, 0.400s; elapsed, 0.400s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 14651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.470s; elapsed, 0.246s', 'individual call time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.048s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.021s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.470s; elapsed, 0.246s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.048s, #calls: 1', 'TOTAL : CPU, 0.620s; elapsed, 0.315s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 14651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.950s; elapsed, 0.249s', 'individual call time for EIGEN_LDLT: CPU, 0.210s; elapsed, 0.053s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.021s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.950s; elapsed, 0.249s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.210s; elapsed, 0.053s, #calls: 1', 'TOTAL : CPU, 1.250s; elapsed, 0.322s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 14651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.900s; elapsed, 0.250s', 'individual call time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.051s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.020s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.900s; elapsed, 0.250s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.020s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.410s; elapsed, 0.051s, #calls: 1', 'TOTAL : CPU, 2.470s; elapsed, 0.322s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 14651.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.350s; elapsed, 0.228s', 'individual call time for EIGEN_LDLT: CPU, 0.880s; elapsed, 0.055s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.340s; elapsed, 0.021s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.350s; elapsed, 0.228s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.340s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.880s; elapsed, 0.055s, #calls: 1', 'TOTAL : CPU, 4.570s; elapsed, 0.305s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 7325.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.350s; elapsed, 0.350s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.049s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.027s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0847781', '# - matching time = 0.0112219', '# - symmetrization time = 0.00341511', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0147491', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.16826 MB', '# - factor time = 0.192201', '# - factor nonzeros = 896,033', '# - factor memory = 7.16826 MB', '# - total flops = 1.94418e+08, min = 3.11578e+07, max = 1.6326e+08', '# - flop rate = 1.01153 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.08278e+08', '# --------------------------------------------', '# total = 2.08278e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.17207e-11\trel.res = 7.64957e-16\tbw.error = 1.74155e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0113499', '# - total flops = 2.3455e+06, min = 815211, max = 1.53029e+06', '# - flop rate = 0.206654 GFlop/s', '# - bytes moved = 21.3766 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.88341 GByte/s', '# - solve arithmetic intensity = 0.109723 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.330s; elapsed, 0.324s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.320s; elapsed, 0.330s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.330s; elapsed, 0.324s, #calls: 1', 'TOTAL : CPU, 0.330s; elapsed, 0.324s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.350s; elapsed, 0.350s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.020s; elapsed, 0.027s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.049s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.320s; elapsed, 0.330s, #calls: 1', 'TOTAL : CPU, 0.740s; elapsed, 0.755s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 7325.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.540s; elapsed, 0.272s', 'individual call time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.053s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.023s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0973461', '# - matching time = 0.0125849', '# - symmetrization time = 0.00424695', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.01792', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.16826 MB', '# - factor time = 0.121015', '# - factor nonzeros = 896,033', '# - factor memory = 7.16826 MB', '# - total flops = 1.94441e+08, min = 3.11578e+07, max = 1.63283e+08', '# - flop rate = 1.60675 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.08278e+08', '# --------------------------------------------', '# total = 2.08278e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.15192e-11\trel.res = 7.5181e-16\tbw.error = 1.67704e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0101171', '# - total flops = 2.3455e+06, min = 815211, max = 1.53029e+06', '# - flop rate = 0.231837 GFlop/s', '# - bytes moved = 21.3893 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.11418 GByte/s', '# - solve arithmetic intensity = 0.109658 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.540s; elapsed, 0.273s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.550s; elapsed, 0.278s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.540s; elapsed, 0.273s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.273s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.540s; elapsed, 0.272s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.053s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.550s; elapsed, 0.278s, #calls: 1', 'TOTAL : CPU, 1.240s; elapsed, 0.625s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 7325.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.850s; elapsed, 0.225s', 'individual call time for EIGEN_LDLT: CPU, 0.210s; elapsed, 0.052s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.021s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0859809', '# - matching time = 0.0114429', '# - symmetrization time = 0.00368285', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.016731', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.16826 MB', '# - factor time = 0.084429', '# - factor nonzeros = 896,033', '# - factor memory = 7.16826 MB', '# - total flops = 1.94443e+08, min = 3.11578e+07, max = 1.63285e+08', '# - flop rate = 2.30304 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.08278e+08', '# --------------------------------------------', '# total = 2.08278e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.14358e-11\trel.res = 7.46362e-16\tbw.error = 1.67704e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0105541', '# - total flops = 2.3455e+06, min = 815211, max = 1.53029e+06', '# - flop rate = 0.222237 GFlop/s', '# - bytes moved = 21.4055 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.02817 GByte/s', '# - solve arithmetic intensity = 0.109575 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.880s; elapsed, 0.223s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.910s; elapsed, 0.229s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.880s; elapsed, 0.223s, #calls: 1', 'TOTAL : CPU, 0.880s; elapsed, 0.223s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.850s; elapsed, 0.225s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.210s; elapsed, 0.052s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.910s; elapsed, 0.229s, #calls: 1', 'TOTAL : CPU, 2.050s; elapsed, 0.527s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 7325.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.830s; elapsed, 0.243s', 'individual call time for EIGEN_LDLT: CPU, 0.420s; elapsed, 0.052s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.022s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.087292', '# - matching time = 0.013016', '# - symmetrization time = 0.00387692', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.018919', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.16826 MB', '# - factor time = 0.093694', '# - factor nonzeros = 896,033', '# - factor memory = 7.16826 MB', '# - total flops = 1.94446e+08, min = 3.11596e+07, max = 1.63286e+08', '# - flop rate = 2.07533 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.08278e+08', '# --------------------------------------------', '# total = 2.08278e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.1387e-11\trel.res = 7.43176e-16\tbw.error = 1.64769e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0109119', '# - total flops = 2.3455e+06, min = 815211, max = 1.53029e+06', '# - flop rate = 0.214948 GFlop/s', '# - bytes moved = 21.418 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.9628 GByte/s', '# - solve arithmetic intensity = 0.109511 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.880s; elapsed, 0.239s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.950s; elapsed, 0.244s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.880s; elapsed, 0.239s, #calls: 1', 'TOTAL : CPU, 1.880s; elapsed, 0.239s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.830s; elapsed, 0.243s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.022s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.420s; elapsed, 0.052s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.950s; elapsed, 0.244s, #calls: 1', 'TOTAL : CPU, 4.380s; elapsed, 0.561s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 7325.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.260s; elapsed, 0.222s', 'individual call time for EIGEN_LDLT: CPU, 0.790s; elapsed, 0.050s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.320s; elapsed, 0.020s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0978191', '# - matching time = 0.0156479', '# - symmetrization time = 0.00438499', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0234442', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.16826 MB', '# - factor time = 0.114914', '# - factor nonzeros = 896,033', '# - factor memory = 7.16826 MB', '# - total flops = 1.94449e+08, min = 3.11613e+07, max = 1.63288e+08', '# - flop rate = 1.69213 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.08278e+08', '# --------------------------------------------', '# total = 2.08278e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.1385e-11\trel.res = 7.43046e-16\tbw.error = 1.64769e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0112841', '# - total flops = 2.3455e+06, min = 815211, max = 1.53029e+06', '# - flop rate = 0.207859 GFlop/s', '# - bytes moved = 21.4212 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.89835 GByte/s', '# - solve arithmetic intensity = 0.109494 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.280s; elapsed, 0.279s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.570s; elapsed, 0.287s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.280s; elapsed, 0.279s, #calls: 1', 'TOTAL : CPU, 4.280s; elapsed, 0.279s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.260s; elapsed, 0.222s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.320s; elapsed, 0.020s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.790s; elapsed, 0.050s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.570s; elapsed, 0.287s, #calls: 1', 'TOTAL : CPU, 8.940s; elapsed, 0.578s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 3662.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.350s; elapsed, 0.350s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.049s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.027s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0856709', '# - matching time = 0.011832', '# - symmetrization time = 0.00476289', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0125041', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.29695 MB', '# - factor time = 0.112067', '# - factor nonzeros = 912,119', '# - factor memory = 7.29695 MB', '# - total flops = 2.03201e+08, min = 3.14439e+07, max = 8.73785e+07', '# - flop rate = 1.81321 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.1638e+08', '# --------------------------------------------', '# total = 2.1638e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.06571e-11\trel.res = 6.95539e-16\tbw.error = 1.13009e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.014571', '# - total flops = 2.7707e+06, min = 524653, max = 800713', '# - flop rate = 0.190153 GFlop/s', '# - bytes moved = 20.2222 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.38784 GByte/s', '# - solve arithmetic intensity = 0.137013 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.230s; elapsed, 0.250s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.240s; elapsed, 0.252s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.250s; elapsed, 0.253s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.250s; elapsed, 0.257s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.240s; elapsed, 0.252s, #calls: 1', 'TOTAL : CPU, 0.240s; elapsed, 0.252s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.230s; elapsed, 0.250s, #calls: 1', 'TOTAL : CPU, 0.230s; elapsed, 0.250s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.250s; elapsed, 0.253s, #calls: 1', 'TOTAL : CPU, 0.250s; elapsed, 0.253s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.350s; elapsed, 0.350s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.027s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.049s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.250s; elapsed, 0.257s, #calls: 1', 'TOTAL : CPU, 0.680s; elapsed, 0.683s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 3662.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.540s; elapsed, 0.273s', 'individual call time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.054s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.023s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# - nd time = # (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.0973389', '# - matching time = 0.013396', '# - symmetrization time = 0.00513792', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.016207', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.29695 MB', '# - factor time = 0.0980279', '# - factor nonzeros = 912,119', '# - factor memory = 7.29695 MB', '# - total flops = 2.03205e+08, min = 3.14439e+07, max = 8.73825e+07', '# - flop rate = 2.07293 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.1638e+08', '# --------------------------------------------', '# total = 2.1638e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.03211e-11\trel.res = 6.73609e-16\tbw.error = 1.13009e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.011354', '# - total flops = 2.7707e+06, min = 524653, max = 800713', '# - flop rate = 0.24403 GFlop/s', '# - bytes moved = 20.2271 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.7815 GByte/s', '# - solve arithmetic intensity = 0.13698 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.480s; elapsed, 0.249s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.500s; elapsed, 0.252s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.490s; elapsed, 0.251s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.510s; elapsed, 0.256s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.480s; elapsed, 0.249s, #calls: 1', 'TOTAL : CPU, 0.480s; elapsed, 0.249s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.490s; elapsed, 0.251s, #calls: 1', 'TOTAL : CPU, 0.490s; elapsed, 0.251s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.500s; elapsed, 0.252s, #calls: 1', 'TOTAL : CPU, 0.500s; elapsed, 0.252s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.540s; elapsed, 0.273s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.054s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.510s; elapsed, 0.256s, #calls: 1', 'TOTAL : CPU, 1.200s; elapsed, 0.606s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 3662.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.890s; elapsed, 0.234s', 'individual call time for EIGEN_LDLT: CPU, 0.220s; elapsed, 0.055s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.020s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0969551', '# - matching time = 0.013485', '# - symmetrization time = 0.00549507', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0172842', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.29695 MB', '# - factor time = 0.104505', '# - factor nonzeros = 912,119', '# - factor memory = 7.29695 MB', '# - total flops = 2.03205e+08, min = 3.14439e+07, max = 8.73825e+07', '# - flop rate = 1.94446 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.1638e+08', '# --------------------------------------------', '# total = 2.1638e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.03158e-11\trel.res = 6.73267e-16\tbw.error = 1.13009e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0154781', '# - total flops = 2.7707e+06, min = 524653, max = 800713', '# - flop rate = 0.179008 GFlop/s', '# - bytes moved = 20.2303 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.30703 GByte/s', '# - solve arithmetic intensity = 0.136958 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.040s; elapsed, 0.264s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.040s; elapsed, 0.266s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.040s; elapsed, 0.266s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.080s; elapsed, 0.270s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.040s; elapsed, 0.266s, #calls: 1', 'TOTAL : CPU, 1.040s; elapsed, 0.266s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.040s; elapsed, 0.264s, #calls: 1', 'TOTAL : CPU, 1.040s; elapsed, 0.264s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.040s; elapsed, 0.266s, #calls: 1', 'TOTAL : CPU, 1.040s; elapsed, 0.266s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.890s; elapsed, 0.234s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.020s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.220s; elapsed, 0.055s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.080s; elapsed, 0.270s, #calls: 1', 'TOTAL : CPU, 2.270s; elapsed, 0.579s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 3662.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.720s; elapsed, 0.231s', 'individual call time for EIGEN_LDLT: CPU, 0.450s; elapsed, 0.057s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.023s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', 'CPP -2', 'CPP -2', '8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,085', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.100645', '# - matching time = 0.015424', '# - symmetrization time = 0.00548506', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,085', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.01616', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.29695 MB', '# - factor time = 0.113947', '# - factor nonzeros = 912,119', '# - factor memory = 7.29695 MB', '# - total flops = 2.03211e+08, min = 3.14458e+07, max = 8.73836e+07', '# - flop rate = 1.78338 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.1638e+08', '# --------------------------------------------', '# total = 2.1638e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.01573e-11\trel.res = 6.62922e-16\tbw.error = 1.13691e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0131159', '# - total flops = 2.7707e+06, min = 524653, max = 800713', '# - flop rate = 0.211248 GFlop/s', '# - bytes moved = 20.2308 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.54246 GByte/s', '# - solve arithmetic intensity = 0.136955 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.160s; elapsed, 0.274s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.160s; elapsed, 0.276s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.150s; elapsed, 0.276s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.240s; elapsed, 0.280s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.150s; elapsed, 0.276s, #calls: 1', 'TOTAL : CPU, 2.150s; elapsed, 0.276s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.160s; elapsed, 0.276s, #calls: 1', 'TOTAL : CPU, 2.160s; elapsed, 0.276s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.160s; elapsed, 0.274s, #calls: 1', 'TOTAL : CPU, 2.160s; elapsed, 0.274s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.720s; elapsed, 0.231s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.180s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.450s; elapsed, 0.057s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.240s; elapsed, 0.280s, #calls: 1', 'TOTAL : CPU, 4.590s; elapsed, 0.591s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 1831.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.390s; elapsed, 0.386s', 'individual call time for EIGEN_LDLT: CPU, 0.060s; elapsed, 0.054s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.030s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,089', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# - nd time = # ******************************************************************', '0.09553', '# - matching time = 0.013062', '# - symmetrization time = 0.00384593', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,089', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.013221', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.52468 MB', '# - factor time = 0.135861', '# - factor nonzeros = 940,585', '# - factor memory = 7.52468 MB', '# - total flops = 2.24119e+08, min = 137603, max = 6.09266e+07', '# - flop rate = 1.64962 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.35469e+08', '# --------------------------------------------', '# total = 2.35469e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.0651e-11\trel.res = 6.95146e-16\tbw.error = 2.45888e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.013973', '# - total flops = 3.85024e+06, min = 18068, max = 796233', '# - flop rate = 0.275549 GFlop/s', '# - bytes moved = 19.5066 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.39602 GByte/s', '# - solve arithmetic intensity = 0.197382 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.270s; elapsed, 0.283s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.260s; elapsed, 0.283s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.290s; elapsed, 0.286s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.250s; elapsed, 0.284s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.250s; elapsed, 0.285s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.250s; elapsed, 0.285s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.260s; elapsed, 0.285s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.280s; elapsed, 0.289s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.270s; elapsed, 0.283s, #calls: 1', 'TOTAL : CPU, 0.270s; elapsed, 0.283s', 'Exiting profilerExiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.250s; elapsed, 0.284s, #calls: 1', 'TOTAL : CPU, 0.250s; elapsed, 0.284s', '', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.260s; elapsed, 0.283s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.283s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.290s; elapsed, 0.286s, #calls: 1Exiting profiler', 'TOTAL : CPU, 0.290s; elapsed, 0.286s', '', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.250s; elapsed, 0.285s, #calls: 1', 'TOTAL : CPU, 0.250s; elapsed, 0.285s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.250s; elapsed, 0.285s, #calls: 1', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.260s; elapsed, 0.285s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.285s', 'TOTAL : CPU, 0.250s; elapsed, 0.285s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.390s; elapsed, 0.386s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.030s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.060s; elapsed, 0.054s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.280s; elapsed, 0.289s, #calls: 1', 'TOTAL : CPU, 0.760s; elapsed, 0.758s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 1831.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.600s; elapsed, 0.303s', 'individual call time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.058s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.023s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', 'CPP -2', 'CPP -2', 'CPP -2', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,089', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.100465', '# - matching time = 0.013921', '# - symmetrization time = 0.00383902', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,089', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.01422', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.52468 MB', '# - factor time = 0.12021', '# - factor nonzeros = 940,585', '# - factor memory = 7.52468 MB', '# - total flops = 2.24122e+08, min = 137603, max = 6.09266e+07', '# - flop rate = 1.86442 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.35469e+08', '# --------------------------------------------', '# total = 2.35469e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.02036e-11\trel.res = 6.65946e-16\tbw.error = 2.45888e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0106299', '# - total flops = 3.85024e+06, min = 18068, max = 796233', '# - flop rate = 0.362209 GFlop/s', '# - bytes moved = 19.5114 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.83552 GByte/s', '# - solve arithmetic intensity = 0.197334 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.540s; elapsed, 0.275s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.540s; elapsed, 0.277s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.530s; elapsed, 0.276sindividual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.540s; elapsed, 0.278s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.540s; elapsed, 0.272s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.530s; elapsed, 0.276sindividual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.540s; elapsed, 0.276s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.560s; elapsed, 0.281s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.540s; elapsed, 0.277s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.277s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.540s; elapsed, 0.275s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.275s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.530s; elapsed, 0.276s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.276s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.540s; elapsed, 0.272s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.272s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.540s; elapsed, 0.276s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.276s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.530s; elapsed, 0.276s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.276s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.540s; elapsed, 0.278s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.278s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.600s; elapsed, 0.303s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.023s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.110s; elapsed, 0.058s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.560s; elapsed, 0.281s, #calls: 1', 'TOTAL : CPU, 1.320s; elapsed, 0.665s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 1831.375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.810s; elapsed, 0.213s', 'individual call time for EIGEN_LDLT: CPU, 0.200s; elapsed, 0.050s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.021s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,089', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.098094', '# - matching time = 0.013752', '# - symmetrization time = 0.00387096', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,089', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.015255', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.52468 MB', '# - factor time = 0.123755', '# - factor nonzeros = 940,585', '# - factor memory = 7.52468 MB', '# - total flops = 2.24127e+08, min = 139438, max = 6.09266e+07', '# - flop rate = 1.81105 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.35469e+08', '# --------------------------------------------', '# total = 2.35469e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.02458e-11\trel.res = 6.68696e-16\tbw.error = 2.45888e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0142281', '# - total flops = 3.85024e+06, min = 18068, max = 796233', '# - flop rate = 0.270608 GFlop/s', '# - bytes moved = 19.513 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.37144 GByte/s', '# - solve arithmetic intensity = 0.197317 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.090s; elapsed, 0.279s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.090s; elapsed, 0.279sindividual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.100s; elapsed, 0.279s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.090s; elapsed, 0.279s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.090s; elapsed, 0.278s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.100s; elapsed, 0.280s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.110s; elapsed, 0.281s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.130s; elapsed, 0.284s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.090s; elapsed, 0.279s, #calls: 1', 'TOTAL : CPU, 1.090s; elapsed, 0.279s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.090s; elapsed, 0.279s, #calls: 1', 'TOTAL : CPU, 1.090s; elapsed, 0.279s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.090s; elapsed, 0.279s, #calls: 1', 'TOTAL : CPU, 1.090s; elapsed, 0.279s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.100s; elapsed, 0.280s, #calls: 1', 'TOTAL : CPU, 1.100s; elapsed, 0.280s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.110s; elapsed, 0.281s, #calls: 1', 'TOTAL : CPU, 1.110s; elapsed, 0.281s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.100s; elapsed, 0.279s, #calls: 1', 'TOTAL : CPU, 1.100s; elapsed, 0.279s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.090s; elapsed, 0.278s, #calls: 1', 'TOTAL : CPU, 1.090s; elapsed, 0.278s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.810s; elapsed, 0.213s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.021s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.200s; elapsed, 0.050s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.130s; elapsed, 0.284s, #calls: 1', 'TOTAL : CPU, 2.230s; elapsed, 0.568s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 915.6875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.410s; elapsed, 0.410s', 'individual call time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.057s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.031s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,089', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.100674', '# - matching time = 0.014241', '# - symmetrization time = 0.00342011', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,089', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0159621', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.52468 MB', '# - factor time = 0.164789', '# - factor nonzeros = 940,585', '# - factor memory = 7.52468 MB', '# - total flops = 2.24119e+08, min = 137603, max = 5.97816e+07', '# - flop rate = 1.36004 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.32465e+08', '# --------------------------------------------', '# total = 2.32465e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.0008e-11\trel.res = 6.53181e-16\tbw.error = 1.63926e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0129209', '# - total flops = 5.89424e+06, min = 13488, max = 780496', '# - flop rate = 0.45618 GFlop/s', '# - bytes moved = 18.6028 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.43975 GByte/s', '# - solve arithmetic intensity = 0.316846 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.290s; elapsed, 0.317s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.260s; elapsed, 0.317s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.220s; elapsed, 0.317s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.300s; elapsed, 0.319s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.230s; elapsed, 0.318s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.260s; elapsed, 0.319s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.290s; elapsed, 0.318s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.260s; elapsed, 0.317s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.270s; elapsed, 0.318s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.310s; elapsed, 0.318s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.300s; elapsed, 0.318s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.280s; elapsed, 0.318s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.290s; elapsed, 0.317s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.290s; elapsed, 0.317s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.260s; elapsed, 0.318s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.320s; elapsed, 0.324s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.290s; elapsed, 0.317s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.317s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.260s; elapsed, 0.317s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.317s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.220s; elapsed, 0.317s, #calls: 1', 'TOTAL : CPU, 0.220s; elapsed, 0.317s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.300s; elapsed, 0.319s, #calls: 1', 'TOTAL : CPU, 0.300s; elapsed, 0.319s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.230s; elapsed, 0.318s, #calls: 1', 'TOTAL : CPU, 0.230s; elapsed, 0.318s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.310s; elapsed, 0.318s, #calls: 1', 'TOTAL : CPU, 0.310s; elapsed, 0.318s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.280s; elapsed, 0.318s, #calls: 1', 'TOTAL : CPU, 0.280s; elapsed, 0.318s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.300s; elapsed, 0.318s, #calls: 1', 'TOTAL : CPU, 0.300s; elapsed, 0.318s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.290s; elapsed, 0.318s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.318s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.270s; elapsed, 0.318s, #calls: 1', 'TOTAL : CPU, 0.270s; elapsed, 0.318s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.290s; elapsed, 0.317s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.317s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.260s; elapsed, 0.318s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.318s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.260s; elapsed, 0.319s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.319s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.260s; elapsed, 0.317s, #calls: 1', 'TOTAL : CPU, 0.260s; elapsed, 0.317s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.290s; elapsed, 0.317s, #calls: 1', 'TOTAL : CPU, 0.290s; elapsed, 0.317s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.410s; elapsed, 0.410s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.030s; elapsed, 0.031s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.050s; elapsed, 0.057s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.320s; elapsed, 0.324s, #calls: 1', 'TOTAL : CPU, 0.810s; elapsed, 0.822s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14651 915.6875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.570s; elapsed, 0.293s', 'individual call time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.050s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.025s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,651', '# - number of nonzeros = 72,805', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,089', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.# - nd time = ', '# ******************************************************************', '0.118312', '# - matching time = 0.0138209', '# - symmetrization time = 0.00447893', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,089', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0187731', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 7.52468 MB', '# - factor time = 0.175355', '# - factor nonzeros = 940,585', '# - factor memory = 7.52468 MB', '# - total flops = 2.24121e+08, min = 137603, max = 5.97816e+07', '# - flop rate = 1.2781 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 2.32465e+08', '# --------------------------------------------', '# total = 2.32465e+08', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 15322\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.0169e-11\trel.res = 6.63682e-16\tbw.error = 1.63926e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0114489', '# - total flops = 5.89424e+06, min = 13488, max = 780496', '# - flop rate = 0.514832 GFlop/s', '# - bytes moved = 18.6044 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.625 GByte/s', '# - solve arithmetic intensity = 0.31682 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.660s; elapsed, 0.359s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.690s; elapsed, 0.357s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.690s; elapsed, 0.357s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.690s; elapsed, 0.356s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.680s; elapsed, 0.356s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.660s; elapsed, 0.358s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.680s; elapsed, 0.354s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.680s; elapsed, 0.358s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.670s; elapsed, 0.357s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.660s; elapsed, 0.359s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.670s; elapsed, 0.357s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.690s; elapsed, 0.356s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.650s; elapsed, 0.357s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.680s; elapsed, 0.358s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.660s; elapsed, 0.350s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.720s; elapsed, 0.362s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.660s; elapsed, 0.359s, #calls: 1', 'TOTAL : CPU, 0.660s; elapsed, 0.359s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.660s; elapsed, 0.358s, #calls: 1', 'TOTAL : CPU, 0.660s; elapsed, 0.358s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.690s; elapsed, 0.357s, #calls: 1Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.690s; elapsed, 0.357s, #calls: 1', 'TOTAL : CPU, 0.690s; elapsed, 0.357s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.660s; elapsed, 0.359s, #calls: 1', 'TOTAL : CPU, 0.660s; elapsed, 0.359s', '', 'TOTAL : CPU, 0.690s; elapsed, 0.357s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.670s; elapsed, 0.357s, #calls: 1', 'TOTAL : CPU, 0.670s; elapsed, 0.357s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.680s; elapsed, 0.354s, #calls: 1', 'TOTAL : CPU, 0.680s; elapsed, 0.354s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.660s; elapsed, 0.350s, #calls: 1', 'TOTAL : CPU, 0.660s; elapsed, 0.350s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.680s; elapsed, 0.358s, #calls: 1', 'TOTAL : CPU, 0.680s; elapsed, 0.358s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.680s; elapsed, 0.356s, #calls: 1', 'TOTAL : CPU, 0.680s; elapsed, 0.356s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.690s; elapsed, 0.356s, #calls: 1', 'TOTAL : CPU, 0.690s; elapsed, 0.356s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.650s; elapsed, 0.357s, #calls: 1', 'TOTAL : CPU, 0.650s; elapsed, 0.357s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.680s; elapsed, 0.358s, #calls: 1', 'TOTAL : CPU, 0.680s; elapsed, 0.358s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.690s; elapsed, 0.356s, #calls: 1', 'TOTAL : CPU, 0.690s; elapsed, 0.356s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.670s; elapsed, 0.357s, #calls: 1', 'TOTAL : CPU, 0.670s; elapsed, 0.357s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.570s; elapsed, 0.293s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.050s; elapsed, 0.025s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.100s; elapsed, 0.050s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.720s; elapsed, 0.362s, #calls: 1', 'TOTAL : CPU, 1.440s; elapsed, 0.731s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 14151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.960s; elapsed, 0.956s', 'individual call time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.231s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.064s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.960s; elapsed, 0.956s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.064s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.231s, #calls: 1', 'TOTAL : CPU, 1.250s; elapsed, 1.251s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 14151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.330s; elapsed, 0.673s', 'individual call time for EIGEN_LDLT: CPU, 0.420s; elapsed, 0.233s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.052s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.330s; elapsed, 0.673s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.052s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.420s; elapsed, 0.233s, #calls: 1', 'TOTAL : CPU, 1.840s; elapsed, 0.957s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 14151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.770s; elapsed, 0.458s', 'individual call time for EIGEN_LDLT: CPU, 0.780s; elapsed, 0.229s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.045s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.770s; elapsed, 0.458s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.045s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.780s; elapsed, 0.229s, #calls: 1', 'TOTAL : CPU, 2.700s; elapsed, 0.732s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 14151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.230s; elapsed, 0.424s', 'individual call time for EIGEN_LDLT: CPU, 1.480s; elapsed, 0.235s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.040s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.230s; elapsed, 0.424s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.040s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.480s; elapsed, 0.235s, #calls: 1', 'TOTAL : CPU, 4.950s; elapsed, 0.699s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 14151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.290s; elapsed, 0.481s', 'individual call time for EIGEN_LDLT: CPU, 3.000s; elapsed, 0.255s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.039s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.290s; elapsed, 0.481s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.039s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.000s; elapsed, 0.255s, #calls: 1', 'TOTAL : CPU, 10.740s; elapsed, 0.775s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 7075.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.010s; elapsed, 1.023s', 'individual call time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.236s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.067s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0927382', '# - matching time = 0.0174839', '# - symmetrization time = 0.00676012', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0222569', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.2436 MB', '# - factor time = 0.627824', '# - factor nonzeros = 2,280,451', '# - factor memory = 18.2436 MB', '# - total flops = 1.23667e+09, min = 2.27838e+08, max = 1.00883e+09', '# - flop rate = 1.96977 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29515e+09', '# --------------------------------------------', '# total = 1.29515e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.61784e-11\trel.res = 9.99011e-16\tbw.error = 1.86482e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0196211', '# - total flops = 5.54918e+06, min = 2.00566e+06, max = 3.54351e+06', '# - flop rate = 0.282816 GFlop/s', '# - bytes moved = 44.0333 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.24418 GByte/s', '# - solve arithmetic intensity = 0.126022 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.780s; elapsed, 0.795s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.810s; elapsed, 0.803s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.780s; elapsed, 0.795s, #calls: 1', 'TOTAL : CPU, 0.780s; elapsed, 0.795s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.010s; elapsed, 1.023s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.067s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.236s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.810s; elapsed, 0.803s, #calls: 1', 'TOTAL : CPU, 2.120s; elapsed, 2.129s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 7075.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.450s; elapsed, 0.736s', 'individual call time for EIGEN_LDLT: CPU, 0.420s; elapsed, 0.232s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.057s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0953112', '# - matching time = 0.0179119', '# - symmetrization time = 0.00730801', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.026722', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.2436 MB', '# - factor time = 0.345325', '# - factor nonzeros = 2,280,451', '# - factor memory = 18.2436 MB', '# - total flops = 1.23673e+09, min = 2.27838e+08, max = 1.00889e+09', '# - flop rate = 3.58135 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29515e+09', '# --------------------------------------------', '# total = 1.29515e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.56822e-11\trel.res = 9.68375e-16\tbw.error = 1.86748e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0163159', '# - total flops = 5.55004e+06, min = 2.00566e+06, max = 3.54438e+06', '# - flop rate = 0.340161 GFlop/s', '# - bytes moved = 44.0936 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.70249 GByte/s', '# - solve arithmetic intensity = 0.12587 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.030s; elapsed, 0.521s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.050s; elapsed, 0.529s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.030s; elapsed, 0.521s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.521s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.450s; elapsed, 0.736s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.057s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.420s; elapsed, 0.232s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.050s; elapsed, 0.529s, #calls: 1', 'TOTAL : CPU, 3.020s; elapsed, 1.553s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 7075.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.650s; elapsed, 0.434s', 'individual call time for EIGEN_LDLT: CPU, 0.810s; elapsed, 0.258s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.045s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.106091', '# - matching time = 0.0196731', '# - symmetrization time = 0.00819397', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.027241', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.2436 MB', '# - factor time = 0.311231', '# - factor nonzeros = 2,280,451', '# - factor memory = 18.2436 MB', '# - total flops = 1.23683e+09, min = 2.27838e+08, max = 1.00899e+09', '# - flop rate = 3.97399 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29515e+09', '# --------------------------------------------', '# total = 1.29515e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.56097e-11\trel.res = 9.63895e-16\tbw.error = 1.88062e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0250859', '# - total flops = 5.55089e+06, min = 2.00566e+06, max = 3.54523e+06', '# - flop rate = 0.221275 GFlop/s', '# - bytes moved = 44.1217 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.75882 GByte/s', '# - solve arithmetic intensity = 0.125809 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.030s; elapsed, 0.512s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.080s; elapsed, 0.521s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.030s; elapsed, 0.512s, #calls: 1', 'TOTAL : CPU, 2.030s; elapsed, 0.512s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.650s; elapsed, 0.434s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.045s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.810s; elapsed, 0.258s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.080s; elapsed, 0.521s, #calls: 1', 'TOTAL : CPU, 4.680s; elapsed, 1.259s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 7075.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.860s; elapsed, 0.502s', 'individual call time for EIGEN_LDLT: CPU, 1.530s; elapsed, 0.244s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.041s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.106309', '# - matching time = 0.021683', '# - symmetrization time = 0.00837016', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0299191', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.2436 MB', '# - factor time = 0.325488', '# - factor nonzeros = 2,280,451', '# - factor memory = 18.2436 MB', '# - total flops = 1.23683e+09, min = 2.2784e+08, max = 1.00899e+09', '# - flop rate = 3.79994 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29515e+09', '# --------------------------------------------', '# total = 1.29515e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.54618e-11\trel.res = 9.54764e-16\tbw.error = 1.72307e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0175741', '# - total flops = 5.55089e+06, min = 2.00566e+06, max = 3.54523e+06', '# - flop rate = 0.315857 GFlop/s', '# - bytes moved = 44.1649 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.51307 GByte/s', '# - solve arithmetic intensity = 0.125686 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.130s; elapsed, 0.524s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.250s; elapsed, 0.533s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.130s; elapsed, 0.524s, #calls: 1', 'TOTAL : CPU, 4.130s; elapsed, 0.524s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.860s; elapsed, 0.502s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.041s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.530s; elapsed, 0.244s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.250s; elapsed, 0.533s, #calls: 1', 'TOTAL : CPU, 9.890s; elapsed, 1.321s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 7075.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 7.190s; elapsed, 0.477s', 'individual call time for EIGEN_LDLT: CPU, 2.960s; elapsed, 0.260s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.039s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0958111', '# - matching time = 0.0230899', '# - symmetrization time = 0.00903606', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0359759', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.2436 MB', '# - factor time = 0.419766', '# - factor nonzeros = 2,280,451', '# - factor memory = 18.2436 MB', '# - total flops = 1.23684e+09, min = 2.27842e+08, max = 1.009e+09', '# - flop rate = 2.94649 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29515e+09', '# --------------------------------------------', '# total = 1.29515e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.53985e-11\trel.res = 9.50857e-16\tbw.error = 1.71176e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0232131', '# - total flops = 5.55089e+06, min = 2.00566e+06, max = 3.54523e+06', '# - flop rate = 0.239127 GFlop/s', '# - bytes moved = 44.1973 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.90398 GByte/s', '# - solve arithmetic intensity = 0.125593 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.570s; elapsed, 0.623s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.190s; elapsed, 0.638s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 9.570s; elapsed, 0.623s, #calls: 1', 'TOTAL : CPU, 9.570s; elapsed, 0.623s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 7.190s; elapsed, 0.477s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.450s; elapsed, 0.039s, #calls: 1', 'time for EIGEN_LDLT: CPU, 2.960s; elapsed, 0.260s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 10.190s; elapsed, 0.638s, #calls: 1', 'TOTAL : CPU, 20.790s; elapsed, 1.413s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 3537.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.090s; elapsed, 1.086s', 'individual call time for EIGEN_LDLT: CPU, 0.250s; elapsed, 0.249s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.075s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.104737', '# - matching time = 0.0201399', '# - symmetrization time = 0.00852895', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.019562', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.337 MB', '# - factor time = 0.363493', '# - factor nonzeros = 2,292,125', '# - factor memory = 18.337 MB', '# - total flops = 1.23992e+09, min = 1.94869e+08, max = 4.82492e+08', '# - flop rate = 3.41113 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29573e+09', '# --------------------------------------------', '# total = 1.29573e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.58604e-11\trel.res = 9.79375e-16\tbw.error = 1.66213e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0190072', '# - total flops = 7.0497e+06, min = 1.35975e+06, max = 2.14762e+06', '# - flop rate = 0.370896 GFlop/s', '# - bytes moved = 39.6406 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.08556 GByte/s', '# - solve arithmetic intensity = 0.17784 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.520s; elapsed, 0.544s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.540s; elapsed, 0.540s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.530s; elapsed, 0.542s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.550s; elapsed, 0.545s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.520s; elapsed, 0.544s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.544s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.540s; elapsed, 0.540s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.540s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.530s; elapsed, 0.542s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.542s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.090s; elapsed, 1.086s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.075s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.250s; elapsed, 0.249s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.550s; elapsed, 0.545s, #calls: 1', 'TOTAL : CPU, 1.970s; elapsed, 1.955s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 3537.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.420s; elapsed, 0.727s', 'individual call time for EIGEN_LDLT: CPU, 0.440s; elapsed, 0.244s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.057s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.107379', '# - matching time = 0.0202992', '# - symmetrization time = 0.010191', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0238872', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.337 MB', '# - factor time = 0.376026', '# - factor nonzeros = 2,292,125', '# - factor memory = 18.337 MB', '# - total flops = 1.23994e+09, min = 1.94869e+08, max = 4.82507e+08', '# - flop rate = 3.29748 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29573e+09', '# --------------------------------------------', '# total = 1.29573e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.57167e-11\trel.res = 9.70501e-16\tbw.error = 1.65132e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0177009', '# - total flops = 7.0497e+06, min = 1.35975e+06, max = 2.14762e+06', '# - flop rate = 0.398268 GFlop/s', '# - bytes moved = 39.6513 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.24007 GByte/s', '# - solve arithmetic intensity = 0.177792 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.120s; elapsed, 0.564s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.120s; elapsed, 0.570s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.120s; elapsed, 0.569s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.140s; elapsed, 0.575s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.120s; elapsed, 0.570s, #calls: 1', 'TOTAL : CPU, 1.120s; elapsed, 0.570s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.120s; elapsed, 0.564s, #calls: 1', 'TOTAL : CPU, 1.120s; elapsed, 0.564s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.120s; elapsed, 0.569s, #calls: 1', 'TOTAL : CPU, 1.120s; elapsed, 0.569s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.420s; elapsed, 0.727s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.100s; elapsed, 0.057s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.440s; elapsed, 0.244s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.140s; elapsed, 0.575s, #calls: 1', 'TOTAL : CPU, 3.100s; elapsed, 1.602s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 3537.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.660s; elapsed, 0.431s', 'individual call time for EIGEN_LDLT: CPU, 0.780s; elapsed, 0.231s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.046s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# - nd time = # ******************************************************************', '0.095103', '# - matching time = 0.0185831', '# - symmetrization time = 0.0106421', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0258', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.337 MB', '# - factor time = 0.31179', '# - factor nonzeros = 2,292,125', '# - factor memory = 18.337 MB', '# - total flops = 1.23994e+09, min = 1.94869e+08, max = 4.82507e+08', '# - flop rate = 3.97684 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29573e+09', '# --------------------------------------------', '# total = 1.29573e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.56816e-11\trel.res = 9.68339e-16\tbw.error = 1.64794e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.024853', '# - total flops = 7.0497e+06, min = 1.35975e+06, max = 2.14762e+06', '# - flop rate = 0.283656 GFlop/s', '# - bytes moved = 39.6649 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.59598 GByte/s', '# - solve arithmetic intensity = 0.177731 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.980s; elapsed, 0.503s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.970s; elapsed, 0.500s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.960s; elapsed, 0.496s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.040s; elapsed, 0.507s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.980s; elapsed, 0.503s, #calls: 1', 'TOTAL : CPU, 1.980s; elapsed, 0.503s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.960s; elapsed, 0.496s, #calls: 1', 'TOTAL : CPU, 1.960s; elapsed, 0.496s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.970s; elapsed, 0.500s, #calls: 1', 'TOTAL : CPU, 1.970s; elapsed, 0.500s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.660s; elapsed, 0.431s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.046s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.780s; elapsed, 0.231s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.040s; elapsed, 0.507s, #calls: 1', 'TOTAL : CPU, 4.640s; elapsed, 1.216s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 3537.75', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.470s; elapsed, 0.456s', 'individual call time for EIGEN_LDLT: CPU, 1.530s; elapsed, 0.247s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.040s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.106532', '# - matching time = 0.0211408', '# - symmetrization time = 0.010596', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0266249', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.337 MB', '# - factor time = 0.344514', '# - factor nonzeros = 2,292,125', '# - factor memory = 18.337 MB', '# - total flops = 1.23995e+09, min = 1.9487e+08, max = 4.82511e+08', '# - flop rate = 3.59911 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29573e+09', '# --------------------------------------------', '# total = 1.29573e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.5656e-11\trel.res = 9.66757e-16\tbw.error = 1.65316e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0284679', '# - total flops = 7.0497e+06, min = 1.35975e+06, max = 2.14762e+06', '# - flop rate = 0.247637 GFlop/s', '# - bytes moved = 39.6928 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.3943 GByte/s', '# - solve arithmetic intensity = 0.177606 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.350s; elapsed, 0.552s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.340s; elapsed, 0.549s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.350s; elapsed, 0.552s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.450s; elapsed, 0.557s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 4.340s; elapsed, 0.549s, #calls: 1', 'TOTAL : CPU, 4.340s; elapsed, 0.549s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 4.350s; elapsed, 0.552s, #calls: 1', 'TOTAL : CPU, 4.350s; elapsed, 0.552s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 4.350s; elapsed, 0.552s, #calls: 1', 'TOTAL : CPU, 4.350s; elapsed, 0.552s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.470s; elapsed, 0.456s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.040s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.530s; elapsed, 0.247s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 4.450s; elapsed, 0.557s, #calls: 1', 'TOTAL : CPU, 9.690s; elapsed, 1.301s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 1768.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.080s; elapsed, 1.084s', 'individual call time for EIGEN_LDLT: CPU, 0.250s; elapsed, 0.251s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.075s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 66', '# - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.104628', '# - matching time = 0.0201089', '# - symmetrization time = 0.0100362', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0207939', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.337 MB', '# - factor time = 0.393152', '# - factor nonzeros = 2,292,125', '# - factor memory = 18.337 MB', '# - total flops = 1.23992e+09, min = 695096, max = 3.55062e+08', '# - flop rate = 3.1538 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29007e+09', '# --------------------------------------------', '# total = 1.29007e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.44884e-11\trel.res = 8.94653e-16\tbw.error = 1.6174e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0274701', '# - total flops = 9.317e+06, min = 26908, max = 2.00677e+06', '# - flop rate = 0.339168 GFlop/s', '# - bytes moved = 38.3707 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.39681 GByte/s', '# - solve arithmetic intensity = 0.242816 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.540s; elapsed, 0.588s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.560s; elapsed, 0.588s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.570s; elapsed, 0.587s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.580s; elapsed, 0.586s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.570s; elapsed, 0.588s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.570s; elapsed, 0.588s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.540s; elapsed, 0.584s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.590s; elapsed, 0.592s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.580s; elapsed, 0.586s, #calls: 1', 'TOTAL : CPU, 0.580s; elapsed, 0.586s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.560s; elapsed, 0.588s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.588s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.540s; elapsed, 0.588s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.588s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.570s; elapsed, 0.587s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.587s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.570s; elapsed, 0.588s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.588s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.570s; elapsed, 0.588s, #calls: 1', 'TOTAL : CPU, 0.570s; elapsed, 0.588s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.540s; elapsed, 0.584s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.584s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.080s; elapsed, 1.084s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.075s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.250s; elapsed, 0.251s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.590s; elapsed, 0.592s, #calls: 1', 'TOTAL : CPU, 2.000s; elapsed, 2.002s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 1768.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.290s; elapsed, 0.660s', 'individual call time for EIGEN_LDLT: CPU, 0.440s; elapsed, 0.256s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.064s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threadsCPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.111865', '# - matching time = 0.0211282', '# - symmetrization time = 0.0121448', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0255361', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.337 MB', '# - factor time = 0.322611', '# - factor nonzeros = 2,292,125', '# - factor memory = 18.337 MB', '# - total flops = 1.23993e+09, min = 695096, max = 3.55071e+08', '# - flop rate = 3.84342 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29007e+09', '# --------------------------------------------', '# total = 1.29007e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.4423e-11\trel.res = 8.90617e-16\tbw.error = 1.6174e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.021487', '# - total flops = 9.317e+06, min = 26908, max = 2.00677e+06', '# - flop rate = 0.433611 GFlop/s', '# - bytes moved = 38.3791 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.78615 GByte/s', '# - solve arithmetic intensity = 0.242762 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.950s; elapsed, 0.531s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.000s; elapsed, 0.532s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.040s; elapsed, 0.531s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.030s; elapsed, 0.526s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.010s; elapsed, 0.531s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.990s; elapsed, 0.533s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.060s; elapsed, 0.531s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.060s; elapsed, 0.536s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.010s; elapsed, 0.531s, #calls: 1', 'TOTAL : CPU, 1.010s; elapsed, 0.531s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.000s; elapsed, 0.532s, #calls: 1', 'TOTAL : CPU, 1.000s; elapsed, 0.532s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.040s; elapsed, 0.531s, #calls: 1', 'TOTAL : CPU, 1.040s; elapsed, 0.531s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.060s; elapsed, 0.531s, #calls: 1', 'TOTAL : CPU, 1.060s; elapsed, 0.531s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.950s; elapsed, 0.531s, #calls: 1', 'TOTAL : CPU, 0.950s; elapsed, 0.531s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.990s; elapsed, 0.533s, #calls: 1', 'TOTAL : CPU, 0.990s; elapsed, 0.533s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 1.030s; elapsed, 0.526s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.526s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.290s; elapsed, 0.660s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.120s; elapsed, 0.064s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.440s; elapsed, 0.256s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.060s; elapsed, 0.536s, #calls: 1', 'TOTAL : CPU, 2.910s; elapsed, 1.515s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 1768.875', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 2.700s; elapsed, 0.692s', 'individual call time for EIGEN_LDLT: CPU, 0.830s; elapsed, 0.269s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.048s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5CPP -2', 'CPP -2', ' = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.122496', '# - matching time = 0.021405', '# - symmetrization time = 0.012846', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0293779', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.337 MB', '# - factor time = 0.345172', '# - factor nonzeros = 2,292,125', '# - factor memory = 18.337 MB', '# - total flops = 1.23994e+09, min = 696756, max = 3.55075e+08', '# - flop rate = 3.59223 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.29007e+09', '# --------------------------------------------', '# total = 1.29007e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.43497e-11\trel.res = 8.86089e-16\tbw.error = 1.61805e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0230069', '# - total flops = 9.317e+06, min = 26908, max = 2.00677e+06', '# - flop rate = 0.404965 GFlop/s', '# - bytes moved = 38.3901 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.66863 GByte/s', '# - solve arithmetic intensity = 0.242693 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.120s; elapsed, 0.569s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.240s; elapsed, 0.569s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.240s; elapsed, 0.569s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.190s; elapsed, 0.569s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.020s; elapsed, 0.568s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.920s; elapsed, 0.568s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.190s; elapsed, 0.563s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.270s; elapsed, 0.573s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 2.120s; elapsed, 0.569s, #calls: 1', 'TOTAL : CPU, 2.120s; elapsed, 0.569s', 'Exiting profiler', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.920s; elapsed, 0.568s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 2.240s; elapsed, 0.569s, #calls: 1', 'TOTAL : CPU, 2.240s; elapsed, 0.569s', 'TOTAL : CPU, 1.920s; elapsed, 0.568s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 2.190s; elapsed, 0.569s, #calls: 1', 'TOTAL : CPU, 2.190s; elapsed, 0.569s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 2.240s; elapsed, 0.569s, #calls: 1', 'TOTAL : CPU, 2.240s; elapsed, 0.569s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 2.190s; elapsed, 0.563s, #calls: 1', 'TOTAL : CPU, 2.190s; elapsed, 0.563s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 2.020s; elapsed, 0.568s, #calls: 1', 'TOTAL : CPU, 2.020s; elapsed, 0.568s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 2.700s; elapsed, 0.692s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.160s; elapsed, 0.048s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.830s; elapsed, 0.269s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 2.270s; elapsed, 0.573s, #calls: 1', 'TOTAL : CPU, 5.960s; elapsed, 1.582s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 884.4375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.110s; elapsed, 1.108s', 'individual call time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.239s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.071s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 65', '# - nd time = 0.0964961# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '', '# - matching time = 0.019587', '# - symmetrization time = 0.00788498', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0203669', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.337 MB', '# - factor time = 0.391499', '# - factor nonzeros = 2,292,125', '# - factor memory = 18.337 MB', '# - total flops = 1.23992e+09, min = 695096, max = 3.15217e+08', '# - flop rate = 3.16711 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.27889e+09', '# --------------------------------------------', '# total = 1.27889e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.32291e-11\trel.res = 8.16895e-16\tbw.error = 1.60964e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.019027', '# - total flops = 1.46854e+07, min = 22488, max = 1.96095e+06', '# - flop rate = 0.771818 GFlop/s', '# - bytes moved = 36.3296 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.90937 GByte/s', '# - solve arithmetic intensity = 0.404226 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.500s; elapsed, 0.565s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.510s; elapsed, 0.564s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.560s; elapsed, 0.564s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.540s; elapsed, 0.566s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.550s; elapsed, 0.562s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.540s; elapsed, 0.560s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.520s; elapsed, 0.561s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.530s; elapsed, 0.562s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.520s; elapsed, 0.563s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.560s; elapsed, 0.565s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.560s; elapsed, 0.563s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.550s; elapsed, 0.563s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.490s; elapsed, 0.563s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.520s; elapsed, 0.564s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.540s; elapsed, 0.562s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.570s; elapsed, 0.569s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 0.540s; elapsed, 0.566s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.566s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 0.500s; elapsed, 0.565s, #calls: 1', 'TOTAL : CPU, 0.500s; elapsed, 0.565s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.560s; elapsed, 0.565s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.565s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.510s; elapsed, 0.564s, #calls: 1', 'TOTAL : CPU, 0.510s; elapsed, 0.564s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 0.560s; elapsed, 0.564s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.564s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 0.550s; elapsed, 0.562s, #calls: 1', 'TOTAL : CPU, 0.550s; elapsed, 0.562s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 0.520s; elapsed, 0.563s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.563s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 0.540s; elapsed, 0.560s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.560s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 0.520s; elapsed, 0.564s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.564s', 'Exiting profiler', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 0.560s; elapsed, 0.563s, #calls: 1', 'TOTAL : CPU, 0.560s; elapsed, 0.563s', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 0.530s; elapsed, 0.562s, #calls: 1', 'TOTAL : CPU, 0.530s; elapsed, 0.562s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.490s; elapsed, 0.563s, #calls: 1', 'TOTAL : CPU, 0.490s; elapsed, 0.563s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 0.540s; elapsed, 0.562s, #calls: 1', 'TOTAL : CPU, 0.540s; elapsed, 0.562s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 0.520s; elapsed, 0.561s, #calls: 1', 'TOTAL : CPU, 0.520s; elapsed, 0.561s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 0.550s; elapsed, 0.563s, #calls: 1', 'TOTAL : CPU, 0.550s; elapsed, 0.563s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.110s; elapsed, 1.108s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.070s; elapsed, 0.071s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.239s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.570s; elapsed, 0.569s, #calls: 1', 'TOTAL : CPU, 1.990s; elapsed, 1.987s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '14151 884.4375', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.500s; elapsed, 0.761s', 'individual call time for EIGEN_LDLT: CPU, 0.440s; elapsed, 0.260s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.060s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3CPP -2', 'CPP -2', 'CPP -2', '', 'CPP -2', 'CPP -2CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 14,151', '# - number of nonzeros = 131,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,059', '# - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.117965', '# - matching time = 0.0231719', '# - symmetrization time = 0.00788593', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,059', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.024848', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.337 MB', '# - factor time = 0.343292', '# - factor nonzeros = 2,292,125', '# - factor memory = 18.337 MB', '# - total flops = 1.23993e+09, min = 695096, max = 3.15217e+08', '# - flop rate = 3.61187 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.27889e+09', '# --------------------------------------------', '# total = 1.27889e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.36283e-11\trel.res = 8.41546e-16\tbw.error = 1.58062e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0186291', '# - total flops = 1.46854e+07, min = 22488, max = 1.96095e+06', '# - flop rate = 0.788305 GFlop/s', '# - bytes moved = 36.3336 MB, min = 0 MB, max = 0 MB', '# - byte rate = 1.95037 GByte/s', '# - solve arithmetic intensity = 0.404182 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.010s; elapsed, 0.547s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.010s; elapsed, 0.549s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU, 1.060s; elapsed, 0.550s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU, 1.060s; elapsed, 0.546s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.020s; elapsed, 0.547s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.990s; elapsed, 0.548s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU, 1.070s; elapsed, 0.548s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU, 1.070s; elapsed, 0.545s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.020s; elapsed, 0.548s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.030s; elapsed, 0.548s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.020s; elapsed, 0.548s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.990s; elapsed, 0.548s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU, 1.060s; elapsed, 0.548s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU, 1.080s; elapsed, 0.548s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU, 1.090s; elapsed, 0.549s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.090s; elapsed, 0.554s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=10: CPU, 1.060s; elapsed, 0.550s, #calls: 1', 'TOTAL : CPU, 1.060s; elapsed, 0.550s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=3: CPU, 1.010s; elapsed, 0.549s, #calls: 1', 'TOTAL : CPU, 1.010s; elapsed, 0.549s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=12: CPU, 1.080s; elapsed, 0.548s, #calls: 1', 'TOTAL : CPU, 1.080s; elapsed, 0.548s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=11: CPU, 1.070s; elapsed, 0.548s, #calls: 1', 'TOTAL : CPU, 1.070s; elapsed, 0.548s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=4: CPU, 1.030s; elapsed, 0.548s, #calls: 1', 'TOTAL : CPU, 1.030s; elapsed, 0.548s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=8: CPU, 0.990s; elapsed, 0.548s, #calls: 1', 'TOTAL : CPU, 0.990s; elapsed, 0.548s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=15: CPU, 1.060s; elapsed, 0.546s, #calls: 1', 'TOTAL : CPU, 1.060s; elapsed, 0.546s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=9: CPU, 1.060s; elapsed, 0.548s, #calls: 1', 'TOTAL : CPU, 1.060s; elapsed, 0.548s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=2: CPU, 1.010s; elapsed, 0.547s, #calls: 1', 'TOTAL : CPU, 1.010s; elapsed, 0.547s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=13: CPU, 1.090s; elapsed, 0.549s, #calls: 1', 'TOTAL : CPU, 1.090s; elapsed, 0.549s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=14: CPU, 1.070s; elapsed, 0.545s, #calls: 1', 'TOTAL : CPU, 1.070s; elapsed, 0.545s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=5: CPU, 1.020s; elapsed, 0.547s, #calls: 1', 'TOTAL : CPU, 1.020s; elapsed, 0.547s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=7: CPU, 0.990s; elapsed, 0.548s, #calls: 1', 'TOTAL : CPU, 0.990s; elapsed, 0.548s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.020s; elapsed, 0.548s, #calls: 1', 'Exiting profiler', 'TOTAL : CPU, 1.020s; elapsed, 0.548s', 'time for STRUMPACK_MPI_DIST_RANK=6: CPU, 1.020s; elapsed, 0.548s, #calls: 1', 'TOTAL : CPU, 1.020s; elapsed, 0.548s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.500s; elapsed, 0.761s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.110s; elapsed, 0.060s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.440s; elapsed, 0.260s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.090s; elapsed, 0.554s, #calls: 1', 'TOTAL : CPU, 3.140s; elapsed, 1.635s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 15151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.930s; elapsed, 0.929s', 'individual call time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.231s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.066s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.930s; elapsed, 0.929s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.060s; elapsed, 0.066s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.230s; elapsed, 0.231s, #calls: 1', 'TOTAL : CPU, 1.220s; elapsed, 1.225s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 15151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.260s; elapsed, 0.640s', 'individual call time for EIGEN_LDLT: CPU, 0.420s; elapsed, 0.235s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.051s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.260s; elapsed, 0.640s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.051s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.420s; elapsed, 0.235s, #calls: 1', 'TOTAL : CPU, 1.770s; elapsed, 0.925s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 15151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.550s; elapsed, 0.406s', 'individual call time for EIGEN_LDLT: CPU, 0.790s; elapsed, 0.237s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.047s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.550s; elapsed, 0.406s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.140s; elapsed, 0.047s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.790s; elapsed, 0.237s, #calls: 1', 'TOTAL : CPU, 2.480s; elapsed, 0.690s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 15151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.170s; elapsed, 0.417s', 'individual call time for EIGEN_LDLT: CPU, 1.500s; elapsed, 0.230s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.040s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.170s; elapsed, 0.417s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.240s; elapsed, 0.040s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.500s; elapsed, 0.230s, #calls: 1', 'TOTAL : CPU, 4.910s; elapsed, 0.687s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 15151.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 6.910s; elapsed, 0.457s', 'individual call time for EIGEN_LDLT: CPU, 3.050s; elapsed, 0.245s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.490s; elapsed, 0.041s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 6.910s; elapsed, 0.457s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.490s; elapsed, 0.041s, #calls: 1', 'time for EIGEN_LDLT: CPU, 3.050s; elapsed, 0.245s, #calls: 1', 'TOTAL : CPU, 10.450s; elapsed, 0.743s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 7575.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 0.930s; elapsed, 0.934s', 'individual call time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.244s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.075s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 15,151', '# - number of nonzeros = 132,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.0941319', '# - matching time = 0.0169628', '# - symmetrization time = 0.00690103', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.0212379', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.3274 MB', '# - factor time = 0.588023', '# - factor nonzeros = 2,290,919', '# - factor memory = 18.3274 MB', '# - total flops = 1.18081e+09, min = 1.91931e+08, max = 9.88882e+08', '# - flop rate = 2.00811 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.23655e+09', '# --------------------------------------------', '# total = 1.23655e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 2.02213e-11\trel.res = 1.24866e-15\tbw.error = 2.16324e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0172689', '# - total flops = 5.5745e+06, min = 1.86431e+06, max = 3.71019e+06', '# - flop rate = 0.322806 GFlop/s', '# - bytes moved = 44.3976 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.57096 GByte/s', '# - solve arithmetic intensity = 0.125559 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.750s; elapsed, 0.753s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.760s; elapsed, 0.760s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 0.750s; elapsed, 0.753s, #calls: 1', 'TOTAL : CPU, 0.750s; elapsed, 0.753s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 0.930s; elapsed, 0.934s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.080s; elapsed, 0.075s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.240s; elapsed, 0.244s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 0.760s; elapsed, 0.760s, #calls: 1', 'TOTAL : CPU, 2.010s; elapsed, 2.013s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 7575.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.260s; elapsed, 0.643s', 'individual call time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.243s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.050s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 15,151', '# - number of nonzeros = 132,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.099951', '# - matching time = 0.018383', '# - symmetrization time = 0.00715709', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.027391', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.3274 MB', '# - factor time = 0.420462', '# - factor nonzeros = 2,290,919', '# - factor memory = 18.3274 MB', '# - total flops = 1.18089e+09, min = 1.91931e+08, max = 9.88957e+08', '# - flop rate = 2.80855 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.23655e+09', '# --------------------------------------------', '# total = 1.23655e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.90247e-11\trel.res = 1.17477e-15\tbw.error = 2.15866e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.0170031', '# - total flops = 5.57537e+06, min = 1.86431e+06, max = 3.71106e+06', '# - flop rate = 0.327904 GFlop/s', '# - bytes moved = 44.459 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.61476 GByte/s', '# - solve arithmetic intensity = 0.125405 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.190s; elapsed, 0.605s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.230s; elapsed, 0.614s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.190s; elapsed, 0.605s, #calls: 1', 'TOTAL : CPU, 1.190s; elapsed, 0.605s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.260s; elapsed, 0.643s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.090s; elapsed, 0.050s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.430s; elapsed, 0.243s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.230s; elapsed, 0.614s, #calls: 1', 'TOTAL : CPU, 3.010s; elapsed, 1.551s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 7575.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 1.850s; elapsed, 0.477s', 'individual call time for EIGEN_LDLT: CPU, 0.810s; elapsed, 0.244s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.045s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 15,151', '# - number of nonzeros = 132,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.100378', '# - matching time = 0.018724', '# - symmetrization time = 0.00724792', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.029701', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.3274 MB', '# - factor time = 0.283501', '# - factor nonzeros = 2,290,919', '# - factor memory = 18.3274 MB', '# - total flops = 1.18098e+09, min = 1.91931e+08, max = 9.89054e+08', '# - flop rate = 4.16572 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.23655e+09', '# --------------------------------------------', '# total = 1.23655e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.91725e-11\trel.res = 1.1839e-15\tbw.error = 2.17012e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.018209', '# - total flops = 5.57619e+06, min = 1.86431e+06, max = 3.71188e+06', '# - flop rate = 0.306233 GFlop/s', '# - bytes moved = 44.4884 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.44321 GByte/s', '# - solve arithmetic intensity = 0.12534 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.880s; elapsed, 0.476s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.940s; elapsed, 0.485s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 1.880s; elapsed, 0.476s, #calls: 1', 'TOTAL : CPU, 1.880s; elapsed, 0.476s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 1.850s; elapsed, 0.477s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.150s; elapsed, 0.045s, #calls: 1', 'time for EIGEN_LDLT: CPU, 0.810s; elapsed, 0.244s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 1.940s; elapsed, 0.485s, #calls: 1', 'TOTAL : CPU, 4.750s; elapsed, 1.251s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', ' try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', ' try: mod = __import__(name)', '15151 7575.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU, 3.440s; elapsed, 0.452s', 'individual call time for EIGEN_LDLT: CPU, 1.550s; elapsed, 0.231s', 'individual call time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.041s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '# - number of unknowns = 15,151', '# - number of nonzeros = 132,459', '# nested dissection reordering:', '# - Scotch reordering', '# - strategy parameter = 8', '# - number of separators = 4,075', '# - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# - nd time = 0.100607', '# - matching time = 0.019443', '# - symmetrization time = 0.00749493', '# symbolic factorization:', '# - nr of dense Frontal matrices = 4,075', '# - nr of HSS Frontal matrices = 0', '# - symb-factor time = 0.028475', '# multifrontal factorization:', '# - estimated memory usage (exact solver) = 18.3274 MB', '# - factor time = 0.277619', '# - factor nonzeros = 2,290,919', '# - factor memory = 18.3274 MB', '# - total flops = 1.18099e+09, min = 1.91933e+08, max = 9.89054e+08', '# - flop rate = 4.25398 GFlop/s', '# - factor memory/nonzeros = 100 % of multifrontal', '# - maximum HSS rank = 0', '# - HSS compression = false', '# - relative compression tolerance = 0.01', '# - absolute compression tolerance = 1e-08', '# - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression = 0', '# random = 0', '# ID = 0', '# QR = 0', '# ortho = 0', '# reduce_samples = 0', '# update_samples = 0', '# extraction = 0', '# sampling = 0', '# CB_sample = 0', '# sparse_sampling = 0', '# ULV_factor = 0', '# Schur = 0', '# full_rank = 1.23655e+09', '# --------------------------------------------', '# total = 1.23655e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres = 16194.4\trel.res = 1\tbw.error = 1', 'REFINEMENT it. 1\tres = 1.86499e-11\trel.res = 1.15163e-15\tbw.error = 2.17127e-15', '# DIRECT/GMRES solve:', '# - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '# - number of Krylov iterations = 1', '# - solve time = 0.017904', '# - total flops = 5.57619e+06, min = 1.86431e+06, max = 3.71188e+06', '# - flop rate = 0.311449 GFlop/s', '# - bytes moved = 44.5335 MB, min = 0 MB, max = 0 MB', '# - byte rate = 2.48734 GByte/s', '# - solve arithmetic intensity = 0.125213 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.670s; elapsed, 0.465s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.790s; elapsed, 0.473s', 'Exiting profiler', 'time for STRUMPACK_MPI_DIST_RANK=1: CPU, 3.670s; elapsed, 0.465s, #calls: 1', 'TOTAL : CPU, 3.670s; elapsed, 0.465s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for STRUMPACK_OMP: CPU, 3.440s; elapsed, 0.452s, #calls: 1', 'time for EIGEN_BICGSTAB: CPU, 0.250s; elapsed, 0.041s, #calls: 1', 'time for EIGEN_LDLT: CPU, 1.550s; elapsed, 0.231s, #calls: 1', 'time for STRUMPACK_MPI_DIST_RANK=0: CPU, 3.790s; elapsed, 0.473s, #calls: 1', 'TOTAL : CPU, 9.030s; elapsed, 1.198s']
mpirun -n 2
OMP_NUM_THREADS:=16
import json, cPickle
with open("OMP_SOLVER_TIME_1k_MPI.json", "w") as OMP_SOLVER_TIME_FILE:
OMP_SOLVER_TIME_FILE.write(json.dumps(str_out))
with open("OMP_SOLVER_TIME_1k_MPI.pickle", "w") as OMP_SOLVER_TIME_FILE:
OMP_SOLVER_TIME_FILE.write(cPickle.dumps(str_out))
import cPickle
str_out=cPickle.load(open('OMP_SOLVER_TIME_1k_MPI.pickle', 'rb'))
str_out_sm={}
for k in str_out.keys():
str_out_sm.update({k.replace('levmar.parameter_flags=','').replace('_strum_1k_omp1_params','_'):str_out[k]})
import numpy as np
u_dat = {}
threads_list = [1,2,4,8,16]
mpi_proc_list = [1,2,4,8,16]
uniq_ref = set([k.split('_')[1] for k in str_out_sm.keys()])
df_list=[]
for u in uniq_ref:
same_t = lambda: None #Functions are objects, so legit
same_t.strum_scotch_a = np.empty(shape=(5,5))
same_t.strum_mpi_ptscotch = np.empty(shape=(5,5))
same_t.eig_ldlt = np.empty(shape=(5,5))
same_t.eig_bicgstab = np.empty(shape=(5,5))
same_t.strum_scotch_a[:] = np.nan
same_t.strum_mpi_ptscotch[:] = np.nan
same_t.eig_ldlt[:] = np.nan
same_t.eig_bicgstab[:] = np.nan
for m_idx,m in enumerate(mpi_proc_list):
for t_idx,t in enumerate(threads_list):
if m*t <=32:
t_STRUMPACK_OMP = [ s for s in str_out_sm["mpi%domp%d_%s"%(m,t,u)] if 'STRUMPACK_OMP' in s]
t_STRUMPACK_MPI_DIST_RANK = [ s for s in str_out_sm["mpi%domp%d_%s"%(m,t,u)] if 'STRUMPACK_MPI_DIST_RANK' in s]
t_EIGEN_LDLT = [ s for s in str_out_sm["mpi%domp%d_%s"%(m,t,u)] if 'EIGEN_LDLT' in s]
t_EIGEN_BICGSTAB = [ s for s in str_out_sm["mpi%domp%d_%s"%(m,t,u)] if 'EIGEN_BICGSTAB' in s]
if m != 1:
same_t.strum_mpi_ptscotch[m_idx,t_idx] = t_STRUMPACK_MPI_DIST_RANK[-1].rsplit(',')[-2].strip()[0:-1]
same_t.strum_scotch_a[m_idx,t_idx] = t_STRUMPACK_OMP[-1].rsplit(',')[-2].strip()[0:-1]
same_t.eig_bicgstab[m_idx,t_idx] = t_EIGEN_BICGSTAB[-1].rsplit(',')[-2].strip()[0:-1]
same_t.eig_ldlt[m_idx,t_idx] = t_EIGEN_LDLT[-1].rsplit(',')[-2].strip()[0:-1]
u_dat.update({u:same_t})
str_E_LDLT = "EIG_LDLT_%s"%u
str_E_BICGSTAB = "EIG_BICGSTAB_%s"%u
str_S_SA = "STR_SA_%s"%u
str_S_MPI = "STR_MPI_%s"%u
df_dict = { str_E_LDLT:same_t.eig_ldlt,
str_E_BICGSTAB:same_t.eig_bicgstab,
str_S_SA:same_t.strum_scotch_a,
str_S_MPI:same_t.strum_mpi_ptscotch
}
df_dict
With the data correctly parsed, we can plot the performance of each solver with $t(\textrm{MPI}, \textrm{OMP})$. Not, that the MPI solver does not operate with a single thread, and so will not have data in this row. Subsequently, the relative performance of the OpenMP STRUMPACK backend to the MPI backend is shown. While it would be best compared for the same upp er limit of threads*processes, it can be sufficient to see scalability across the board to identify MPI-enabled sweet-spots. Additionally, as no MPI benefit exists for the other solvers, the launching of processes and overheads can slow performance overal. Thus, it is best to examine the single process data for the non MPI-enabled data.
STRUMPACK MPI
# define the figure size and grid layout properties
figsize = (15, 12)
cols = 4
gs = gridspec.GridSpec(cols, cols)
gs.update(hspace=0.6)
fig1 = plt.figure(num=1, figsize=figsize)
fig1.suptitle('1k strum_mpi_ptscotch', size=16)
ax = []
for i, u in enumerate(uniq_ref):
row = (i // cols)
col = i % cols
ax.append(fig1.add_subplot(gs[row, col]))
cset = ax[-1].imshow(u_dat[u].strum_mpi_ptscotch)
ax[-1].set_xlabel('OMP_NUM_THREADS')
ax[-1].set_ylabel('MPI PROCS')
plt.colorbar(cset)
ax[-1].set_xticks([l for l in xrange(0,5)])
ax[-1].set_xticklabels([str(2**l) for l in xrange(0,5)])
ax[-1].set_yticks([l for l in xrange(0,5)])
ax[-1].set_yticklabels([str(2**l) for l in xrange(0,5)])
 |
STRUMPACK OpenMP
# define the figure size and grid layout properties
figsize = (15, 12)
cols = 4
gs = gridspec.GridSpec(cols, cols)
gs.update(hspace=0.6)
fig1 = plt.figure(num=1, figsize=figsize)
fig1.suptitle('1k strum_scotch_a', size=16)
ax = []
cbars= []
for i, u in enumerate(uniq_ref):
data = u_dat[u].strum_scotch_a
row = (i // cols)
col = i % cols
ax.append(fig1.add_subplot(gs[row, col]))
cset = ax[-1].imshow(data)
ax[-1].set_title(u)
ax[-1].set_xlabel('OMP_NUM_THREADS')
ax[-1].set_ylabel('MPI PROCS')
plt.colorbar(cset)
ax[-1].set_xticks([l for l in xrange(0,5)])
ax[-1].set_xticklabels([str(2**l) for l in xrange(0,5)])
ax[-1].set_yticks([l for l in xrange(0,5)])
ax[-1].set_yticklabels([str(2**l) for l in xrange(0,5)])
plt.show()
 |
EIGEN LDLT
# define the figure size and grid layout properties
figsize = (15, 12)
cols = 4
gs = gridspec.GridSpec(cols, cols)
gs.update(hspace=0.6)
fig1 = plt.figure(num=1, figsize=figsize)
fig1.suptitle('1k eig_ldlt', size=16)
ax = []
cbars= []
for i, u in enumerate(uniq_ref):
data = u_dat[u].eig_ldlt
row = (i // cols)
col = i % cols
ax.append(fig1.add_subplot(gs[row, col]))
cset = ax[-1].imshow(data)
ax[-1].set_title(u)
ax[-1].set_xlabel('OMP_NUM_THREADS')
ax[-1].set_ylabel('MPI PROCS')
plt.colorbar(cset)
ax[-1].set_xticks([l for l in xrange(0,5)])
ax[-1].set_xticklabels([str(2**l) for l in xrange(0,5)])
ax[-1].set_yticks([l for l in xrange(0,5)])
ax[-1].set_yticklabels([str(2**l) for l in xrange(0,5)])
 |
EIGEN BICGSTAB
# define the figure size and grid layout properties
figsize = (15, 12)
cols = 4
gs = gridspec.GridSpec(cols, cols)
gs.update(hspace=0.6)
fig1 = plt.figure(num=1, figsize=figsize)
fig1.suptitle('1k eig_bicgstab', size=16)
ax = []
cbars= []
for i, u in enumerate(uniq_ref):
data = u_dat[u].eig_bicgstab
row = (i // cols)
col = i % cols
ax.append(fig1.add_subplot(gs[row, col]))
cset = ax[-1].imshow(data)
ax[-1].set_title(u)
ax[-1].set_xlabel('OMP_NUM_THREADS')
ax[-1].set_ylabel('MPI PROCS')
plt.colorbar(cset)
ax[-1].set_xticks([l for l in xrange(0,5)])
ax[-1].set_xticklabels([str(2**l) for l in xrange(0,5)])
ax[-1].set_yticks([l for l in xrange(0,5)])
ax[-1].set_yticklabels([str(2**l) for l in xrange(0,5)])
 |
# define the figure size and grid layout properties
figsize = (15, 12)
cols = 4
gs = gridspec.GridSpec(cols, cols)
gs.update(hspace=0.6)
fig1 = plt.figure(num=1, figsize=figsize)
fig1.suptitle('1k strum_mpi_ptscotch / strum_scotch_a', size=16)
ax = []
cbars= []
for i, u in enumerate(uniq_ref):
data = np.divide(u_dat[u].strum_mpi_ptscotch , u_dat[u].strum_scotch_a)
row = (i // cols)
col = i % cols
ax.append(fig1.add_subplot(gs[row, col]))
cset = ax[-1].imshow(data,cmap='bwr')
ax[-1].set_title(u)
ax[-1].set_xlabel('OMP_NUM_THREADS')
ax[-1].set_ylabel('MPI PROCS')
plt.colorbar(cset)
ax[-1].set_xticks([l for l in xrange(0,5)])
ax[-1].set_xticklabels([str(2**l) for l in xrange(0,5)])
ax[-1].set_yticks([l for l in xrange(0,5)])
ax[-1].set_yticklabels([str(2**l) for l in xrange(0,5)])
 |
