Levenberg-Marquardt sparse solver scaling: 5K data set

STRUMPACK vs EIGEN performance

The goal of this notebook is to allow the documentation of STRUMPACK vs EIGEN performance to be maintained in a single accessible location. The environment within which this notebook is run follows the standard cctbx conda build instructions available here. For this instance, we are using the STRUMPACK-enabled build of cctbx located at ExaFEL:cctbx_project(str_merge). STRUMPACK is currently built using the installation script STRUMPACK_installer_shared.sh, and if the installation takes place within the same directory as moddules and build, the cctbx build process can make use of it as a backend. After the STRUMPACK solver finishes, enter the build directory, run libtbx.refresh and make. The STRUMPACK-supported modules should now build and link with the new backend.

The solution of the below systems assume that we have some A and b data already, and we can provide these in CSV file format to the solver (A is in “row col value” format). We can begin by importing all of the required modules. We make use of Numpy’s ability to parse CSV files into numeric format, and SciPy’s sparse storage format to format the matrices into the required CSR sparse storage format for solving.

%matplotlib inline
%config InlineBackend.figure_format = 'retina'
import matplotlib
import matplotlib.pyplot as plt
font = {'family' : 'serif',
        #'weight' : 'bold',
        'size'   : 12}

matplotlib.rc('font', **font)
matplotlib.rcParams['figure.figsize'] = (16,10)
matplotlib.rcParams['figure.dpi']= 150
import matplotlib.gridspec as gridspec
from __future__ import division
from cctbx.array_family import flex
from libtbx.test_utils import approx_equal
from libtbx.development.timers import Profiler
import sys
import numpy as np
import scipy.sparse as sps

%env BOOST_ADAPTBX_FPE_DEFAULT=1
%env BOOST_ADAPTBX_SIGNALS_DEFAULT=1
env: BOOST_ADAPTBX_FPE_DEFAULT=1
env: BOOST_ADAPTBX_SIGNALS_DEFAULT=1

We load the A and b data set for solving from the locations specified below. Ideally, we can loop over and load all different data sets and process each individually. For a working example, we begin by processing a single data set, comparing Eigen and Strumpack for varying levels of shared parallelism.

A_path="/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv"
A_mat = np.loadtxt(A_path,dtype={'names':('rows','cols','vals'),'formats':('i8','i8','f8')})

b_path="/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv"
b_vec = np.loadtxt(b_path)

We are solving system of normal equations, and so we can safely assume that we have a square matrix. This ensures the number of rows and colums will be identical, which can be read from the length of the numpy array b_vec.

n_rows = len(b_vec)
n_cols = n_rows

We next wish to create a SciPy CSR sparse matrix type using the data from A_mat.

A_sp = sps.csr_matrix((A_mat['vals'],(A_mat['rows'],A_mat['cols'])))

With this, we can now create the appropriate flex data types for solving the linear system. While the operations performed upto and including this point involve many copy steps, it can be stated that the time to solve the system is much greater than the time to perform internal copies, and so we can ignore the times required for these steps for any realistic datasets.

For the solvers we require the row index pointers, the column indices, and sparse matrix values. However, it is import to first ensure that our A matrix is symmetric, and not the upper or lower triangle only. While Eigen’s Cholesky deocmposition can use the upper or lower triangle, other algorithms and solvers tend to expect a full matrix. We can create this as follows.

tu=sps.triu(A_sp)
tl=sps.tril(A_sp)
sd=sps.diags(A_sp.diagonal())
if tu.nnz == sd.getnnz() or tl.nnz == sd.getnnz():
    A_sp = A_sp + A_sp.transpose() - sd
plt.spy(A_sp, marker='.',markersize=1.0)
plt.show()
png

The resulting data sets can be read directly and converted to flex types as follows.

A_indptr = flex.int(A_sp.indptr)
A_indices = flex.int(A_sp.indices)
A_values = flex.double(A_sp.data)
b = flex.double(b_vec)

With the data in the required format, we can now create a solver object and solve the system. We also include a built-in profiler object to time this solver step for both the STRUMPACK and EIGEN backends.

Next, we load the OpenMP-enabled STRUMPACK solver module and the Eigen based solver. These are contained within “modules/cctbx_project/scitbx/examples/bevington/strumpack_solver_ext.cpp” and where a Boost.Python wrapper for both an EIGEN and STRUMPACK Ax=b solver object is provided.

A non-notebook script exists to test this functionality at “modules/cctbx_project/scitbx/examples/bevington/strumpack_eigen_solver.py”, which can be called as:

OMP_NUM_THREADS=x libtbx.python strumpack_eigen_solver.py A_mat.csv b_vec.csv

where A_mat.csv and b_vec.csv are the CSV files with the linear system to be solved, and x specifies the number of OpenMP threads to spawn. Additionally, the MPI-enabled STRUMPACK backend is provided by “modules/cctbx_project/scitbx/examples/bevington/strumpack_solver_ext_mpi_dist.cpp”, with the Python script to test the result located at “modules/cctbx_project/scitbx/examples/bevington/strumpack_eigen_solver_mpi_dist.py”. This is called as

OMP_NUM_THREADS=x mpirun -n y libtbx.python strumpack_eigen_solver.py A_mat.csv b_vec.csv

As an example, the previously loaded can be provided to the solvers as follows:

import boost.python
ext_omp = boost.python.import_ext("scitbx_examples_strumpack_solver_ext")
ext_mpi = boost.python.import_ext("scitbx_examples_strumpack_mpi_dist_solver_ext")

es   = ext_omp.eigen_solver
ss   = ext_omp.strumpack_solver
ssmd = ext_mpi.strumpack_mpi_dist_solver

def run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b):
    P = Profiler("STRUMPACK_SCOTCH")
    res_strum_sc = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.scotch, ext_omp.auto)
    del P
    P = Profiler("STRUMPACK_METIS")
    res_strum_me = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.metis, ext_omp.auto)
    del P
    
    P = Profiler("EIGEN_LDLT")
    res_eig_ldlt = es(1, n_rows, n_cols, A_indptr, A_indices, A_values, b)
    del P
    P = Profiler("EIGEN_BICGSTAB")
    res_eig_bicgstab = es(2, n_rows, n_cols, A_indptr, A_indices, A_values, b)
    del P
    return res_strum_sc, res_strum_me, res_eig_ldlt, res_eig_bicgstab
    
r_s_sc, r_s_me, r_e_ldlt, r_e_bicgstab = run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b)
for i in xrange(len(r_s_sc.x)):
    assert( approx_equal(r_s_sc.x[i], r_s_me.x[i]) )
    assert( approx_equal(r_s_sc.x[i], r_e_ldlt.x[i]) )
    assert( approx_equal(r_s_sc.x[i], r_e_bicgstab.x[i]) )
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.
  try: mod = __import__(name)
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.
  try: mod = __import__(name)


individual call time for STRUMPACK_SCOTCH: CPU,   19.210s; elapsed,    0.424s
individual call time for STRUMPACK_METIS: CPU,  176.190s; elapsed,    2.805s
individual call time for EIGEN_LDLT: CPU,    3.940s; elapsed,    0.063s
individual call time for EIGEN_BICGSTAB: CPU,    1.400s; elapsed,    0.023s

While we can use the above method to process the data, we must respecify the OMP_NUM_THREADS variable to observe different levels of scalability. %env OMP_NUM_THREADS is sufficient to test this for the first run, however for an undetermined reason the notebook must be started to allow the value to be changed. There, we can run the above tasks from a !<command> cell, specifying the thread count here instead.

OMP_SOLVER='''
from __future__ import division
from cctbx.array_family import flex
from libtbx.test_utils import approx_equal
from libtbx.development.timers import Profiler
import sys
import numpy as np
import scipy.sparse as sps
import matplotlib
import matplotlib.pyplot as plt
import matplotlib
import matplotlib.pyplot as plt
matplotlib.rcParams['figure.figsize'] = (16,10)

A_path=sys.argv[1]
A_mat = np.loadtxt(A_path,dtype={'names':('rows','cols','vals'),'formats':('i8','i8','f8')})

b_path=sys.argv[2] 
b_vec = np.loadtxt(b_path)

n_rows = len(b_vec)
n_cols = n_rows

A_sp = sps.csr_matrix((A_mat['vals'],(A_mat['rows'],A_mat['cols'])))

#Check for triangular matrix. If so, A_sp := A+A^T - diag(A)

tu=sps.triu(A_sp)
tl=sps.tril(A_sp)
sd=sps.diags(A_sp.diagonal())

A_spS = A_sp
if tu.nnz == sd.getnnz() or tl.nnz == sd.getnnz():
    A_spS = A_sp + A_sp.transpose() - sd

A_indptr = flex.int(A_sp.indptr)
A_indices = flex.int(A_sp.indices)
A_values = flex.double(A_sp.data)
b = flex.double(b_vec)

#import time
#timing_dict = {"strum":0, "eigen":0}

import boost.python
ext_omp = boost.python.import_ext("scitbx_examples_strumpack_solver_ext")
ext_mpi = boost.python.import_ext("scitbx_examples_strumpack_mpi_dist_solver_ext")

es   = ext_omp.eigen_solver
ss   = ext_omp.strumpack_solver

def run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS):

    P = Profiler("EIGEN_LLT_CHOL")
    res_eig_llt_chol = es(0, n_rows, n_cols, A_indptr, A_indices, A_values, b)
    del P
    
    P = Profiler("EIGEN_LDLT_CHOL")
    res_eig_ldlt_chol = es(1, n_rows, n_cols, A_indptr, A_indices, A_values, b)
    del P
    
    A_indptr = flex.int(A_spS.indptr)
    A_indices = flex.int(A_spS.indices)
    A_values = flex.double(A_spS.data)
    
    P = Profiler("EIGEN_BICGSTAB")
    res_eig_bicgstab = es(2, n_rows, n_cols, A_indptr, A_indices, A_values, b)
    del P
    
    P = Profiler("STRUMPACK_SCOTCH_AUTO")
    res_strum_sc_a = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.scotch, ext_omp.auto)
    del P
    
    P = Profiler("STRUMPACK_METIS_AUTO")
    res_strum_mt_a = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.metis, ext_omp.auto)
    del P

    P = Profiler("STRUMPACK_SCOTCH_BICGSTAB")
    res_strum_sc_bi = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.scotch, ext_omp.bicgstab)
    del P
    
    P = Profiler("STRUMPACK_METIS_BICGSTAB")
    res_strum_mt_bi = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.metis, ext_omp.bicgstab)
    del P
    
    P = Profiler("STRUMPACK_SCOTCH_PRECBICGSTAB")
    res_strum_sc_pr = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.scotch, ext_omp.prec_bicgstab)
    del P
    
    P = Profiler("STRUMPACK_METIS_PRECBICGSTAB")
    res_strum_mt_pr = ss(n_rows, n_cols, A_indptr, A_indices, A_values, b, ext_omp.metis, ext_omp.prec_bicgstab)
    del P
    
    for i in xrange(len(res_strum_sc.x)):
        assert( approx_equal(res_eig_llt_chol.x[i], res_eig_ldlt_chol.x[i]) )
        assert( approx_equal(res_strum_sc_a.x[i],     res_eig_bicgstab.x[i])  )
        assert( approx_equal(res_strum_sc_a.x[i],     res_eig_ldlt_chol.x[i]) )
        assert( approx_equal(res_strum_mt_a.x[i],     res_strum_sc_a.x[i])      )
        assert( approx_equal(res_strum_mt_a.x[i],     res_strum_sc_bi.x[i])      )
        assert( approx_equal(res_strum_mt_a.x[i],     res_strum_mt_bi.x[i])      )
        assert( approx_equal(res_strum_mt_a.x[i],     res_strum_sc_pr.x[i])      )
        assert( approx_equal(res_strum_mt_a.x[i],     res_strum_mt_pr.x[i])      )

run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)
'''
OMP_SOLVER = OMP_SOLVER
OMP_SOLVER_FILE = open("OMP_SOLVER.py", "w")
OMP_SOLVER_FILE.write(OMP_SOLVER)
OMP_SOLVER_FILE.close()

DATAPATH="/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/"
A_LIST = !find {DATAPATH} -iname "A*.csv"
B_LIST = [ii.replace('/A_','/b_') for ii in A_LIST]

We now record the indices of the data with a specific number of images. We can use these indices to later submit jobs with a given number of frames, and hence resulting matrix size.

list_idx={}
for imgs in ['1k','5k','10k','32k']:
    list_idx.update({imgs:[i for i, j in enumerate(A_LIST) if imgs in j]})
list_idx
{'10k': [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14],
 '1k': [15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29],
 '32k': [30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44],
 '5k': [45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59]}

With the file writtent to disk, we now create a loop in mixed Python/Bash to call the solver, then extract the standard output from the profiler to examine the OpenMP scalability of STRUMPACK. The profiler code wraps each solver, and does not take into account the load-times for the data.

str_out={}
import os
threads_list = [16]
for imgs_size in list_idx:
    print "Data Set Size:=%s"%imgs_size
    #Subselect the smallest data size for now
    if imgs_size != "1k":
        print "Skipping %s"%imgs_size
        continue
        
    for imgs_idx in list_idx[imgs_size]:
        
        A_path = A_LIST[imgs_idx]; b_path = B_LIST[imgs_idx]
        dat_name = A_path.split('/')[-1][2:-4]

        print "Data Set Name:=%s"%(dat_name)

        #Ensure the A and b data are matched correctly
        assert(os.path.dirname(A_path) == os.path.dirname(b_path))
        print {A_path}, {b_path}
        
        for threads in threads_list:
            print "OMP_NUM_THREADS:=%d"%threads

            val = !OMP_NUM_THREADS={threads} libtbx.python ./OMP_SOLVER.py {A_path} {b_path}
            print val
            for s in val:
                if "assert" in s:
                    raise Exception("Solutions not equal! Halting")
            key = 'omp' + str(threads) + '_' + dat_name
            str_out.update({key:val})

—CLICK FOR OUTPUT—

Data Set Size:=10k
Skipping 10k
Data Set Size:=1k
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU,    0.810s; elapsed,    0.815s', 'individual call time for EIGEN_LDLT_CHOL: CPU,    0.870s; elapsed,    0.863s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.250s; elapsed,    0.025s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU,    4.280s; elapsed,    0.271s', 'individual call time for STRUMPACK_METIS_AUTO: CPU,   31.080s; elapsed,    1.960s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU,    1.930s; elapsed,    0.121s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU,    1.880s; elapsed,    0.117s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    2.920s; elapsed,    0.182s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU,   33.550s; elapsed,    2.101s', 'Traceback (most recent call last):', '  File "./OMP_SOLVER.py", line 103, in <module>', '    run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', '  File "./OMP_SOLVER.py", line 93, in run_solver', '    for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for   STRUMPACK_METIS_PRECBICGSTAB: CPU,   33.550s; elapsed,    2.101s, #calls:   1', 'time for                 EIGEN_LLT_CHOL: CPU,    0.810s; elapsed,    0.815s, #calls:   1', 'time for      STRUMPACK_SCOTCH_BICGSTAB: CPU,    1.930s; elapsed,    0.121s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.250s; elapsed,    0.025s, #calls:   1', 'time for          STRUMPACK_SCOTCH_AUTO: CPU,    4.280s; elapsed,    0.271s, #calls:   1', 'time for  STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    2.920s; elapsed,    0.182s, #calls:   1', 'time for       STRUMPACK_METIS_BICGSTAB: CPU,    1.880s; elapsed,    0.117s, #calls:   1', 'time for                EIGEN_LDLT_CHOL: CPU,    0.870s; elapsed,    0.863s, #calls:   1', 'time for           STRUMPACK_METIS_AUTO: CPU,   31.080s; elapsed,    1.960s, #calls:   1', 'TOTAL                                  : CPU,   77.570s; elapsed,    6.455s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU,    0.220s; elapsed,    0.232s', 'individual call time for EIGEN_LDLT_CHOL: CPU,    0.240s; elapsed,    0.238s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.570s; elapsed,    0.049s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU,    7.060s; elapsed,    0.447s', 'individual call time for STRUMPACK_METIS_AUTO: CPU,   28.970s; elapsed,    1.831s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU,    2.720s; elapsed,    0.169s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU,    2.830s; elapsed,    0.177s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    6.750s; elapsed,    0.422s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU,   31.770s; elapsed,    2.001s', 'Traceback (most recent call last):', '  File "./OMP_SOLVER.py", line 103, in <module>', '    run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', '  File "./OMP_SOLVER.py", line 93, in run_solver', '    for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for   STRUMPACK_METIS_PRECBICGSTAB: CPU,   31.770s; elapsed,    2.001s, #calls:   1', 'time for                 EIGEN_LLT_CHOL: CPU,    0.220s; elapsed,    0.232s, #calls:   1', 'time for      STRUMPACK_SCOTCH_BICGSTAB: CPU,    2.720s; elapsed,    0.169s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.570s; elapsed,    0.049s, #calls:   1', 'time for          STRUMPACK_SCOTCH_AUTO: CPU,    7.060s; elapsed,    0.447s, #calls:   1', 'time for  STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    6.750s; elapsed,    0.422s, #calls:   1', 'time for       STRUMPACK_METIS_BICGSTAB: CPU,    2.830s; elapsed,    0.177s, #calls:   1', 'time for                EIGEN_LDLT_CHOL: CPU,    0.240s; elapsed,    0.238s, #calls:   1', 'time for           STRUMPACK_METIS_AUTO: CPU,   28.970s; elapsed,    1.831s, #calls:   1', 'TOTAL                                  : CPU,   81.130s; elapsed,    5.566s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Deff-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU,    0.240s; elapsed,    0.237s', 'individual call time for EIGEN_LDLT_CHOL: CPU,    0.230s; elapsed,    0.232s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.240s; elapsed,    0.024s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU,    3.400s; elapsed,    0.215s', 'individual call time for STRUMPACK_METIS_AUTO: CPU,   28.710s; elapsed,    1.812s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU,    1.750s; elapsed,    0.109s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU,    1.740s; elapsed,    0.109s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    3.070s; elapsed,    0.192s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU,   30.190s; elapsed,    1.893s', 'Traceback (most recent call last):', '  File "./OMP_SOLVER.py", line 103, in <module>', '    run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', '  File "./OMP_SOLVER.py", line 93, in run_solver', '    for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for   STRUMPACK_METIS_PRECBICGSTAB: CPU,   30.190s; elapsed,    1.893s, #calls:   1', 'time for                 EIGEN_LLT_CHOL: CPU,    0.240s; elapsed,    0.237s, #calls:   1', 'time for      STRUMPACK_SCOTCH_BICGSTAB: CPU,    1.750s; elapsed,    0.109s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.240s; elapsed,    0.024s, #calls:   1', 'time for          STRUMPACK_SCOTCH_AUTO: CPU,    3.400s; elapsed,    0.215s, #calls:   1', 'time for  STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    3.070s; elapsed,    0.192s, #calls:   1', 'time for       STRUMPACK_METIS_BICGSTAB: CPU,    1.740s; elapsed,    0.109s, #calls:   1', 'time for                EIGEN_LDLT_CHOL: CPU,    0.230s; elapsed,    0.232s, #calls:   1', 'time for           STRUMPACK_METIS_AUTO: CPU,   28.710s; elapsed,    1.812s, #calls:   1', 'TOTAL                                  : CPU,   69.570s; elapsed,    4.823s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU,    0.070s; elapsed,    0.064s', 'individual call time for EIGEN_LDLT_CHOL: CPU,    0.050s; elapsed,    0.052s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.230s; elapsed,    0.025s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU,    3.350s; elapsed,    0.212s', 'individual call time for STRUMPACK_METIS_AUTO: CPU,   28.510s; elapsed,    1.799s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU,    1.670s; elapsed,    0.105s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU,    1.760s; elapsed,    0.110s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    3.310s; elapsed,    0.207s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU,   26.660s; elapsed,    1.673s', 'Traceback (most recent call last):', '  File "./OMP_SOLVER.py", line 103, in <module>', '    run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', '  File "./OMP_SOLVER.py", line 93, in run_solver', '    for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for   STRUMPACK_METIS_PRECBICGSTAB: CPU,   26.660s; elapsed,    1.673s, #calls:   1', 'time for                 EIGEN_LLT_CHOL: CPU,    0.070s; elapsed,    0.064s, #calls:   1', 'time for      STRUMPACK_SCOTCH_BICGSTAB: CPU,    1.670s; elapsed,    0.105s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.230s; elapsed,    0.025s, #calls:   1', 'time for          STRUMPACK_SCOTCH_AUTO: CPU,    3.350s; elapsed,    0.212s, #calls:   1', 'time for  STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    3.310s; elapsed,    0.207s, #calls:   1', 'time for       STRUMPACK_METIS_BICGSTAB: CPU,    1.760s; elapsed,    0.110s, #calls:   1', 'time for                EIGEN_LDLT_CHOL: CPU,    0.050s; elapsed,    0.052s, #calls:   1', 'time for           STRUMPACK_METIS_AUTO: CPU,   28.510s; elapsed,    1.799s, #calls:   1', 'TOTAL                                  : CPU,   65.610s; elapsed,    4.247s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU,    1.990s; elapsed,    1.995s', 'individual call time for EIGEN_LDLT_CHOL: CPU,    2.110s; elapsed,    2.120s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.520s; elapsed,    0.048s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU,    6.590s; elapsed,    0.417s', 'individual call time for STRUMPACK_METIS_AUTO: CPU,   34.870s; elapsed,    2.191s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU,    2.440s; elapsed,    0.153s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU,    2.620s; elapsed,    0.163s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    6.920s; elapsed,    0.433s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU,   38.160s; elapsed,    2.391s', 'Traceback (most recent call last):', '  File "./OMP_SOLVER.py", line 103, in <module>', '    run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', '  File "./OMP_SOLVER.py", line 93, in run_solver', '    for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for   STRUMPACK_METIS_PRECBICGSTAB: CPU,   38.160s; elapsed,    2.391s, #calls:   1', 'time for                 EIGEN_LLT_CHOL: CPU,    1.990s; elapsed,    1.995s, #calls:   1', 'time for      STRUMPACK_SCOTCH_BICGSTAB: CPU,    2.440s; elapsed,    0.153s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.520s; elapsed,    0.048s, #calls:   1', 'time for          STRUMPACK_SCOTCH_AUTO: CPU,    6.590s; elapsed,    0.417s, #calls:   1', 'time for  STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    6.920s; elapsed,    0.433s, #calls:   1', 'time for       STRUMPACK_METIS_BICGSTAB: CPU,    2.620s; elapsed,    0.163s, #calls:   1', 'time for                EIGEN_LDLT_CHOL: CPU,    2.110s; elapsed,    2.120s, #calls:   1', 'time for           STRUMPACK_METIS_AUTO: CPU,   34.870s; elapsed,    2.191s, #calls:   1', 'TOTAL                                  : CPU,   96.220s; elapsed,    9.909s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU,    2.040s; elapsed,    2.044s', 'individual call time for EIGEN_LDLT_CHOL: CPU,    2.090s; elapsed,    2.091s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.250s; elapsed,    0.024s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU,    3.470s; elapsed,    0.221s', 'individual call time for STRUMPACK_METIS_AUTO: CPU,   35.250s; elapsed,    2.219s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU,    1.910s; elapsed,    0.118s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU,    1.840s; elapsed,    0.115s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    2.990s; elapsed,    0.188s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU,   38.560s; elapsed,    2.412s', 'Traceback (most recent call last):', '  File "./OMP_SOLVER.py", line 103, in <module>', '    run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', '  File "./OMP_SOLVER.py", line 93, in run_solver', '    for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for   STRUMPACK_METIS_PRECBICGSTAB: CPU,   38.560s; elapsed,    2.412s, #calls:   1', 'time for                 EIGEN_LLT_CHOL: CPU,    2.040s; elapsed,    2.044s, #calls:   1', 'time for      STRUMPACK_SCOTCH_BICGSTAB: CPU,    1.910s; elapsed,    0.118s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.250s; elapsed,    0.024s, #calls:   1', 'time for          STRUMPACK_SCOTCH_AUTO: CPU,    3.470s; elapsed,    0.221s, #calls:   1', 'time for  STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    2.990s; elapsed,    0.188s, #calls:   1', 'time for       STRUMPACK_METIS_BICGSTAB: CPU,    1.840s; elapsed,    0.115s, #calls:   1', 'time for                EIGEN_LDLT_CHOL: CPU,    2.090s; elapsed,    2.091s, #calls:   1', 'time for           STRUMPACK_METIS_AUTO: CPU,   35.250s; elapsed,    2.219s, #calls:   1', 'TOTAL                                  : CPU,   88.400s; elapsed,    9.434s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU,    0.830s; elapsed,    0.819s', 'individual call time for EIGEN_LDLT_CHOL: CPU,    0.860s; elapsed,    0.858s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.300s; elapsed,    0.030s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU,    3.520s; elapsed,    0.223s', 'individual call time for STRUMPACK_METIS_AUTO: CPU,   30.270s; elapsed,    1.907s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU,    2.000s; elapsed,    0.125s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU,    2.070s; elapsed,    0.130s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    3.260s; elapsed,    0.204s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU,   30.920s; elapsed,    1.944s', 'Traceback (most recent call last):', '  File "./OMP_SOLVER.py", line 103, in <module>', '    run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', '  File "./OMP_SOLVER.py", line 93, in run_solver', '    for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for   STRUMPACK_METIS_PRECBICGSTAB: CPU,   30.920s; elapsed,    1.944s, #calls:   1', 'time for                 EIGEN_LLT_CHOL: CPU,    0.830s; elapsed,    0.819s, #calls:   1', 'time for      STRUMPACK_SCOTCH_BICGSTAB: CPU,    2.000s; elapsed,    0.125s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.300s; elapsed,    0.030s, #calls:   1', 'time for          STRUMPACK_SCOTCH_AUTO: CPU,    3.520s; elapsed,    0.223s, #calls:   1', 'time for  STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    3.260s; elapsed,    0.204s, #calls:   1', 'time for       STRUMPACK_METIS_BICGSTAB: CPU,    2.070s; elapsed,    0.130s, #calls:   1', 'time for                EIGEN_LDLT_CHOL: CPU,    0.860s; elapsed,    0.858s, #calls:   1', 'time for           STRUMPACK_METIS_AUTO: CPU,   30.270s; elapsed,    1.907s, #calls:   1', 'TOTAL                                  : CPU,   74.030s; elapsed,    6.238s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU,    0.850s; elapsed,    0.843s', 'individual call time for EIGEN_LDLT_CHOL: CPU,    0.860s; elapsed,    0.864s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.530s; elapsed,    0.050s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU,    7.220s; elapsed,    0.459s', 'individual call time for STRUMPACK_METIS_AUTO: CPU,   31.850s; elapsed,    2.009s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU,    2.370s; elapsed,    0.148s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU,    2.580s; elapsed,    0.162s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    6.480s; elapsed,    0.406s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU,   34.160s; elapsed,    2.146s', 'Traceback (most recent call last):', '  File "./OMP_SOLVER.py", line 103, in <module>', '    run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', '  File "./OMP_SOLVER.py", line 93, in run_solver', '    for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for   STRUMPACK_METIS_PRECBICGSTAB: CPU,   34.160s; elapsed,    2.146s, #calls:   1', 'time for                 EIGEN_LLT_CHOL: CPU,    0.850s; elapsed,    0.843s, #calls:   1', 'time for      STRUMPACK_SCOTCH_BICGSTAB: CPU,    2.370s; elapsed,    0.148s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.530s; elapsed,    0.050s, #calls:   1', 'time for          STRUMPACK_SCOTCH_AUTO: CPU,    7.220s; elapsed,    0.459s, #calls:   1', 'time for  STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    6.480s; elapsed,    0.406s, #calls:   1', 'time for       STRUMPACK_METIS_BICGSTAB: CPU,    2.580s; elapsed,    0.162s, #calls:   1', 'time for                EIGEN_LDLT_CHOL: CPU,    0.860s; elapsed,    0.864s, #calls:   1', 'time for           STRUMPACK_METIS_AUTO: CPU,   31.850s; elapsed,    2.009s, #calls:   1', 'TOTAL                                  : CPU,   86.900s; elapsed,    7.087s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU,    0.220s; elapsed,    0.231s', 'individual call time for EIGEN_LDLT_CHOL: CPU,    0.240s; elapsed,    0.236s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.530s; elapsed,    0.046s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU,    7.200s; elapsed,    0.455s', 'individual call time for STRUMPACK_METIS_AUTO: CPU,   28.890s; elapsed,    1.823s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU,    2.610s; elapsed,    0.163s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU,    2.930s; elapsed,    0.183s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    7.140s; elapsed,    0.447s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU,   30.950s; elapsed,    1.941s', 'Traceback (most recent call last):', '  File "./OMP_SOLVER.py", line 103, in <module>', '    run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', '  File "./OMP_SOLVER.py", line 93, in run_solver', '    for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for   STRUMPACK_METIS_PRECBICGSTAB: CPU,   30.950s; elapsed,    1.941s, #calls:   1', 'time for                 EIGEN_LLT_CHOL: CPU,    0.220s; elapsed,    0.231s, #calls:   1', 'time for      STRUMPACK_SCOTCH_BICGSTAB: CPU,    2.610s; elapsed,    0.163s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.530s; elapsed,    0.046s, #calls:   1', 'time for          STRUMPACK_SCOTCH_AUTO: CPU,    7.200s; elapsed,    0.455s, #calls:   1', 'time for  STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    7.140s; elapsed,    0.447s, #calls:   1', 'time for       STRUMPACK_METIS_BICGSTAB: CPU,    2.930s; elapsed,    0.183s, #calls:   1', 'time for                EIGEN_LDLT_CHOL: CPU,    0.240s; elapsed,    0.236s, #calls:   1', 'time for           STRUMPACK_METIS_AUTO: CPU,   28.890s; elapsed,    1.823s, #calls:   1', 'TOTAL                                  : CPU,   80.710s; elapsed,    5.525s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU,    0.240s; elapsed,    0.238s', 'individual call time for EIGEN_LDLT_CHOL: CPU,    0.240s; elapsed,    0.232s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.210s; elapsed,    0.023s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU,    3.330s; elapsed,    0.212s', 'individual call time for STRUMPACK_METIS_AUTO: CPU,   29.220s; elapsed,    1.835s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU,    1.750s; elapsed,    0.109s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU,    1.750s; elapsed,    0.109s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    3.090s; elapsed,    0.194s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU,   28.990s; elapsed,    1.830s', 'Traceback (most recent call last):', '  File "./OMP_SOLVER.py", line 103, in <module>', '    run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', '  File "./OMP_SOLVER.py", line 93, in run_solver', '    for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for   STRUMPACK_METIS_PRECBICGSTAB: CPU,   28.990s; elapsed,    1.830s, #calls:   1', 'time for                 EIGEN_LLT_CHOL: CPU,    0.240s; elapsed,    0.238s, #calls:   1', 'time for      STRUMPACK_SCOTCH_BICGSTAB: CPU,    1.750s; elapsed,    0.109s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.210s; elapsed,    0.023s, #calls:   1', 'time for          STRUMPACK_SCOTCH_AUTO: CPU,    3.330s; elapsed,    0.212s, #calls:   1', 'time for  STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    3.090s; elapsed,    0.194s, #calls:   1', 'time for       STRUMPACK_METIS_BICGSTAB: CPU,    1.750s; elapsed,    0.109s, #calls:   1', 'time for                EIGEN_LDLT_CHOL: CPU,    0.240s; elapsed,    0.232s, #calls:   1', 'time for           STRUMPACK_METIS_AUTO: CPU,   29.220s; elapsed,    1.835s, #calls:   1', 'TOTAL                                  : CPU,   68.820s; elapsed,    4.782s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU,    3.880s; elapsed,    3.883s', 'individual call time for EIGEN_LDLT_CHOL: CPU,    4.090s; elapsed,    4.098s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.550s; elapsed,    0.050s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU,    6.870s; elapsed,    0.435s', 'individual call time for STRUMPACK_METIS_AUTO: CPU,   37.710s; elapsed,    2.365s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU,    2.460s; elapsed,    0.154s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU,    2.680s; elapsed,    0.167s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    7.180s; elapsed,    0.450s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU,   40.580s; elapsed,    2.539s', 'Traceback (most recent call last):', '  File "./OMP_SOLVER.py", line 103, in <module>', '    run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', '  File "./OMP_SOLVER.py", line 93, in run_solver', '    for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for   STRUMPACK_METIS_PRECBICGSTAB: CPU,   40.580s; elapsed,    2.539s, #calls:   1', 'time for                 EIGEN_LLT_CHOL: CPU,    3.880s; elapsed,    3.883s, #calls:   1', 'time for      STRUMPACK_SCOTCH_BICGSTAB: CPU,    2.460s; elapsed,    0.154s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.550s; elapsed,    0.050s, #calls:   1', 'time for          STRUMPACK_SCOTCH_AUTO: CPU,    6.870s; elapsed,    0.435s, #calls:   1', 'time for  STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    7.180s; elapsed,    0.450s, #calls:   1', 'time for       STRUMPACK_METIS_BICGSTAB: CPU,    2.680s; elapsed,    0.167s, #calls:   1', 'time for                EIGEN_LDLT_CHOL: CPU,    4.090s; elapsed,    4.098s, #calls:   1', 'time for           STRUMPACK_METIS_AUTO: CPU,   37.710s; elapsed,    2.365s, #calls:   1', 'TOTAL                                  : CPU,  106.000s; elapsed,   14.140s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU,    2.020s; elapsed,    2.023s', 'individual call time for EIGEN_LDLT_CHOL: CPU,    2.110s; elapsed,    2.119s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.530s; elapsed,    0.048s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU,    6.330s; elapsed,    0.401s', 'individual call time for STRUMPACK_METIS_AUTO: CPU,   38.360s; elapsed,    2.411s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU,    2.420s; elapsed,    0.151s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU,    2.580s; elapsed,    0.161s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    6.920s; elapsed,    0.434s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU,   37.560s; elapsed,    2.354s', 'Traceback (most recent call last):', '  File "./OMP_SOLVER.py", line 103, in <module>', '    run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', '  File "./OMP_SOLVER.py", line 93, in run_solver', '    for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for   STRUMPACK_METIS_PRECBICGSTAB: CPU,   37.560s; elapsed,    2.354s, #calls:   1', 'time for                 EIGEN_LLT_CHOL: CPU,    2.020s; elapsed,    2.023s, #calls:   1', 'time for      STRUMPACK_SCOTCH_BICGSTAB: CPU,    2.420s; elapsed,    0.151s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.530s; elapsed,    0.048s, #calls:   1', 'time for          STRUMPACK_SCOTCH_AUTO: CPU,    6.330s; elapsed,    0.401s, #calls:   1', 'time for  STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    6.920s; elapsed,    0.434s, #calls:   1', 'time for       STRUMPACK_METIS_BICGSTAB: CPU,    2.580s; elapsed,    0.161s, #calls:   1', 'time for                EIGEN_LDLT_CHOL: CPU,    2.110s; elapsed,    2.119s, #calls:   1', 'time for           STRUMPACK_METIS_AUTO: CPU,   38.360s; elapsed,    2.411s, #calls:   1', 'TOTAL                                  : CPU,   98.830s; elapsed,   10.102s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU,    1.990s; elapsed,    1.994s', 'individual call time for EIGEN_LDLT_CHOL: CPU,    2.120s; elapsed,    2.119s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.240s; elapsed,    0.024s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU,    3.440s; elapsed,    0.220s', 'individual call time for STRUMPACK_METIS_AUTO: CPU,   34.260s; elapsed,    2.157s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU,    1.770s; elapsed,    0.111s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU,    1.670s; elapsed,    0.104s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    2.980s; elapsed,    0.187s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU,   37.390s; elapsed,    2.350s', 'Traceback (most recent call last):', '  File "./OMP_SOLVER.py", line 103, in <module>', '    run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', '  File "./OMP_SOLVER.py", line 93, in run_solver', '    for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for   STRUMPACK_METIS_PRECBICGSTAB: CPU,   37.390s; elapsed,    2.350s, #calls:   1', 'time for                 EIGEN_LLT_CHOL: CPU,    1.990s; elapsed,    1.994s, #calls:   1', 'time for      STRUMPACK_SCOTCH_BICGSTAB: CPU,    1.770s; elapsed,    0.111s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.240s; elapsed,    0.024s, #calls:   1', 'time for          STRUMPACK_SCOTCH_AUTO: CPU,    3.440s; elapsed,    0.220s, #calls:   1', 'time for  STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    2.980s; elapsed,    0.187s, #calls:   1', 'time for       STRUMPACK_METIS_BICGSTAB: CPU,    1.670s; elapsed,    0.104s, #calls:   1', 'time for                EIGEN_LDLT_CHOL: CPU,    2.120s; elapsed,    2.119s, #calls:   1', 'time for           STRUMPACK_METIS_AUTO: CPU,   34.260s; elapsed,    2.157s, #calls:   1', 'TOTAL                                  : CPU,   85.860s; elapsed,    9.266s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU,    0.830s; elapsed,    0.835s', 'individual call time for EIGEN_LDLT_CHOL: CPU,    0.870s; elapsed,    0.863s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.540s; elapsed,    0.051s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU,    7.370s; elapsed,    0.466s', 'individual call time for STRUMPACK_METIS_AUTO: CPU,   32.030s; elapsed,    2.025s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU,    2.370s; elapsed,    0.148s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU,    2.660s; elapsed,    0.166s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    6.810s; elapsed,    0.426s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU,   33.250s; elapsed,    2.094s', 'Traceback (most recent call last):', '  File "./OMP_SOLVER.py", line 103, in <module>', '    run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', '  File "./OMP_SOLVER.py", line 93, in run_solver', '    for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for   STRUMPACK_METIS_PRECBICGSTAB: CPU,   33.250s; elapsed,    2.094s, #calls:   1', 'time for                 EIGEN_LLT_CHOL: CPU,    0.830s; elapsed,    0.835s, #calls:   1', 'time for      STRUMPACK_SCOTCH_BICGSTAB: CPU,    2.370s; elapsed,    0.148s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.540s; elapsed,    0.051s, #calls:   1', 'time for          STRUMPACK_SCOTCH_AUTO: CPU,    7.370s; elapsed,    0.466s, #calls:   1', 'time for  STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    6.810s; elapsed,    0.426s, #calls:   1', 'time for       STRUMPACK_METIS_BICGSTAB: CPU,    2.660s; elapsed,    0.166s, #calls:   1', 'time for                EIGEN_LDLT_CHOL: CPU,    0.870s; elapsed,    0.863s, #calls:   1', 'time for           STRUMPACK_METIS_AUTO: CPU,   32.030s; elapsed,    2.025s, #calls:   1', 'TOTAL                                  : CPU,   86.730s; elapsed,    7.075s']
Data Set Name:=strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-levmar.parameter_flags=Deff-levmar.parameter_flags=Eta-.csv'])
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', 'individual call time for EIGEN_LLT_CHOL: CPU,    3.890s; elapsed,    3.903s', 'individual call time for EIGEN_LDLT_CHOL: CPU,    4.070s; elapsed,    4.073s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.520s; elapsed,    0.047s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_AUTO: CPU,    7.190s; elapsed,    0.454s', 'individual call time for STRUMPACK_METIS_AUTO: CPU,   38.310s; elapsed,    2.408s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_BICGSTAB: CPU,    2.510s; elapsed,    0.157s', 'individual call time for STRUMPACK_METIS_BICGSTAB: CPU,    2.690s; elapsed,    0.168s', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    6.390s; elapsed,    0.401s', 'individual call time for STRUMPACK_METIS_PRECBICGSTAB: CPU,   41.380s; elapsed,    2.597s', 'Traceback (most recent call last):', '  File "./OMP_SOLVER.py", line 103, in <module>', '    run_solver(n_rows, n_cols, A_indptr ,A_indices, A_values, b, A_spS)', '  File "./OMP_SOLVER.py", line 93, in run_solver', '    for i in xrange(len(res_strum_sc.x)):', "NameError: global name 'res_strum_sc' is not defined", 'Exiting profiler', 'time for   STRUMPACK_METIS_PRECBICGSTAB: CPU,   41.380s; elapsed,    2.597s, #calls:   1', 'time for                 EIGEN_LLT_CHOL: CPU,    3.890s; elapsed,    3.903s, #calls:   1', 'time for      STRUMPACK_SCOTCH_BICGSTAB: CPU,    2.510s; elapsed,    0.157s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.520s; elapsed,    0.047s, #calls:   1', 'time for          STRUMPACK_SCOTCH_AUTO: CPU,    7.190s; elapsed,    0.454s, #calls:   1', 'time for  STRUMPACK_SCOTCH_PRECBICGSTAB: CPU,    6.390s; elapsed,    0.401s, #calls:   1', 'time for       STRUMPACK_METIS_BICGSTAB: CPU,    2.690s; elapsed,    0.168s, #calls:   1', 'time for                EIGEN_LDLT_CHOL: CPU,    4.070s; elapsed,    4.073s, #calls:   1', 'time for           STRUMPACK_METIS_AUTO: CPU,   38.310s; elapsed,    2.408s, #calls:   1', 'TOTAL                                  : CPU,  106.950s; elapsed,   14.208s']
Data Set Size:=5k
Skipping 5k
Data Set Size:=32k
Skipping 32k

In the event of a crash, it is wise to save the resulting data (JSON and Pickle) to disk…

import json, cPickle
with open("OMP_SOLVER_TIME_1k_all.json", "w") as OMP_SOLVER_TIME_FILE:
    OMP_SOLVER_TIME_FILE.write(json.dumps(str_out)) 
with open("OMP_SOLVER_TIME_1k_all.pickle", "w") as OMP_SOLVER_TIME_FILE:
    OMP_SOLVER_TIME_FILE.write(cPickle.dumps(str_out))

…which can then be easily reloaded later. Now, we parse the string, extract the timing data, and add it to the python object at the specified thread number index. This allows for easily sorting and plot the data later.

import cPickle
str_out=cPickle.load(open('OMP_SOLVER_TIME_1k_all.pickle', 'rb'))

To simplify indexing later for plots, we create shortened keys. Additionally, we ca also separate the data based upon unique index set and thread numbers.

str_out_sm={}
for k in str_out.keys():
    str_out_sm.update({k.replace('levmar.parameter_flags=','').replace('_strum_1k_omp1_params','_'):str_out[k]})
import pandas as pd
u_dat={}
threads_list=[16]
uniq_ref = set([k.split('_')[1] for k in str_out_sm.keys()])
df_list=[]
for u in uniq_ref:
    same_t = lambda: None #Functions are objects, so legit
    same_t.strum_scotch_a = []
    same_t.strum_metis_a = []
    same_t.strum_scotch_bi = []
    same_t.strum_metis_bi = []
    same_t.strum_scotch_pr = []
    same_t.strum_metis_pr = []
    same_t.eig_llt = []
    same_t.eig_ldlt = []
    same_t.eig_bicgstab = []
    for t in threads_list:
        same_t.strum_metis_a.append(str_out_sm["omp%d_%s"%(t,u)][-2].rsplit(',')[-2].strip()[0:-1])
        same_t.eig_ldlt.append(str_out_sm["omp%d_%s"%(t,u)][-3].rsplit(',')[-2].strip()[0:-1])
        same_t.strum_metis_bi.append(str_out_sm["omp%d_%s"%(t,u)][-4].rsplit(',')[-2].strip()[0:-1])
        same_t.strum_scotch_pr.append(str_out_sm["omp%d_%s"%(t,u)][-5].rsplit(',')[-2].strip()[0:-1])
        same_t.strum_scotch_a.append(str_out_sm["omp%d_%s"%(t,u)][-6].rsplit(',')[-2].strip()[0:-1])
        same_t.eig_bicgstab.append(str_out_sm["omp%d_%s"%(t,u)][-7].rsplit(',')[-2].strip()[0:-1]) 
        same_t.strum_scotch_bi.append(str_out_sm["omp%d_%s"%(t,u)][-8].rsplit(',')[-2].strip()[0:-1])
        same_t.eig_llt.append(str_out_sm["omp%d_%s"%(t,u)][-9].rsplit(',')[-2].strip()[0:-1])
        same_t.strum_metis_pr.append(str_out_sm["omp%d_%s"%(t,u)][-10].rsplit(',')[-2].strip()[0:-1])

    u_dat.update({u:same_t})
    str_E_LLT = "EIG_LLT_%s"%u
    str_E_LDLT = "EIG_LDLT_%s"%u
    str_E_BICGSTAB = "EIG_BICGSTAB_%s"%u
    str_S_SCOTCH_A = "STRUM_SCOTCHA_%s"%u
    str_S_METIS_A = "STRUM_METISA_%s"%u
    str_S_SCOTCH_BI = "STRUM_SCOTCHBI_%s"%u
    str_S_METIS_BI = "STRUM_METISBI_%s"%u
    str_S_SCOTCH_PR = "STRUM_SCOTCHPR_%s"%u
    str_S_METIS_PR = "STRUM_METISPR_%s"%u
    df_list.append( pd.DataFrame({str_E_LLT:same_t.eig_llt, str_E_LDLT:same_t.eig_ldlt, str_E_BICGSTAB:same_t.eig_bicgstab, str_S_SCOTCH_A:same_t.strum_scotch_a, str_S_METIS_A:same_t.strum_metis_a, str_S_SCOTCH_BI:same_t.strum_scotch_bi, str_S_METIS_BI:same_t.strum_metis_bi, str_S_SCOTCH_PR:same_t.strum_scotch_pr, str_S_METIS_PR:same_t.strum_metis_pr }, index=threads_list).transpose() )

We now have a list of Pandas Dataframes, with which we can combine to a single entity. We can then proceed to plot the resulting columns, comparing both Eigen and STRUMPACK for each specific set of refined parameters.

perf_s = pd.concat(df_list).transpose()
perf_s.transpose()

—CLICK FOR OUTPUT—

16
EIG_BICGSTAB_Rxy-Deff-Eta- 0.024
EIG_LDLT_Rxy-Deff-Eta- 2.119
EIG_LLT_Rxy-Deff-Eta- 1.994
STRUM_METISA_Rxy-Deff-Eta- 2.157
STRUM_METISBI_Rxy-Deff-Eta- 0.104
STRUM_METISPR_Rxy-Deff-Eta- 2.350
STRUM_SCOTCHA_Rxy-Deff-Eta- 0.220
STRUM_SCOTCHBI_Rxy-Deff-Eta- 0.111
STRUM_SCOTCHPR_Rxy-Deff-Eta- 0.187
EIG_BICGSTAB_Bfactor- 0.049
EIG_LDLT_Bfactor- 0.238
EIG_LLT_Bfactor- 0.232
STRUM_METISA_Bfactor- 1.831
STRUM_METISBI_Bfactor- 0.177
STRUM_METISPR_Bfactor- 2.001
STRUM_SCOTCHA_Bfactor- 0.447
STRUM_SCOTCHBI_Bfactor- 0.169
STRUM_SCOTCHPR_Bfactor- 0.422
EIG_BICGSTAB_Bfactor-Deff-Eta- 0.051
EIG_LDLT_Bfactor-Deff-Eta- 0.863
EIG_LLT_Bfactor-Deff-Eta- 0.835
STRUM_METISA_Bfactor-Deff-Eta- 2.025
STRUM_METISBI_Bfactor-Deff-Eta- 0.166
STRUM_METISPR_Bfactor-Deff-Eta- 2.094
STRUM_SCOTCHA_Bfactor-Deff-Eta- 0.466
STRUM_SCOTCHBI_Bfactor-Deff-Eta- 0.148
STRUM_SCOTCHPR_Bfactor-Deff-Eta- 0.426
EIG_BICGSTAB_Rxy-Bfactor-Deff-Eta- 0.047
EIG_LDLT_Rxy-Bfactor-Deff-Eta- 4.073
EIG_LLT_Rxy-Bfactor-Deff-Eta- 3.903
... ...
STRUM_SCOTCHA_Deff- 0.215
STRUM_SCOTCHBI_Deff- 0.109
STRUM_SCOTCHPR_Deff- 0.192
EIG_BICGSTAB_Rxy-Bfactor- 0.048
EIG_LDLT_Rxy-Bfactor- 2.120
EIG_LLT_Rxy-Bfactor- 1.995
STRUM_METISA_Rxy-Bfactor- 2.191
STRUM_METISBI_Rxy-Bfactor- 0.163
STRUM_METISPR_Rxy-Bfactor- 2.391
STRUM_SCOTCHA_Rxy-Bfactor- 0.417
STRUM_SCOTCHBI_Rxy-Bfactor- 0.153
STRUM_SCOTCHPR_Rxy-Bfactor- 0.433
EIG_BICGSTAB_Rxy-Bfactor-Eta- 0.048
EIG_LDLT_Rxy-Bfactor-Eta- 2.119
EIG_LLT_Rxy-Bfactor-Eta- 2.023
STRUM_METISA_Rxy-Bfactor-Eta- 2.411
STRUM_METISBI_Rxy-Bfactor-Eta- 0.161
STRUM_METISPR_Rxy-Bfactor-Eta- 2.354
STRUM_SCOTCHA_Rxy-Bfactor-Eta- 0.401
STRUM_SCOTCHBI_Rxy-Bfactor-Eta- 0.151
STRUM_SCOTCHPR_Rxy-Bfactor-Eta- 0.434
EIG_BICGSTAB_Rxy-Bfactor-Deff- 0.050
EIG_LDLT_Rxy-Bfactor-Deff- 4.098
EIG_LLT_Rxy-Bfactor-Deff- 3.883
STRUM_METISA_Rxy-Bfactor-Deff- 2.365
STRUM_METISBI_Rxy-Bfactor-Deff- 0.167
STRUM_METISPR_Rxy-Bfactor-Deff- 2.539
STRUM_SCOTCHA_Rxy-Bfactor-Deff- 0.435
STRUM_SCOTCHBI_Rxy-Bfactor-Deff- 0.154
STRUM_SCOTCHPR_Rxy-Bfactor-Deff- 0.450

135 rows × 1 columns

With the performance data in an easily accessible format, we can plot each respective data set for all solvers. Each bar corresponds to the parameters being refined, where the colour indicates the respective solver used, and the y-axis is the time to solve the system in seconds.

# define the figure size and grid layout properties
matplotlib.rcParams.update({'font.size': 22})
figsize = (20, 16)
cols = 4
fig3 = plt.figure(num=3, figsize=figsize)
ax3 = plt.axes()
ii = []
uu=[]
keys=['EIG_LLT','EIG_LDLT','STRUM_METISA','STRUM_SCOTCHA','STRUM_METISBI','STRUM_SCOTCHBI','STRUM_METISPR','STRUM_SCOTCHPR','EIG_BICGSTAB']
colours = ['#a6cee3','#1f78b4','#b2df8a','#33a02c','#fb9a99','#e31a1c','#fdbf6f','#ff7f00','#cab2d6']
key_col = {k:v for (k,v) in zip(keys,colours)}
hatch_pattern = [ '/' , '\\' , '|' , '-' , '+' , 'x' ,'o' ,'O' , '.', '*']
for i, u in enumerate(uniq_ref):
    p_list=[]
    ax3.set_ylabel('time [s]')
    keys_df = [k+'_'+u for k in keys]
    dfl=perf_s[keys_df]
    
    m = zip(dfl.as_matrix().astype(np.float)[0], dfl.keys().tolist() )
    m = sorted(m,key=lambda x: x[0])

    for p in reversed(xrange(len(m))):
        c_label =  "_".join(m[p][1].split('_')[0:2])
        p_list.append( plt.bar(i, m[p][0], label=m[p][1], color=key_col[c_label], edgecolor='k') )
    
    ii.append(i)
    uu.append(u)
ax3.set_yscale('log',basey=10)
plt.title('1k frames OMP_NUM_THREADS=16', fontsize=24)
plt.xticks(ii, uu)
plt.setp(ax3.get_xticklabels(), rotation=90, ha='left')
ax3.legend(p_list,keys,bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.,)

plt.savefig('omp_1kframes_allsolvers_omp16.pdf', pad_inches=10)
png

So far we have examined the single threaded and OpenMP enabled solver performance. Strumpack also allows us to use the distributed MPI backend. We can set this up as follows using the ipyparallel backend, or invoke an mpirun call using the cell bash bash environment. For completeness, the ipyparallel environment can be set up as follows:

conda install ipyparallel
jupyter serverextension enable --py ipyparallel --user
jupyter nbextension install --py ipyparallel --user 
jupyter nbextension enable --py ipyparallel --user
ipcluster start --profile=mpi -n 4

Take note of the controller client json file <HOME>/.ipython/profile_<PROFILE_NAME>/security/ipcontroller-client.json. This will be used to allow the notebook to connect to the ipyparallel cluster profile mpi. We can then return the handle of the cluster environment with:

import ipyparallel as ipp
#c = ipp.Client('/net/cci-filer2/raid1/home/mlxd/.ipython/profile_mpi/security/ipcontroller-client.json')
c = ipp.Client()
rc=c[:]
c.ids #Instance IDs
[0]

For our purposes it can be easier to call the solver using the ! bash environment, as was used for the OpenMP studies earlier. A simple test script can be performed as follows, which tests the listed solvers against the MPI backend.

!OMP_NUM_THREADS=1 mpirun -np 2 libtbx.python \
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/\
modules/cctbx_project/scitbx/examples/bevington/strumpack_eigen_solver_mpi_dist.py \
/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/\
out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/A_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv \
/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_1k/\
out_strumpack_1k_omp1_paramslevmar.parameter_flags=Rxy-/b_strum_1k_omp1_paramslevmar.parameter_flags=Rxy-.csv
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.
  try: mod = __import__(name)
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.
  try: mod = __import__(name)
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.
  try: mod = __import__(name)
/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.
  try: mod = __import__(name)
14151 7075.5
# ***** WARNING ****************************************************
# Detected a large number of levels in the frontal/elimination tree.
# STRUMPACK currently does not handle this safely, which
# could lead to segmentation faults due to stack overflows.
# As a remedy, you can try to increase the stack size,
# or try a different ordering (metis, scotch, ..).
# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,
# (enabled by default) iso --sp_enable_METIS_NodeND.
# ******************************************************************
individual call time for STRUMPACK_OMP: CPU,    0.350s; elapsed,    0.344s
individual call time for EIGEN_LDLT: CPU,    0.050s; elapsed,    0.050s
individual call time for EIGEN_BICGSTAB: CPU,    0.020s; elapsed,    0.027s
CPP -2
# Initializing STRUMPACK
# using 1 OpenMP thread
CPP -2
# using 2 MPI processes
# matching job: maximum matching with row and column scaling
# initial matrix:
#   - number of unknowns = 14,151
#   - number of nonzeros = 72,305
# nested dissection reordering:
#   - Scotch reordering
#   - strategy parameter = 8
#   - number of separators = 4,075
#   - number of levels = 68
# ***** WARNING ****************************************************
# Detected a large number of levels in the frontal/elimination tree.
# STRUMPACK currently does not handle this safely, which
# could lead to segmentation faults due to stack overflows.
# As a remedy, you can try to increase the stack size,
# or try a different ordering (metis, scotch, ..).
# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,
# (enabled by default) iso --sp_enable_METIS_NodeND.
# ******************************************************************
#   - nd time = 0.0810261
#   - matching time = 0.0116782
#   - symmetrization time = 0.00337219
# symbolic factorization:
#   - nr of dense Frontal matrices = 4,075
#   - nr of HSS Frontal matrices = 0
#   - symb-factor time = 0.021524
# multifrontal factorization:
#   - estimated memory usage (exact solver) = 7.70359 MB
#   - factor time = 0.223322
#   - factor nonzeros = 962,949
#   - factor memory = 7.70359 MB
#   - total flops = 2.49112e+08, min = 3.46562e+07, max = 2.14456e+08
#   - flop rate = 1.11548 GFlop/s
#   - factor memory/nonzeros = 100 % of multifrontal
#   - maximum HSS rank = 0
#   - HSS compression = false
#   - relative compression tolerance = 0.01
#   - absolute compression tolerance = 1e-08
#   - normal(0,1) distribution with minstd_rand engine

# ----- FLOP BREAKDOWN ---------------------
# compression           = 0
#    random             = 0
#    ID                 = 0
#    QR                 = 0
#    ortho              = 0
#    reduce_samples     = 0
#    update_samples     = 0
#    extraction         = 0
#    sampling           = 0
#       CB_sample       = 0
#       sparse_sampling = 0
# ULV_factor            = 0
# Schur                 = 0
# full_rank             = 2.63271e+08
# --------------------------------------------
# total                 = 2.63271e+08
# --------------------------------------------

REFINEMENT it. 0    res =        15322  rel.res =            1  bw.error =            1
REFINEMENT it. 1    res =   1.1079e-11  rel.res =  7.23077e-16  bw.error =  1.25632e-15
# DIRECT/GMRES solve:
#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000
#   - number of Krylov iterations = 1
#   - solve time = 0.011821
#   - total flops = 2.47602e+06, min = 838849, max = 1.63717e+06
#   - flop rate = 0.209459 GFlop/s
#   - bytes moved = 21.8361 MB, min = 0 MB, max = 0 MB
#   - byte rate = 1.84722 GByte/s
#   - solve arithmetic intensity = 0.113391 flop/byte
individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    0.350s; elapsed,    0.359s
individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    0.370s; elapsed,    0.365s
Exiting profiler
time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    0.350s; elapsed,    0.359s, #calls:   1
TOTAL                                  : CPU,    0.350s; elapsed,    0.359s
Strumpack solutions agree with Eigen
Exiting profiler
time for                  STRUMPACK_OMP: CPU,    0.350s; elapsed,    0.344s, #calls:   1
time for                 EIGEN_BICGSTAB: CPU,    0.020s; elapsed,    0.027s, #calls:   1
time for                     EIGEN_LDLT: CPU,    0.050s; elapsed,    0.050s, #calls:   1
time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    0.370s; elapsed,    0.365s, #calls:   1
TOTAL                                  : CPU,    0.790s; elapsed,    0.786s

We now use the same procedure of creating a solver file, write to disk, and call with the respective matrix arguments with OMP_NUM_THREADS and mpirun -n. For simplicity, we examine the OpenMP STRUM_SCOTCHA, EIGEN_LDLT and EIGEN_BICGSTAB, compared with the MPI-enabled STRUM_SCOTCHA (we may also examine the full range of algorithms as used in the previous example).

MPI_SOLVER='''
from __future__ import division

import mpi4py
mpi4py.rc.threads = True
mpi4py.rc.thread_level = "funneled"
from mpi4py import MPI

assert MPI.Is_initialized()
assert MPI.Query_thread() == MPI.THREAD_FUNNELED

comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size()

from scitbx.matrix import sqr,col
from cctbx.array_family import flex
from libtbx.test_utils import approx_equal
from libtbx.development.timers import Profiler

import boost.python
ext_omp = boost.python.import_ext("scitbx_examples_strumpack_solver_ext")
ext_mpi = boost.python.import_ext("scitbx_examples_strumpack_mpi_dist_solver_ext")
import sys
import numpy as np

import scipy.sparse as sps

if rank==0:
  A_mat = np.loadtxt(sys.argv[1],dtype={'names':('rows','cols','vals'),'formats':('i8','i8','f8')})
  b_vec = np.loadtxt(sys.argv[2])
  n_rows = len(b_vec)
  n_cols = n_rows
  nnz = len(A_mat['vals'])
  print n_rows, n_rows/size
  #Convert the sparse CSR to flex doubles, then use them to solve using the implemented framework
  A_sp = sps.csr_matrix((A_mat['vals'],(A_mat['rows'],A_mat['cols']))) 

  #Check if upper/lower triangular only, and generate full if so
  tu=sps.triu(A_sp)
  tl=sps.tril(A_sp)
  sd=sps.diags(A_sp.diagonal())

  A_spS = A_sp
  if tu.nnz == sd.getnnz() or tl.nnz == sd.getnnz():
    A_spS = A_sp + A_sp.transpose() - sd

  import numpy as np
  row_idx_split = np.array_split(np.arange(n_rows),size)
  len_row_idx_split = flex.int( np.cumsum( np.append([0], [len(i) for i in row_idx_split]) ).tolist() )
  P = Profiler("STRUMPACK_OMP")
  A_indptr = flex.int( A_spS.indptr )
  A_indices = flex.int( A_spS.indices )
  A_data = flex.double( A_spS.data )
  b = flex.double( b_vec )
  res_strum_omp = ext_omp.strumpack_solver(
                    n_rows, n_cols, 
                    A_indptr, A_indices,
                    A_data, b,
                    ext_omp.scotch, 
                    ext_omp.auto
                 )
  del P
  P = Profiler("EIGEN_LDLT")
  res_eig_ldlt = ext_omp.eigen_solver(1, n_rows, n_cols, A_indptr, A_indices, A_data, b)
  del P
  P = Profiler("EIGEN_BICGSTAB")
  res_eig_bicgstab = ext_omp.eigen_solver(2, n_rows, n_cols, A_indptr, A_indices, A_data, b)
  del P

else:
  A_spS=None
  row_idx_split=None
  len_row_idx_split=None
  b_vec=None
  n_cols=None

if size>1:
  #Broadcast data to each rank
  A_spS = comm.bcast(A_spS, root=0)
  row_idx_split = comm.bcast(row_idx_split, root=0)
  len_row_idx_split = comm.bcast(len_row_idx_split, root=0)
  b_vec = comm.bcast(b_vec, root=0)
  n_cols = comm.bcast(n_cols, root=0)

  #Take subset of data for each rank
  A_row_offset = flex.int(A_spS[row_idx_split[rank],:].indptr)
  A_col_offset = flex.int(A_spS[row_idx_split[rank],:].indices)
  A_values = flex.double(A_spS[row_idx_split[rank],:].data)
  b = flex.double(b_vec[row_idx_split[rank]])

  P = Profiler("STRUMPACK_MPI_DIST_RANK=%d"%rank)
  res_strum_mpi_local = ext_mpi.strumpack_mpi_dist_solver(len(row_idx_split[rank]), n_cols, comm, A_row_offset, A_col_offset, A_values, b, len_row_idx_split, ext_mpi.scotch, ext_mpi.auto)
  strum_result_mpi_list = comm.gather(res_strum_mpi_local.x, root=0)
  del P
  if rank==0:
    strum_result_mpi = flex.double()
    for l in strum_result_mpi_list:
      strum_result_mpi.extend(l)

#MPI.Finalize()
if rank==0:
  strum_result_omp = res_strum_omp.x
  eig_result_ldlt = res_eig_ldlt.x
  eig_result_bicgstab = res_eig_bicgstab.x

  for ii in xrange(len(strum_result_omp)):
    assert approx_equal( strum_result_omp[ii], eig_result_ldlt[ii]  )
    assert approx_equal( strum_result_omp[ii], eig_result_bicgstab[ii]  )
    if size>1:
      assert approx_equal( strum_result_omp[ii], strum_result_mpi[ii]  )
  print "Strumpack solutions agree with Eigen"

'''
MPI_SOLVER_FILE = open("MPI_SOLVER.py", "w")
MPI_SOLVER_FILE.write(MPI_SOLVER)
MPI_SOLVER_FILE.close()

DATAPATH="/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/"
A_LIST = !find {DATAPATH} -iname "A*.csv"
B_LIST = [ii.replace('/A_','/b_') for ii in A_LIST]

We intend to examine performance while varying both OMP_NUM_THREADS and the MPI process number to determine an optimal set of parameters for the distributed backend. In the following example we test between 1 to 16 OpenMP threads per process, and from 1 to 16 MPI processes, assuming a total of threads*processes <= 32 cores. The resulting data set will be a scalar field of time as a function of both variables, which we can plot as a heatmap. We can begin by running the analysis on a 1000 image dataset.

str_out={}
import os
threads_list = [1,2,4,8,16]
mpi_proc_list = [1,2,4,8,16]
for imgs_size in list_idx:
    print "Data Set Size:=%s"%imgs_size
    #Subselect the smallest data size for now
    if imgs_size != "5k":
        print "Skipping %s"%imgs_size
        continue

    for imgs_idx in list_idx[imgs_size]:

        A_path = A_LIST[imgs_idx]; b_path = B_LIST[imgs_idx]
        dat_name = A_path.split('/')[-1][2:-4]

        print "Data Set Name:=%s"%(dat_name)

        #Ensure the A and b data are matched correctly
        assert(os.path.dirname(A_path) == os.path.dirname(b_path))
        print {A_path}, {b_path}

        for procs in mpi_proc_list:
            for threads in threads_list:
                if threads*procs <= 32:
                    print "mpirun -n %d"%procs
                    print "OMP_NUM_THREADS:=%d"%threads

                    val = !OMP_NUM_THREADS={threads} mpirun -n {procs} libtbx.python ./MPI_SOLVER.py {A_path} {b_path}
                    print val
                    for s in val:
                        if "assert" in s:
                            raise Exception("Solutions not equal! Halting with log: %s"%val)
                    key = 'mpi' + str(procs)  + 'omp' + str(threads) + '_' + dat_name
                    str_out.update({key:val})

—CLICK FOR OUTPUT—

Data Set Size:=10k
Skipping 10k
Data Set Size:=1k
Skipping 1k
Data Set Size:=5k
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Rxy-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-/A_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-/b_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    5.630s; elapsed,    5.624s', 'individual call time for EIGEN_LDLT: CPU,    2.490s; elapsed,    2.496s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.056s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    5.630s; elapsed,    5.624s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.056s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.490s; elapsed,    2.496s, #calls:   1', 'TOTAL                                  : CPU,    8.170s; elapsed,    8.175s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    6.920s; elapsed,    3.464s', 'individual call time for EIGEN_LDLT: CPU,    2.660s; elapsed,    2.483s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.060s; elapsed,    0.044s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    6.920s; elapsed,    3.464s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.060s; elapsed,    0.044s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.660s; elapsed,    2.483s, #calls:   1', 'TOTAL                                  : CPU,    9.640s; elapsed,    5.992s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    9.110s; elapsed,    2.283s', 'individual call time for EIGEN_LDLT: CPU,    3.630s; elapsed,    3.089s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.100s; elapsed,    0.041s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    9.110s; elapsed,    2.283s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.100s; elapsed,    0.041s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.630s; elapsed,    3.089s, #calls:   1', 'TOTAL                                  : CPU,   12.840s; elapsed,    5.413s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   22.270s; elapsed,    2.789s', 'individual call time for EIGEN_LDLT: CPU,    5.570s; elapsed,    4.334s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.180s; elapsed,    0.042s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   22.270s; elapsed,    2.789s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.180s; elapsed,    0.042s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    5.570s; elapsed,    4.334s, #calls:   1', 'TOTAL                                  : CPU,   28.020s; elapsed,    7.164s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   38.440s; elapsed,    2.409s', 'individual call time for EIGEN_LDLT: CPU,    5.150s; elapsed,    2.538s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.280s; elapsed,    0.041s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   38.440s; elapsed,    2.409s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.280s; elapsed,    0.041s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    5.150s; elapsed,    2.538s, #calls:   1', 'TOTAL                                  : CPU,   43.870s; elapsed,    4.989s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    5.910s; elapsed,    5.904s', 'individual call time for EIGEN_LDLT: CPU,    2.560s; elapsed,    2.559s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.058s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,927', '#   - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.173102', '#   - matching time = 0.031208', '#   - symmetrization time = 0.015681', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,927', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0421081', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.1376 MB', '#   - factor time = 3.87764', '#   - factor nonzeros = 6,142,201', '#   - factor memory = 49.1376 MB', '#   - total flops = 7.46369e+09, min = 2.94438e+09, max = 4.51931e+09', '#   - flop rate = 1.92481 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.06944e+09', '# --------------------------------------------', '# total                 = 8.06944e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =   3.6413e-11\trel.res =  1.55862e-15\tbw.error =  3.62098e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.042994', '#   - total flops = 1.47533e+07, min = 6.98046e+06, max = 7.77288e+06', '#   - flop rate = 0.343149 GFlop/s', '#   - bytes moved = 105.81 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.46104 GByte/s', '#   - solve arithmetic intensity = 0.139433 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    4.200s; elapsed,    4.201s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    4.200s; elapsed,    4.211s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    4.200s; elapsed,    4.201s, #calls:   1', 'TOTAL                                  : CPU,    4.200s; elapsed,    4.201s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    5.910s; elapsed,    5.904s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.058s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.560s; elapsed,    2.559s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    4.200s; elapsed,    4.211s, #calls:   1', 'TOTAL                                  : CPU,   12.720s; elapsed,   12.732s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    7.910s; elapsed,    3.958s', 'individual call time for EIGEN_LDLT: CPU,    2.650s; elapsed,    2.465s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.048s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,927', '#   - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.165474', '#   - matching time = 0.032366', '#   - symmetrization time = 0.0156758', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,927', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0455151', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.1376 MB', '#   - factor time = 2.88028', '#   - factor nonzeros = 6,142,201', '#   - factor memory = 49.1376 MB', '#   - total flops = 7.46376e+09, min = 2.94438e+09, max = 4.51938e+09', '#   - flop rate = 2.59133 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.06944e+09', '# --------------------------------------------', '# total                 = 8.06944e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.67124e-11\trel.res =  1.57144e-15\tbw.error =  3.63192e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0395958', '#   - total flops = 1.47589e+07, min = 6.98046e+06, max = 7.77839e+06', '#   - flop rate = 0.372737 GFlop/s', '#   - bytes moved = 106.027 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.67773 GByte/s', '#   - solve arithmetic intensity = 0.139199 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    5.760s; elapsed,    3.205s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    6.430s; elapsed,    3.215s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    5.760s; elapsed,    3.205s, #calls:   1', 'TOTAL                                  : CPU,    5.760s; elapsed,    3.205s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    7.910s; elapsed,    3.958s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.048s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.650s; elapsed,    2.465s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    6.430s; elapsed,    3.215s, #calls:   1', 'TOTAL                                  : CPU,   17.060s; elapsed,    9.685s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   10.940s; elapsed,    2.741s', 'individual call time for EIGEN_LDLT: CPU,    6.260s; elapsed,    5.728s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.120s; elapsed,    0.057s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,927', '#   - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.165105', '#   - matching time = 0.0321209', '#   - symmetrization time = 0.01565', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,927', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0445859', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.1376 MB', '#   - factor time = 2.10367', '#   - factor nonzeros = 6,142,201', '#   - factor memory = 49.1376 MB', '#   - total flops = 7.46376e+09, min = 2.94438e+09, max = 4.51938e+09', '#   - flop rate = 3.54796 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.06944e+09', '# --------------------------------------------', '# total                 = 8.06944e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =   3.6699e-11\trel.res =  1.57086e-15\tbw.error =  3.63192e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0426569', '#   - total flops = 1.47589e+07, min = 6.98046e+06, max = 7.77839e+06', '#   - flop rate = 0.34599 GFlop/s', '#   - bytes moved = 106.038 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.48584 GByte/s', '#   - solve arithmetic intensity = 0.139184 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    9.690s; elapsed,    2.430s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    9.750s; elapsed,    2.439s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    9.690s; elapsed,    2.430s, #calls:   1', 'TOTAL                                  : CPU,    9.690s; elapsed,    2.430s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   10.940s; elapsed,    2.741s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.120s; elapsed,    0.057s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    6.260s; elapsed,    5.728s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    9.750s; elapsed,    2.439s, #calls:   1', 'TOTAL                                  : CPU,   27.070s; elapsed,   10.966s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   22.810s; elapsed,    2.860s', 'individual call time for EIGEN_LDLT: CPU,    3.760s; elapsed,    2.491s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.150s; elapsed,    0.038s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,927', '#   - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.194211', '#   - matching time = 0.0363538', '#   - symmetrization time = 0.0187259', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,927', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0537779', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.1376 MB', '#   - factor time = 2.60438', '#   - factor nonzeros = 6,142,201', '#   - factor memory = 49.1376 MB', '#   - total flops = 7.46376e+09, min = 2.94438e+09, max = 4.51938e+09', '#   - flop rate = 2.86585 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.06944e+09', '# --------------------------------------------', '# total                 = 8.06944e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.66517e-11\trel.res =  1.56884e-15\tbw.error =  3.63192e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0529561', '#   - total flops = 1.47589e+07, min = 6.98046e+06, max = 7.77839e+06', '#   - flop rate = 0.2787 GFlop/s', '#   - bytes moved = 106.061 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.0028 GByte/s', '#   - solve arithmetic intensity = 0.139155 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   21.600s; elapsed,    2.991s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   23.830s; elapsed,    3.000s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   21.600s; elapsed,    2.991s, #calls:   1', 'TOTAL                                  : CPU,   21.600s; elapsed,    2.991s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   22.810s; elapsed,    2.860s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.150s; elapsed,    0.038s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.760s; elapsed,    2.491s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   23.830s; elapsed,    3.000s, #calls:   1', 'TOTAL                                  : CPU,   50.550s; elapsed,    8.389s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   40.870s; elapsed,    2.576s', 'individual call time for EIGEN_LDLT: CPU,    5.590s; elapsed,    2.941s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.260s; elapsed,    0.039s', 'CPP -2', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,927', '#   - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.208532', '#   - matching time = 0.05004', '#   - symmetrization time = 0.02074', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,927', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.061316', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.1376 MB', '#   - factor time = 2.22156', '#   - factor nonzeros = 6,142,201', '#   - factor memory = 49.1376 MB', '#   - total flops = 7.46395e+09, min = 2.94438e+09, max = 4.51957e+09', '#   - flop rate = 3.35978 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.06944e+09', '# --------------------------------------------', '# total                 = 8.06944e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.65568e-11\trel.res =  1.56478e-15\tbw.error =  3.57131e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0466371', '#   - total flops = 1.47589e+07, min = 6.98046e+06, max = 7.77839e+06', '#   - flop rate = 0.316462 GFlop/s', '#   - bytes moved = 106.109 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.2752 GByte/s', '#   - solve arithmetic intensity = 0.139092 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   41.450s; elapsed,    2.640s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   41.650s; elapsed,    2.644s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   41.450s; elapsed,    2.640s, #calls:   1', 'TOTAL                                  : CPU,   41.450s; elapsed,    2.640s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   40.870s; elapsed,    2.576s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.260s; elapsed,    0.039s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    5.590s; elapsed,    2.941s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   41.650s; elapsed,    2.644s, #calls:   1', 'TOTAL                                  : CPU,   88.370s; elapsed,    8.199s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    6.280s; elapsed,    6.282s', 'individual call time for EIGEN_LDLT: CPU,    2.610s; elapsed,    2.614s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.060s; elapsed,    0.060s', 'CPP -2', '# Initializing STRUMPACK', 'CPP -2', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,931', '#   - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.189291', '#   - matching time = 0.036289', '#   - symmetrization time = 0.012743', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,931', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.036417', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.3058 MB', '#   - factor time = 2.50507', '#   - factor nonzeros = 6,163,223', '#   - factor memory = 49.3058 MB', '#   - total flops = 7.53221e+09, min = 865688, max = 2.95202e+09', '#   - flop rate = 3.00679 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.13207e+09', '# --------------------------------------------', '# total                 = 8.13207e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.02608e-11\trel.res =  1.29528e-15\tbw.error =  2.43099e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0349491', '#   - total flops = 1.6493e+07, min = 43055, max = 6.92318e+06', '#   - flop rate = 0.471915 GFlop/s', '#   - bytes moved = 98.2063 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.80998 GByte/s', '#   - solve arithmetic intensity = 0.167942 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    2.800s; elapsed,    2.823s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    2.810s; elapsed,    2.824s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    2.810s; elapsed,    2.821s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    2.830s; elapsed,    2.829s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    2.800s; elapsed,    2.823s, #calls:   1', 'TOTAL                                  : CPU,    2.800s; elapsed,    2.823s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    2.810s; elapsed,    2.824s, #calls:   1', 'TOTAL                                  : CPU,    2.810s; elapsed,    2.824s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    2.810s; elapsed,    2.821s, #calls:   1', 'TOTAL                                  : CPU,    2.810s; elapsed,    2.821s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    6.280s; elapsed,    6.282s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.060s; elapsed,    0.060s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.610s; elapsed,    2.614s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    2.830s; elapsed,    2.829s, #calls:   1', 'TOTAL                                  : CPU,   11.780s; elapsed,   11.785s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    8.020s; elapsed,    4.018s', 'individual call time for EIGEN_LDLT: CPU,    2.700s; elapsed,    2.522s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.100s; elapsed,    0.060s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,931', '#   - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.174702', '#   - matching time = 0.032347', '#   - symmetrization time = 0.014987', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,931', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.039263', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.3058 MB', '#   - factor time = 2.04756', '#   - factor nonzeros = 6,163,223', '#   - factor memory = 49.3058 MB', '#   - total flops = 7.53223e+09, min = 865688, max = 2.95202e+09', '#   - flop rate = 3.67864 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.13207e+09', '# --------------------------------------------', '# total                 = 8.13207e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.01024e-11\trel.res =   1.2885e-15\tbw.error =  2.41537e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0343449', '#   - total flops = 1.64951e+07, min = 43055, max = 6.92318e+06', '#   - flop rate = 0.480278 GFlop/s', '#   - bytes moved = 98.2807 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.86158 GByte/s', '#   - solve arithmetic intensity = 0.167837 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    4.070s; elapsed,    2.357s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    4.710s; elapsed,    2.358s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    2.990s; elapsed,    2.362s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    4.700s; elapsed,    2.366s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    4.710s; elapsed,    2.358s, #calls:   1', 'TOTAL                                  : CPU,    4.710s; elapsed,    2.358s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    2.990s; elapsed,    2.362s, #calls:   1', 'TOTAL                                  : CPU,    2.990s; elapsed,    2.362s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    4.070s; elapsed,    2.357s, #calls:   1', 'TOTAL                                  : CPU,    4.070s; elapsed,    2.357s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    8.020s; elapsed,    4.018s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.100s; elapsed,    0.060s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.700s; elapsed,    2.522s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    4.700s; elapsed,    2.366s, #calls:   1', 'TOTAL                                  : CPU,   15.520s; elapsed,    8.966s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   10.990s; elapsed,    2.758s', 'individual call time for EIGEN_LDLT: CPU,    4.350s; elapsed,    3.797s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.110s; elapsed,    0.045s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,931', '#   - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.206648', '#   - matching time = 0.039171', '#   - symmetrization time = 0.0150321', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,931', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0403199', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.3058 MB', '#   - factor time = 2.20638', '#   - factor nonzeros = 6,163,223', '#   - factor memory = 49.3058 MB', '#   - total flops = 7.53223e+09, min = 866384, max = 2.95202e+09', '#   - flop rate = 3.41384 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.13207e+09', '# --------------------------------------------', '# total                 = 8.13207e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.01136e-11\trel.res =  1.28898e-15\tbw.error =  2.43099e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.038558', '#   - total flops = 1.64951e+07, min = 43055, max = 6.92318e+06', '#   - flop rate = 0.4278 GFlop/s', '#   - bytes moved = 98.2891 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.54912 GByte/s', '#   - solve arithmetic intensity = 0.167822 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    7.960s; elapsed,    2.562s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    4.530s; elapsed,    2.565sindividual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    9.930s; elapsed,    2.563s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   10.060s; elapsed,    2.570s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    9.930s; elapsed,    2.563s, #calls:   1', 'TOTAL                                  : CPU,    9.930s; elapsed,    2.563s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    7.960s; elapsed,    2.562s, #calls:   1', 'TOTAL                                  : CPU,    7.960s; elapsed,    2.562s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    4.530s; elapsed,    2.565s, #calls:   1', 'TOTAL                                  : CPU,    4.530s; elapsed,    2.565s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   10.990s; elapsed,    2.758s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.110s; elapsed,    0.045s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    4.350s; elapsed,    3.797s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   10.060s; elapsed,    2.570s, #calls:   1', 'TOTAL                                  : CPU,   25.510s; elapsed,    9.170s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   19.800s; elapsed,    2.481s', 'individual call time for EIGEN_LDLT: CPU,    5.070s; elapsed,    3.836s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.140s; elapsed,    0.038s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,931', '#   - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.202113', '#   - matching time = 0.0367491', '#   - symmetrization time = 0.015816', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,931', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0441282', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.3058 MB', '#   - factor time = 2.54222', '#   - factor nonzeros = 6,163,223', '#   - factor memory = 49.3058 MB', '#   - total flops = 7.53224e+09, min = 866384, max = 2.95202e+09', '#   - flop rate = 2.96286 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.13207e+09', '# --------------------------------------------', '# total                 = 8.13207e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.01093e-11\trel.res =   1.2888e-15\tbw.error =  2.43099e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0427811', '#   - total flops = 1.64951e+07, min = 43055, max = 6.92318e+06', '#   - flop rate = 0.38557 GFlop/s', '#   - bytes moved = 98.3065 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.2979 GByte/s', '#   - solve arithmetic intensity = 0.167793 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    7.520s; elapsed,    2.905s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,   20.270s; elapsed,    2.903s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,   16.040s; elapsed,    2.904s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   22.860s; elapsed,    2.911s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,   16.040s; elapsed,    2.904s, #calls:   1', 'TOTAL                                  : CPU,   16.040s; elapsed,    2.904s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,   20.270s; elapsed,    2.903s, #calls:   1', 'TOTAL                                  : CPU,   20.270s; elapsed,    2.903s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    7.520s; elapsed,    2.905s, #calls:   1', 'TOTAL                                  : CPU,    7.520s; elapsed,    2.905s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   19.800s; elapsed,    2.481s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.140s; elapsed,    0.038s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    5.070s; elapsed,    3.836s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   22.860s; elapsed,    2.911s, #calls:   1', 'TOTAL                                  : CPU,   47.870s; elapsed,    9.266s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 2663.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    6.430s; elapsed,    6.432s', 'individual call time for EIGEN_LDLT: CPU,    2.470s; elapsed,    2.473s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.060s; elapsed,    0.063s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,945', '#   - number of levels = 126', '#   - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.163474', '#   - matching time = 0.0298009', '#   - symmetrization time = 0.0128169', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,945', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0309041', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.269 MB', '#   - factor time = 2.14071', '#   - factor nonzeros = 6,158,621', '#   - factor memory = 49.269 MB', '#   - total flops = 7.49292e+09, min = 698726, max = 2.95246e+09', '#   - flop rate = 3.50021 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.05389e+09', '# --------------------------------------------', '# total                 = 8.05389e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.57236e-11\trel.res =  1.10107e-15\tbw.error =  2.56229e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0334761', '#   - total flops = 2.07556e+07, min = 27135, max = 6.90876e+06', '#   - flop rate = 0.620012 GFlop/s', '#   - bytes moved = 93.7947 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.80184 GByte/s', '#   - solve arithmetic intensity = 0.221288 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    2.400s; elapsed,    2.421s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    2.400s; elapsed,    2.417s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    2.420s; elapsed,    2.423s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    2.380s; elapsed,    2.421s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    2.410s; elapsed,    2.422s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    2.390s; elapsed,    2.419s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    2.410s; elapsed,    2.422s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    2.430s; elapsed,    2.427s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    2.400s; elapsed,    2.417s, #calls:   1', 'TOTAL                                  : CPU,    2.400s; elapsed,    2.417s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    2.380s; elapsed,    2.421s, #calls:   1', 'TOTAL                                  : CPU,    2.380s; elapsed,    2.421s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    2.390s; elapsed,    2.419s, #calls:   1', 'TOTAL                                  : CPU,    2.390s; elapsed,    2.419s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    2.410s; elapsed,    2.422s, #calls:   1', 'TOTAL                                  : CPU,    2.410s; elapsed,    2.422s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    2.400s; elapsed,    2.421s, #calls:   1', 'TOTAL                                  : CPU,    2.400s; elapsed,    2.421s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    2.410s; elapsed,    2.422s, #calls:   1', 'TOTAL                                  : CPU,    2.410s; elapsed,    2.422s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    2.420s; elapsed,    2.423s, #calls:   1', 'TOTAL                                  : CPU,    2.420s; elapsed,    2.423s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    6.430s; elapsed,    6.432s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.060s; elapsed,    0.063s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.470s; elapsed,    2.473s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    2.430s; elapsed,    2.427s, #calls:   1', 'TOTAL                                  : CPU,   11.390s; elapsed,   11.394s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 2663.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    7.380s; elapsed,    3.692s', 'individual call time for EIGEN_LDLT: CPU,    2.740s; elapsed,    2.572s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.080s; elapsed,    0.053s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,945', '#   - number of levels = 126', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************#   - nd time = ', '0.176074', '#   - matching time = 0.033097', '#   - symmetrization time = 0.013762', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,945', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0355759', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.269 MB', '#   - factor time = 2.18519', '#   - factor nonzeros = 6,158,621', '#   - factor memory = 49.269 MB', '#   - total flops = 7.49293e+09, min = 698726, max = 2.95246e+09', '#   - flop rate = 3.42897 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.05389e+09', '# --------------------------------------------', '# total                 = 8.05389e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.55859e-11\trel.res =  1.09518e-15\tbw.error =  2.50926e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.041822', '#   - total flops = 2.07556e+07, min = 27135, max = 6.90876e+06', '#   - flop rate = 0.496285 GFlop/s', '#   - bytes moved = 93.8014 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.24287 GByte/s', '#   - solve arithmetic intensity = 0.221272 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    4.330s; elapsed,    2.499s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    4.050s; elapsed,    2.504s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    3.190s; elapsed,    2.504s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    3.140s; elapsed,    2.503s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    4.910s; elapsed,    2.503s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    3.430s; elapsed,    2.503s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    3.350s; elapsed,    2.501s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    5.010s; elapsed,    2.508s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    4.330s; elapsed,    2.499s, #calls:   1', 'TOTAL                                  : CPU,    4.330s; elapsed,    2.499s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    4.050s; elapsed,    2.504s, #calls:   1', 'TOTAL                                  : CPU,    4.050s; elapsed,    2.504s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    3.430s; elapsed,    2.503s, #calls:   1', 'TOTAL                                  : CPU,    3.430s; elapsed,    2.503s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    3.350s; elapsed,    2.501s, #calls:   1', 'TOTAL                                  : CPU,    3.350s; elapsed,    2.501s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    3.140s; elapsed,    2.503s, #calls:   1', 'TOTAL                                  : CPU,    3.140s; elapsed,    2.503s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    4.910s; elapsed,    2.503s, #calls:   1', 'TOTAL                                  : CPU,    4.910s; elapsed,    2.503s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    3.190s; elapsed,    2.504s, #calls:   1', 'TOTAL                                  : CPU,    3.190s; elapsed,    2.504s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    7.380s; elapsed,    3.692s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.080s; elapsed,    0.053s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.740s; elapsed,    2.572s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    5.010s; elapsed,    2.508s, #calls:   1', 'TOTAL                                  : CPU,   15.210s; elapsed,    8.825s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 2663.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   11.610s; elapsed,    2.911s', 'individual call time for EIGEN_LDLT: CPU,    3.150s; elapsed,    2.638s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.110s; elapsed,    0.053s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,945', '#   - number of levels = 126', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.202794', '#   - matching time = 0.0369658', '#   - symmetrization time = 0.0166161', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,945', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0442638', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.269 MB', '#   - factor time = 2.17352', '#   - factor nonzeros = 6,158,621', '#   - factor memory = 49.269 MB', '#   - total flops = 7.49293e+09, min = 699330, max = 2.95246e+09', '#   - flop rate = 3.44737 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.05389e+09', '# --------------------------------------------', '# total                 = 8.05389e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.55917e-11\trel.res =  1.09543e-15\tbw.error =  2.50926e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0405071', '#   - total flops = 2.07556e+07, min = 27135, max = 6.90876e+06', '#   - flop rate = 0.512394 GFlop/s', '#   - bytes moved = 93.8099 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.31589 GByte/s', '#   - solve arithmetic intensity = 0.221252 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    4.570s; elapsed,    2.535s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    5.270s; elapsed,    2.532s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    5.450s; elapsed,    2.537sindividual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    8.080s; elapsed,    2.530s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    7.050s; elapsed,    2.537s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    4.480s; elapsed,    2.536s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    9.790s; elapsed,    2.535s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    9.950s; elapsed,    2.541s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    5.270s; elapsed,    2.532s, #calls:   1', 'TOTAL                                  : CPU,    5.270s; elapsed,    2.532s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    4.570s; elapsed,    2.535s, #calls:   1', 'TOTAL                                  : CPU,    4.570s; elapsed,    2.535s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    5.450s; elapsed,    2.537s, #calls:   1', 'TOTAL                                  : CPU,    5.450s; elapsed,    2.537s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    8.080s; elapsed,    2.530s, #calls:   1', 'TOTAL                                  : CPU,    8.080s; elapsed,    2.530s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    4.480s; elapsed,    2.536s, #calls:   1', 'TOTAL                                  : CPU,    4.480s; elapsed,    2.536s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    9.790s; elapsed,    2.535s, #calls:   1', 'TOTAL                                  : CPU,    9.790s; elapsed,    2.535s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    7.050s; elapsed,    2.537s, #calls:   1', 'TOTAL                                  : CPU,    7.050s; elapsed,    2.537s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   11.610s; elapsed,    2.911s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.110s; elapsed,    0.053s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.150s; elapsed,    2.638s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    9.950s; elapsed,    2.541s, #calls:   1', 'TOTAL                                  : CPU,   24.820s; elapsed,    8.143s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 1331.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    5.950s; elapsed,    5.950s', 'individual call time for EIGEN_LDLT: CPU,    2.530s; elapsed,    2.529s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.060s; elapsed,    0.054s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,953', '#   - number of levels = 125', '#   - nd time = 0.176164', '#   - matching time = 0.0321488', '#   - symmetrization time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.016413', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,953', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0375569', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.1334 MB', '#   - factor time = 2.02451', '#   - factor nonzeros = 6,141,669', '#   - factor memory = 49.1334 MB', '#   - total flops = 7.46606e+09, min = 582404, max = 2.95054e+09', '#   - flop rate = 3.68784 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.98609e+09', '# --------------------------------------------', '# total                 = 7.98609e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.31462e-11\trel.res =  9.90751e-16\tbw.error =  2.15469e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0347619', '#   - total flops = 3.13629e+07, min = 19616, max = 6.85515e+06', '#   - flop rate = 0.902221 GFlop/s', '#   - bytes moved = 90.1412 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.5931 GByte/s', '#   - solve arithmetic intensity = 0.347931 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU,    2.290s; elapsed,    2.334s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    2.330s; elapsed,    2.335s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    2.280s; elapsed,    2.334s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU,    2.320s; elapsed,    2.333s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    2.310s; elapsed,    2.333s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU,    2.280s; elapsed,    2.335s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU,    2.310s; elapsed,    2.333s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    2.280s; elapsed,    2.334s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU,    2.250s; elapsed,    2.334s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU,    2.250s; elapsed,    2.330s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    2.320s; elapsed,    2.333s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    2.300s; elapsed,    2.334s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    2.300s; elapsed,    2.334s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU,    2.280s; elapsed,    2.333s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU,    2.290s; elapsed,    2.328s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    2.330s; elapsed,    2.340s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=9: CPU,    2.280s; elapsed,    2.335s, #calls:   1', 'TOTAL                                  : CPU,    2.280s; elapsed,    2.335s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    2.330s; elapsed,    2.335s, #calls:   1', 'TOTAL                                  : CPU,    2.330s; elapsed,    2.335s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    2.310s; elapsed,    2.333s, #calls:   1', 'TOTAL                                  : CPU,    2.310s; elapsed,    2.333s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=13: CPU,    2.320s; elapsed,    2.333s, #calls:   1', 'TOTAL                                  : CPU,    2.320s; elapsed,    2.333s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=15: CPU,    2.290s; elapsed,    2.334s, #calls:   1', 'TOTAL                                  : CPU,    2.290s; elapsed,    2.334s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    2.280s; elapsed,    2.334s, #calls:   1', 'TOTAL                                  : CPU,    2.280s; elapsed,    2.334s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=14: CPU,    2.310s; elapsed,    2.333s, #calls:   1', 'TOTAL                                  : CPU,    2.310s; elapsed,    2.333s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    2.320s; elapsed,    2.333s, #calls:   1', 'TOTAL                                  : CPU,    2.320s; elapsed,    2.333s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=8: CPU,    2.280s; elapsed,    2.333s, #calls:   1', 'TOTAL                                  : CPU,    2.280s; elapsed,    2.333s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    2.280s; elapsed,    2.334s, #calls:   1', 'TOTAL                                  : CPU,    2.280s; elapsed,    2.334s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    2.300s; elapsed,    2.334s, #calls:   1', 'TOTAL                                  : CPU,    2.300s; elapsed,    2.334s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=10: CPU,    2.250s; elapsed,    2.334s, #calls:   1', 'TOTAL                                  : CPU,    2.250s; elapsed,    2.334s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    2.300s; elapsed,    2.334s, #calls:   1', 'TOTAL                                  : CPU,    2.300s; elapsed,    2.334s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=11: CPU,    2.250s; elapsed,    2.330s, #calls:   1', 'TOTAL                                  : CPU,    2.250s; elapsed,    2.330s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=12: CPU,    2.290s; elapsed,    2.328s, #calls:   1', 'TOTAL                                  : CPU,    2.290s; elapsed,    2.328s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    5.950s; elapsed,    5.950s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.060s; elapsed,    0.054s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.530s; elapsed,    2.529s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    2.330s; elapsed,    2.340s, #calls:   1', 'TOTAL                                  : CPU,   10.870s; elapsed,   10.873s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 1331.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    7.840s; elapsed,    3.934s', 'individual call time for EIGEN_LDLT: CPU,    5.270s; elapsed,    5.094s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.080s; elapsed,    0.056s', 'CPP -2', '# Initializing STRUMPACK', '# using 2CPP -2', ' OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2CPP -2', 'CPP -2CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '', '', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,953', '#   - number of levels = 125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.20229', '#   - matching time = 0.0385211', '#   - symmetrization time = 0.0163429', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,953', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.044646', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.1334 MB', '#   - factor time = 3.59134', '#   - factor nonzeros = 6,141,669', '#   - factor memory = 49.1334 MB', '#   - total flops = 7.46607e+09, min = 582404, max = 2.95054e+09', '#   - flop rate = 2.07891 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.98609e+09', '# --------------------------------------------', '# total                 = 7.98609e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.31206e-11\trel.res =  9.89653e-16\tbw.error =  2.15469e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0408621', '#   - total flops = 3.13629e+07, min = 19616, max = 6.85515e+06', '#   - flop rate = 0.767531 GFlop/s', '#   - bytes moved = 90.1488 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.20617 GByte/s', '#   - solve arithmetic intensity = 0.347902 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    4.680s; elapsed,    3.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU,    4.860s; elapsed,    3.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU,    5.100s; elapsed,    3.957s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    6.690s; elapsed,    3.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    4.950s; elapsed,    3.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU,    4.840s; elapsed,    3.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    4.640s; elapsed,    3.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    4.610s; elapsed,    3.954s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    4.580s; elapsed,    3.955s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU,    5.750s; elapsed,    3.951s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    4.500s; elapsed,    3.955s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU,    5.550s; elapsed,    3.956s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU,    4.920s; elapsed,    3.955s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU,    5.150s; elapsed,    3.951s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU,    5.170s; elapsed,    3.956s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    7.840s; elapsed,    3.961s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    4.640s; elapsed,    3.954s, #calls:   1', 'TOTAL                                  : CPU,    4.640s; elapsed,    3.954s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    6.690s; elapsed,    3.954s, #calls:   1', 'TOTAL                                  : CPU,    6.690s; elapsed,    3.954s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=14: CPU,    5.550s; elapsed,    3.956s, #calls:   1', 'TOTAL                                  : CPU,    5.550s; elapsed,    3.956s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=13: CPU,    5.100s; elapsed,    3.957s, #calls:   1', 'TOTAL                                  : CPU,    5.100s; elapsed,    3.957s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=9: CPU,    4.860s; elapsed,    3.954s, #calls:   1', 'TOTAL                                  : CPU,    4.860s; elapsed,    3.954s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    4.580s; elapsed,    3.955s, #calls:   1', 'TOTAL                                  : CPU,    4.580s; elapsed,    3.955s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    4.950s; elapsed,    3.954s, #calls:   1', 'TOTAL                                  : CPU,    4.950s; elapsed,    3.954s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=11: CPU,    5.750s; elapsed,    3.951s, #calls:   1', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    4.610s; elapsed,    3.954s, #calls:   1', 'TOTAL                                  : CPU,    4.610s; elapsed,    3.954s', 'TOTAL                                  : CPU,    5.750s; elapsed,    3.951s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    4.680s; elapsed,    3.954s, #calls:   1', 'TOTAL                                  : CPU,    4.680s; elapsed,    3.954s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    4.500s; elapsed,    3.955s, #calls:   1', 'TOTAL                                  : CPU,    4.500s; elapsed,    3.955s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=10: CPU,    4.840s; elapsed,    3.954s, #calls:   1', 'TOTAL                                  : CPU,    4.840s; elapsed,    3.954s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=12: CPU,    5.150s; elapsed,    3.951s, #calls:   1', 'TOTAL                                  : CPU,    5.150s; elapsed,    3.951s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=15: CPU,    5.170s; elapsed,    3.956s, #calls:   1', 'TOTAL                                  : CPU,    5.170s; elapsed,    3.956s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=8: CPU,    4.920s; elapsed,    3.955s, #calls:   1', 'TOTAL                                  : CPU,    4.920s; elapsed,    3.955s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    7.840s; elapsed,    3.934s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.080s; elapsed,    0.056s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    5.270s; elapsed,    5.094s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    7.840s; elapsed,    3.961s, #calls:   1', 'TOTAL                                  : CPU,   21.030s; elapsed,   13.045s']
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Bfactor-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Bfactor-/A_strum_5k_omp1_paramslevmar.parameter_flags=Bfactor-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Bfactor-/b_strum_5k_omp1_paramslevmar.parameter_flags=Bfactor-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   18.170s; elapsed,   18.179s', 'individual call time for EIGEN_LDLT: CPU,   18.950s; elapsed,   18.960s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.160s; elapsed,    0.167s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   18.170s; elapsed,   18.179s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.160s; elapsed,    0.167s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   18.950s; elapsed,   18.960s, #calls:   1', 'TOTAL                                  : CPU,   37.280s; elapsed,   37.307s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   21.090s; elapsed,   10.559s', 'individual call time for EIGEN_LDLT: CPU,   18.850s; elapsed,   18.686s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.250s; elapsed,    0.151s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   21.090s; elapsed,   10.559s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.250s; elapsed,    0.151s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   18.850s; elapsed,   18.686s, #calls:   1', 'TOTAL                                  : CPU,   40.190s; elapsed,   29.396s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   43.330s; elapsed,   10.849s', 'individual call time for EIGEN_LDLT: CPU,   20.710s; elapsed,   20.170s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.270s; elapsed,    0.105s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   43.330s; elapsed,   10.849s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.270s; elapsed,    0.105s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   20.710s; elapsed,   20.170s, #calls:   1', 'TOTAL                                  : CPU,   64.310s; elapsed,   31.124s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   65.200s; elapsed,    8.179s', 'individual call time for EIGEN_LDLT: CPU,   20.040s; elapsed,   18.752s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.430s; elapsed,    0.094s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   65.200s; elapsed,    8.179s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.430s; elapsed,    0.094s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   20.040s; elapsed,   18.752s, #calls:   1', 'TOTAL                                  : CPU,   85.670s; elapsed,   27.025s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,  140.140s; elapsed,    8.804s', 'individual call time for EIGEN_LDLT: CPU,   21.870s; elapsed,   19.155s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.790s; elapsed,    0.095s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,  140.140s; elapsed,    8.804s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.790s; elapsed,    0.095s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   21.870s; elapsed,   19.155s, #calls:   1', 'TOTAL                                  : CPU,  162.800s; elapsed,   28.054s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   19.130s; elapsed,   19.160s', 'individual call time for EIGEN_LDLT: CPU,   18.960s; elapsed,   18.984s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.150s; elapsed,    0.160s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 487,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,121', '#   - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.163049', '#   - matching time = 0.052469', '#   - symmetrization time = 0.0295382', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,121', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.069936', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 169.261 MB', '#   - factor time = 10.8386', '#   - factor nonzeros = 21,157,571', '#   - factor memory = 169.261 MB', '#   - total flops = 5.06936e+10, min = 2.21744e+10, max = 2.85192e+10', '#   - flop rate = 4.67714 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.21012e+10', '# --------------------------------------------', '# total                 = 5.21012e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  4.85836e-11\trel.res =  1.98671e-15\tbw.error =  5.28691e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0650389', '#   - total flops = 4.62998e+07, min = 1.70328e+07, max = 2.9267e+07', '#   - flop rate = 0.711879 GFlop/s', '#   - bytes moved = 199.391 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 3.06573 GByte/s', '#   - solve arithmetic intensity = 0.232206 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   11.230s; elapsed,   11.233s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   11.240s; elapsed,   11.251s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   11.230s; elapsed,   11.233s, #calls:   1', 'TOTAL                                  : CPU,   11.230s; elapsed,   11.233s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   19.130s; elapsed,   19.160s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.150s; elapsed,    0.160s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   18.960s; elapsed,   18.984s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   11.240s; elapsed,   11.251s, #calls:   1', 'TOTAL                                  : CPU,   49.480s; elapsed,   49.556s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   21.130s; elapsed,   10.588s', 'individual call time for EIGEN_LDLT: CPU,   19.020s; elapsed,   18.857s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.230s; elapsed,    0.131s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 487,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,121', '#   - number of levels = 68', '#   - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.161264', '#   - matching time = 0.0531778', '#   - symmetrization time = 0.0313871', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,121', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0867958', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 169.261 MB', '#   - factor time = 8.11767', '#   - factor nonzeros = 21,157,571', '#   - factor memory = 169.261 MB', '#   - total flops = 5.06937e+10, min = 2.21745e+10, max = 2.85192e+10', '#   - flop rate = 6.24486 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.21012e+10', '# --------------------------------------------', '# total                 = 5.21012e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  4.89278e-11\trel.res =  2.00078e-15\tbw.error =  5.28761e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0683291', '#   - total flops = 4.631e+07, min = 1.7043e+07, max = 2.9267e+07', '#   - flop rate = 0.677749 GFlop/s', '#   - bytes moved = 199.824 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.92444 GByte/s', '#   - solve arithmetic intensity = 0.231754 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   15.050s; elapsed,    8.537s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   17.020s; elapsed,    8.556s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   15.050s; elapsed,    8.537s, #calls:   1', 'TOTAL                                  : CPU,   15.050s; elapsed,    8.537s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   21.130s; elapsed,   10.588s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.230s; elapsed,    0.131s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   19.020s; elapsed,   18.857s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   17.020s; elapsed,    8.556s, #calls:   1', 'TOTAL                                  : CPU,   57.400s; elapsed,   38.132s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   41.850s; elapsed,   10.474s', 'individual call time for EIGEN_LDLT: CPU,   51.090s; elapsed,   50.552s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.330s; elapsed,    0.120s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 487,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,121', '#   - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.170684', '#   - matching time = 0.054136', '#   - symmetrization time = 0.0304191', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,121', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.080976', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 169.261 MB', '#   - factor time = 7.24615', '#   - factor nonzeros = 21,157,571', '#   - factor memory = 169.261 MB', '#   - total flops = 5.06937e+10, min = 2.21745e+10, max = 2.85192e+10', '#   - flop rate = 6.99595 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.21012e+10', '# --------------------------------------------', '# total                 = 5.21012e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  4.88728e-11\trel.res =  1.99853e-15\tbw.error =  5.28761e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0733261', '#   - total flops = 4.631e+07, min = 1.7043e+07, max = 2.9267e+07', '#   - flop rate = 0.631562 GFlop/s', '#   - bytes moved = 199.843 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.7254 GByte/s', '#   - solve arithmetic intensity = 0.231732 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   25.400s; elapsed,    7.673s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   30.550s; elapsed,    7.690s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   25.400s; elapsed,    7.673s, #calls:   1', 'TOTAL                                  : CPU,   25.400s; elapsed,    7.673s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   41.850s; elapsed,   10.474s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.330s; elapsed,    0.120s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   51.090s; elapsed,   50.552s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   30.550s; elapsed,    7.690s, #calls:   1', 'TOTAL                                  : CPU,  123.820s; elapsed,   68.837s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   64.410s; elapsed,    8.073s', 'individual call time for EIGEN_LDLT: CPU,   20.740s; elapsed,   19.487s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.430s; elapsed,    0.110s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = CPP -2', '6 = log_2(#threads) + 3', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 487,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,121', '#   - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.17955', '#   - matching time = 0.0581422', '#   - symmetrization time = 0.0359781', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,121', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0912349', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 169.261 MB', '#   - factor time = 8.43808', '#   - factor nonzeros = 21,157,571', '#   - factor memory = 169.261 MB', '#   - total flops = 5.06937e+10, min = 2.21745e+10, max = 2.85192e+10', '#   - flop rate = 6.00773 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.21012e+10', '# --------------------------------------------', '# total                 = 5.21012e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  4.88477e-11\trel.res =  1.99751e-15\tbw.error =  5.28761e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0835271', '#   - total flops = 4.631e+07, min = 1.7043e+07, max = 2.9267e+07', '#   - flop rate = 0.554431 GFlop/s', '#   - bytes moved = 199.881 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.39301 GByte/s', '#   - solve arithmetic intensity = 0.231688 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   54.290s; elapsed,    8.911s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   70.590s; elapsed,    8.926s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   54.290s; elapsed,    8.911s, #calls:   1', 'TOTAL                                  : CPU,   54.290s; elapsed,    8.911s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   64.410s; elapsed,    8.073s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.430s; elapsed,    0.110s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   20.740s; elapsed,   19.487s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   70.590s; elapsed,    8.926s, #calls:   1', 'TOTAL                                  : CPU,  156.170s; elapsed,   36.596s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,  139.560s; elapsed,    8.753s', 'individual call time for EIGEN_LDLT: CPU,   37.630s; elapsed,   34.892s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.680s; elapsed,    0.093s', 'CPP -2', 'CPP -2# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', '', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 487,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,121', '#   - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.205065', '#   - matching time = 0.104195', '#   - symmetrization time = 0.04724', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,121', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.097409', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 169.261 MB', '#   - factor time = 8.62634', '#   - factor nonzeros = 21,157,571', '#   - factor memory = 169.261 MB', '#   - total flops = 5.06943e+10, min = 2.21751e+10, max = 2.85192e+10', '#   - flop rate = 5.87669 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.21012e+10', '# --------------------------------------------', '# total                 = 5.21012e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  4.88318e-11\trel.res =  1.99685e-15\tbw.error =  5.28757e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.10304', '#   - total flops = 4.63137e+07, min = 1.70467e+07, max = 2.9267e+07', '#   - flop rate = 0.449474 GFlop/s', '#   - bytes moved = 200.018 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 1.94117 GByte/s', '#   - solve arithmetic intensity = 0.231548 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,  109.000s; elapsed,    9.209s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,  144.330s; elapsed,    9.229s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,  109.000s; elapsed,    9.209s, #calls:   1', 'TOTAL                                  : CPU,  109.000s; elapsed,    9.209s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,  139.560s; elapsed,    8.753s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.680s; elapsed,    0.093s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   37.630s; elapsed,   34.892s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,  144.330s; elapsed,    9.229s, #calls:   1', 'TOTAL                                  : CPU,  322.200s; elapsed,   52.968s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   19.630s; elapsed,   19.640s', 'individual call time for EIGEN_LDLT: CPU,   19.440s; elapsed,   19.452s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.180s; elapsed,    0.185s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 487,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,131', '#   - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.168702', '#   - matching time = 0.0535901', '#   - symmetrization time = 0.0412691', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,131', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0673561', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 170.719 MB', '#   - factor time = 7.78362', '#   - factor nonzeros = 21,339,853', '#   - factor memory = 170.719 MB', '#   - total flops = 5.15107e+10, min = 1.48212e+07, max = 2.85626e+10', '#   - flop rate = 6.61783 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.2862e+10', '# --------------------------------------------', '# total                 = 5.2862e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.69603e-11\trel.res =   1.5114e-15\tbw.error =  4.67637e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.079797', '#   - total flops = 5.12613e+07, min = 105197, max = 2.92123e+07', '#   - flop rate = 0.642397 GFlop/s', '#   - bytes moved = 180.23 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.2586 GByte/s', '#   - solve arithmetic intensity = 0.284422 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    8.220s; elapsed,    8.218s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    8.210s; elapsed,    8.217s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    8.190s; elapsed,    8.203s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    8.220s; elapsed,    8.222s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    8.220s; elapsed,    8.218s, #calls:   1', 'TOTAL                                  : CPU,    8.220s; elapsed,    8.218s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    8.210s; elapsed,    8.217s, #calls:   1', 'TOTAL                                  : CPU,    8.210s; elapsed,    8.217s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    8.190s; elapsed,    8.203s, #calls:   1', 'TOTAL                                  : CPU,    8.190s; elapsed,    8.203s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   19.630s; elapsed,   19.640s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.180s; elapsed,    0.185s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   19.440s; elapsed,   19.452s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    8.220s; elapsed,    8.222s, #calls:   1', 'TOTAL                                  : CPU,   47.470s; elapsed,   47.498s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   22.560s; elapsed,   11.301s', 'individual call time for EIGEN_LDLT: CPU,   19.320s; elapsed,   19.146s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.250s; elapsed,    0.150s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 487,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,131', '#   - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.177807', '#   - matching time = 0.056633', '#   - symmetrization time = 0.041754', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,131', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0772901', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 170.719 MB', '#   - factor time = 7.72084', '#   - factor nonzeros = 21,339,853', '#   - factor memory = 170.719 MB', '#   - total flops = 5.15107e+10, min = 1.48212e+07, max = 2.85626e+10', '#   - flop rate = 6.67165 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.2862e+10', '# --------------------------------------------', '# total                 = 5.2862e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.69771e-11\trel.res =  1.51208e-15\tbw.error =  4.67141e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.08533', '#   - total flops = 5.12668e+07, min = 105197, max = 2.92123e+07', '#   - flop rate = 0.600807 GFlop/s', '#   - bytes moved = 180.446 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.11468 GByte/s', '#   - solve arithmetic intensity = 0.284112 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    9.540s; elapsed,    8.183s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,   14.060s; elapsed,    8.182s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,   12.650s; elapsed,    8.170s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   16.110s; elapsed,    8.188s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    9.540s; elapsed,    8.183s, #calls:   1', 'TOTAL                                  : CPU,    9.540s; elapsed,    8.183s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,   12.650s; elapsed,    8.170s, #calls:   1', 'TOTAL                                  : CPU,   12.650s; elapsed,    8.170s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,   14.060s; elapsed,    8.182s, #calls:   1', 'TOTAL                                  : CPU,   14.060s; elapsed,    8.182s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   22.560s; elapsed,   11.301s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.250s; elapsed,    0.150s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   19.320s; elapsed,   19.146s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   16.110s; elapsed,    8.188s, #calls:   1', 'TOTAL                                  : CPU,   58.240s; elapsed,   38.785s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   34.180s; elapsed,    8.582s', 'individual call time for EIGEN_LDLT: CPU,   31.470s; elapsed,   30.940s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.320s; elapsed,    0.121s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 487,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,131', '#   - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.202503', '#   - matching time = 0.0815771', '#   - symmetrization time = 0.04374', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,131', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0870862', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 170.719 MB', '#   - factor time = 7.7136', '#   - factor nonzeros = 21,339,853', '#   - factor memory = 170.719 MB', '#   - total flops = 5.15107e+10, min = 1.48226e+07, max = 2.85626e+10', '#   - flop rate = 6.67791 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.2862e+10', '# --------------------------------------------', '# total                 = 5.2862e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.69687e-11\trel.res =  1.51174e-15\tbw.error =  4.67141e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0909681', '#   - total flops = 5.12668e+07, min = 105197, max = 2.92123e+07', '#   - flop rate = 0.563569 GFlop/s', '#   - bytes moved = 180.456 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 1.98373 GByte/s', '#   - solve arithmetic intensity = 0.284096 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   12.260s; elapsed,    8.247s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,   26.040s; elapsed,    8.247s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,   21.320s; elapsed,    8.234s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   32.300s; elapsed,    8.247s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   12.260s; elapsed,    8.247s, #calls:   1', 'TOTAL                                  : CPU,   12.260s; elapsed,    8.247s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,   21.320s; elapsed,    8.234s, #calls:   1', 'TOTAL                                  : CPU,   21.320s; elapsed,    8.234s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,   26.040s; elapsed,    8.247s, #calls:   1', 'TOTAL                                  : CPU,   26.040s; elapsed,    8.247s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   34.180s; elapsed,    8.582s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.320s; elapsed,    0.121s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   31.470s; elapsed,   30.940s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   32.300s; elapsed,    8.247s, #calls:   1', 'TOTAL                                  : CPU,   98.270s; elapsed,   47.891s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   69.120s; elapsed,    8.667s', 'individual call time for EIGEN_LDLT: CPU,   20.650s; elapsed,   19.369s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.420s; elapsed,    0.103s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 487,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,131', '#   - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.191953', '#   - matching time = 0.0693421', '#   - symmetrization time = 0.0466621', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,131', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.097029', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 170.719 MB', '#   - factor time = 8.21027', '#   - factor nonzeros = 21,339,853', '#   - factor memory = 170.719 MB', '#   - total flops = 5.15107e+10, min = 1.48228e+07, max = 2.85626e+10', '#   - flop rate = 6.27394 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.2862e+10', '# --------------------------------------------', '# total                 = 5.2862e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =    3.694e-11\trel.res =  1.51057e-15\tbw.error =  4.67141e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.107361', '#   - total flops = 5.12668e+07, min = 105197, max = 2.92123e+07', '#   - flop rate = 0.477518 GFlop/s', '#   - bytes moved = 180.478 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 1.68104 GByte/s', '#   - solve arithmetic intensity = 0.284062 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,   49.340s; elapsed,    8.750s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,   35.760s; elapsed,    8.733s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   17.810s; elapsed,    8.749s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   68.220s; elapsed,    8.756s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,   49.340s; elapsed,    8.750s, #calls:   1', 'TOTAL                                  : CPU,   49.340s; elapsed,    8.750s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   17.810s; elapsed,    8.749s, #calls:   1', 'TOTAL                                  : CPU,   17.810s; elapsed,    8.749s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,   35.760s; elapsed,    8.733s, #calls:   1', 'TOTAL                                  : CPU,   35.760s; elapsed,    8.733s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   69.120s; elapsed,    8.667s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.420s; elapsed,    0.103s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   20.650s; elapsed,   19.369s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   68.220s; elapsed,    8.756s, #calls:   1', 'TOTAL                                  : CPU,  158.410s; elapsed,   36.895s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 2413.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   20.180s; elapsed,   20.203s', 'individual call time for EIGEN_LDLT: CPU,   20.090s; elapsed,   20.102s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.200s; elapsed,    0.192s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 487,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,143', '#   - number of levels = 66', '#   - nd time = 0.19664', '#   - matching time = 0.0616531', '#   - symmetrization time = 0.029217', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,143', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0707021', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 168.522 MB', '#   - factor time = 8.70793', '#   - factor nonzeros = 21,065,199', '#   - factor memory = 168.522 MB', '#   - total flops = 5.03174e+10, min = 1.50222e+07, max = 2.85789e+10', '#   - flop rate = 5.77834 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.15536e+10', '# --------------------------------------------', '# total                 = 5.15536e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  4.42119e-11\trel.res =  1.80794e-15\tbw.error =   4.5859e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0707011', '#   - total flops = 6.23363e+07, min = 92713, max = 2.90586e+07', '#   - flop rate = 0.881688 GFlop/s', '#   - bytes moved = 166.966 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.36157 GByte/s', '#   - solve arithmetic intensity = 0.373348 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    9.120s; elapsed,    9.153s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    9.120s; elapsed,    9.153s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    9.150s; elapsed,    9.156s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    9.130s; elapsed,    9.153s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    9.120s; elapsed,    9.143s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    9.140s; elapsed,    9.155s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    9.130s; elapsed,    9.149s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    9.150s; elapsed,    9.159s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    9.130s; elapsed,    9.153s, #calls:   1', 'TOTAL                                  : CPU,    9.130s; elapsed,    9.153s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    9.120s; elapsed,    9.143s, #calls:   1', 'TOTAL                                  : CPU,    9.120s; elapsed,    9.143s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    9.120s; elapsed,    9.153s, #calls:   1', 'TOTAL                                  : CPU,    9.120s; elapsed,    9.153s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    9.120s; elapsed,    9.153s, #calls:   1', 'TOTAL                                  : CPU,    9.120s; elapsed,    9.153s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    9.130s; elapsed,    9.149s, #calls:   1', 'TOTAL                                  : CPU,    9.130s; elapsed,    9.149s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    9.140s; elapsed,    9.155s, #calls:   1', 'TOTAL                                  : CPU,    9.140s; elapsed,    9.155s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    9.150s; elapsed,    9.156s, #calls:   1', 'TOTAL                                  : CPU,    9.150s; elapsed,    9.156s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   20.180s; elapsed,   20.203s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.200s; elapsed,    0.192s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   20.090s; elapsed,   20.102s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    9.150s; elapsed,    9.159s, #calls:   1', 'TOTAL                                  : CPU,   49.620s; elapsed,   49.657s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 2413.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   22.470s; elapsed,   11.260s', 'individual call time for EIGEN_LDLT: CPU,   20.050s; elapsed,   19.876s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.210s; elapsed,    0.129s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 487,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,143', '#   - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.177551', '#   - matching time = 0.0605941', '#   - symmetrization time = 0.0263829', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,143', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0786829', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 168.522 MB', '#   - factor time = 7.7616', '#   - factor nonzeros = 21,065,199', '#   - factor memory = 168.522 MB', '#   - total flops = 5.03174e+10, min = 1.50222e+07, max = 2.85789e+10', '#   - flop rate = 6.48287 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.15536e+10', '# --------------------------------------------', '# total                 = 5.15536e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  4.40836e-11\trel.res =  1.80269e-15\tbw.error =  4.58556e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0617771', '#   - total flops = 6.23363e+07, min = 92713, max = 2.90586e+07', '#   - flop rate = 1.00905 GFlop/s', '#   - bytes moved = 166.973 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.70282 GByte/s', '#   - solve arithmetic intensity = 0.373333 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    9.640s; elapsed,    8.185s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    9.880s; elapsed,    8.183s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    9.360s; elapsed,    8.184s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    9.420s; elapsed,    8.184s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,   13.660s; elapsed,    8.183s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,   12.010s; elapsed,    8.179s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,   11.690s; elapsed,    8.175s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   16.130s; elapsed,    8.188s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    9.640s; elapsed,    8.185s, #calls:   1', 'TOTAL                                  : CPU,    9.640s; elapsed,    8.185s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    9.880s; elapsed,    8.183s, #calls:   1', 'TOTAL                                  : CPU,    9.880s; elapsed,    8.183s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    9.420s; elapsed,    8.184s, #calls:   1', 'TOTAL                                  : CPU,    9.420s; elapsed,    8.184s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    9.360s; elapsed,    8.184s, #calls:   1', 'TOTAL                                  : CPU,    9.360s; elapsed,    8.184s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,   12.010s; elapsed,    8.179s, #calls:   1', 'TOTAL                                  : CPU,   12.010s; elapsed,    8.179s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,   13.660s; elapsed,    8.183s, #calls:   1', 'TOTAL                                  : CPU,   13.660s; elapsed,    8.183s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,   11.690s; elapsed,    8.175s, #calls:   1', 'TOTAL                                  : CPU,   11.690s; elapsed,    8.175s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   22.470s; elapsed,   11.260s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.210s; elapsed,    0.129s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   20.050s; elapsed,   19.876s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   16.130s; elapsed,    8.188s, #calls:   1', 'TOTAL                                  : CPU,   58.860s; elapsed,   39.453s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 2413.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   31.790s; elapsed,    7.969s', 'individual call time for EIGEN_LDLT: CPU,   20.310s; elapsed,   19.784s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.340s; elapsed,    0.117s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 487,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,143', '#   - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.208347', '#   - matching time = 0.065809', '#   - symmetrization time = 0.0293422', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,143', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0798979', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 168.522 MB', '#   - factor time = 7.94199', '#   - factor nonzeros = 21,065,199', '#   - factor memory = 168.522 MB', '#   - total flops = 5.03174e+10, min = 1.50234e+07, max = 2.85789e+10', '#   - flop rate = 6.33562 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.15536e+10', '# --------------------------------------------', '# total                 = 5.15536e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  4.40619e-11\trel.res =   1.8018e-15\tbw.error =  4.58422e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0710061', '#   - total flops = 6.23363e+07, min = 92713, max = 2.90586e+07', '#   - flop rate = 0.877902 GFlop/s', '#   - bytes moved = 166.981 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.35165 GByte/s', '#   - solve arithmetic intensity = 0.373313 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,   20.540s; elapsed,    8.409s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,   17.540s; elapsed,    8.407s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   13.020s; elapsed,    8.415s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,   12.400s; elapsed,    8.415s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,   12.110s; elapsed,    8.414s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,   13.590s; elapsed,    8.415s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,   25.060s; elapsed,    8.415s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   32.960s; elapsed,    8.420s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,   20.540s; elapsed,    8.409s, #calls:   1', 'TOTAL                                  : CPU,   20.540s; elapsed,    8.409s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,   17.540s; elapsed,    8.407s, #calls:   1', 'TOTAL                                  : CPU,   17.540s; elapsed,    8.407s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,   13.590s; elapsed,    8.415s, #calls:   1', 'TOTAL                                  : CPU,   13.590s; elapsed,    8.415s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,   12.110s; elapsed,    8.414s, #calls:   1', 'TOTAL                                  : CPU,   12.110s; elapsed,    8.414s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   13.020s; elapsed,    8.415s, #calls:   1', 'TOTAL                                  : CPU,   13.020s; elapsed,    8.415s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,   25.060s; elapsed,    8.415s, #calls:   1', 'TOTAL                                  : CPU,   25.060s; elapsed,    8.415s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,   12.400s; elapsed,    8.415s, #calls:   1', 'TOTAL                                  : CPU,   12.400s; elapsed,    8.415s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   31.790s; elapsed,    7.969s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.340s; elapsed,    0.117s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   20.310s; elapsed,   19.784s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   32.960s; elapsed,    8.420s, #calls:   1', 'TOTAL                                  : CPU,   85.400s; elapsed,   36.290s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 1206.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   19.490s; elapsed,   19.505s', 'individual call time for EIGEN_LDLT: CPU,   19.820s; elapsed,   19.835s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.180s; elapsed,    0.185s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 487,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,143', '#   - number of levels = 65', '#   - nd time = 0.174265', '#   - matching time = 0.0576799', '#   - symmetrization time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.0151649', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,143', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0594389', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 168.522 MB', '#   - factor time = 7.76313', '#   - factor nonzeros = 21,065,199', '#   - factor memory = 168.522 MB', '#   - total flops = 5.03174e+10, min = 1.38671e+07, max = 2.85337e+10', '#   - flop rate = 6.48159 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.13241e+10', '# --------------------------------------------', '# total                 = 5.13241e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.22003e-11\trel.res =  1.31675e-15\tbw.error =  3.37627e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.043978', '#   - total flops = 9.15661e+07, min = 86678, max = 2.87842e+07', '#   - flop rate = 2.08209 GFlop/s', '#   - bytes moved = 158.412 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 3.60207 GByte/s', '#   - solve arithmetic intensity = 0.578027 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    8.070s; elapsed,    8.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU,    8.090s; elapsed,    8.122s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    8.110s; elapsed,    8.126s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    8.070s; elapsed,    8.126s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    8.080s; elapsed,    8.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    8.100s; elapsed,    8.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU,    8.120s; elapsed,    8.126s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU,    8.110s; elapsed,    8.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU,    8.110s; elapsed,    8.126s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    8.080s; elapsed,    8.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    8.100s; elapsed,    8.127s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU,    8.070s; elapsed,    8.131s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU,    8.070s; elapsed,    8.129s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU,    8.070s; elapsed,    8.128s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU,    8.090s; elapsed,    8.126s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    8.140s; elapsed,    8.136s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=14: CPU,    8.070s; elapsed,    8.128s, #calls:   1', 'TOTAL                                  : CPU,    8.070s; elapsed,    8.128s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    8.070s; elapsed,    8.127s, #calls:   1', 'TOTAL                                  : CPU,    8.070s; elapsed,    8.127s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=13: CPU,    8.090s; elapsed,    8.122s, #calls:   1', 'TOTAL                                  : CPU,    8.090s; elapsed,    8.122s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    8.070s; elapsed,    8.126s, #calls:   1', 'TOTAL                                  : CPU,    8.070s; elapsed,    8.126s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    8.100s; elapsed,    8.127s, #calls:   1', 'TOTAL                                  : CPU,    8.100s; elapsed,    8.127s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    8.080s; elapsed,    8.127s, #calls:   1', 'Exiting profiler', 'TOTAL                                  : CPU,    8.080s; elapsed,    8.127s', 'time for     STRUMPACK_MPI_DIST_RANK=10: CPU,    8.110s; elapsed,    8.127s, #calls:   1', 'TOTAL                                  : CPU,    8.110s; elapsed,    8.127s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    8.100s; elapsed,    8.127s, #calls:   1', 'TOTAL                                  : CPU,    8.100s; elapsed,    8.127s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=11: CPU,    8.110s; elapsed,    8.126s, #calls:   1', 'TOTAL                                  : CPU,    8.110s; elapsed,    8.126s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    8.080s; elapsed,    8.127s, #calls:   1', 'TOTAL                                  : CPU,    8.080s; elapsed,    8.127s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    8.110s; elapsed,    8.126s, #calls:   1', 'TOTAL                                  : CPU,    8.110s; elapsed,    8.126s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=9: CPU,    8.120s; elapsed,    8.126s, #calls:   1', 'TOTAL                                  : CPU,    8.120s; elapsed,    8.126s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=8: CPU,    8.070s; elapsed,    8.131s, #calls:   1', 'TOTAL                                  : CPU,    8.070s; elapsed,    8.131s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=15: CPU,    8.090s; elapsed,    8.126s, #calls:   1', 'TOTAL                                  : CPU,    8.090s; elapsed,    8.126s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=12: CPU,    8.070s; elapsed,    8.129s, #calls:   1', 'TOTAL                                  : CPU,    8.070s; elapsed,    8.129s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   19.490s; elapsed,   19.505s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.180s; elapsed,    0.185s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   19.820s; elapsed,   19.835s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    8.140s; elapsed,    8.136s, #calls:   1', 'TOTAL                                  : CPU,   47.630s; elapsed,   47.662s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 1206.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   23.220s; elapsed,   11.636s', 'individual call time for EIGEN_LDLT: CPU,   20.790s; elapsed,   20.616s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.280s; elapsed,    0.164s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 487,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,143', '#   - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.203531', '#   - matching time = 0.067708', '#   - symmetrization time = 0.015831', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,143', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0727611', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 168.522 MB', '#   - factor time = 7.91772', '#   - factor nonzeros = 21,065,199', '#   - factor memory = 168.522 MB', '#   - total flops = 5.03174e+10, min = 1.38671e+07, max = 2.85337e+10', '#   - flop rate = 6.35504 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.13241e+10', '# --------------------------------------------', '# total                 = 5.13241e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.21233e-11\trel.res =   1.3136e-15\tbw.error =  3.37627e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0509582', '#   - total flops = 9.15661e+07, min = 86678, max = 2.87842e+07', '#   - flop rate = 1.79689 GFlop/s', '#   - bytes moved = 158.417 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 3.10877 GByte/s', '#   - solve arithmetic intensity = 0.578006 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    9.450s; elapsed,    8.343s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    9.940s; elapsed,    8.344s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,   14.340s; elapsed,    8.342s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU,   10.040s; elapsed,    8.342s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    9.450s; elapsed,    8.342s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU,   10.270s; elapsed,    8.344s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU,    9.870s; elapsed,    8.343s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU,    9.790s; elapsed,    8.343s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU,   12.290s; elapsed,    8.343s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU,   10.220s; elapsed,    8.340s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU,   11.120s; elapsed,    8.341s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU,   11.580s; elapsed,    8.339s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    9.400s; elapsed,    8.344s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    9.770s; elapsed,    8.342s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    9.630s; elapsed,    8.343s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   16.400s; elapsed,    8.349s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=10: CPU,    9.870s; elapsed,    8.343s, #calls:   1', 'TOTAL                                  : CPU,    9.870s; elapsed,    8.343s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    9.450s; elapsed,    8.343s, #calls:   1', 'TOTAL                                  : CPU,    9.450s; elapsed,    8.343s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    9.940s; elapsed,    8.344s, #calls:   1', 'TOTAL                                  : CPU,    9.940s; elapsed,    8.344s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=12: CPU,   12.290s; elapsed,    8.343s, #calls:   1', 'TOTAL                                  : CPU,   12.290s; elapsed,    8.343s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=13: CPU,   10.220s; elapsed,    8.340s, #calls:   1', 'TOTAL                                  : CPU,   10.220s; elapsed,    8.340s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,   14.340s; elapsed,    8.342s, #calls:   1', 'TOTAL                                  : CPU,   14.340s; elapsed,    8.342s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=15: CPU,   11.580s; elapsed,    8.339s, #calls:   1', 'TOTAL                                  : CPU,   11.580s; elapsed,    8.339s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=14: CPU,   11.120s; elapsed,    8.341s, #calls:   1', 'TOTAL                                  : CPU,   11.120s; elapsed,    8.341s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    9.450s; elapsed,    8.342s, #calls:   1', 'TOTAL                                  : CPU,    9.450s; elapsed,    8.342s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=11: CPU,    9.790s; elapsed,    8.343s, #calls:   1', 'TOTAL                                  : CPU,    9.790s; elapsed,    8.343s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    9.770s; elapsed,    8.342s, #calls:   1', 'TOTAL                                  : CPU,    9.770s; elapsed,    8.342s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    9.400s; elapsed,    8.344s, #calls:   1', 'TOTAL                                  : CPU,    9.400s; elapsed,    8.344s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=9: CPU,   10.040s; elapsed,    8.342s, #calls:   1', 'TOTAL                                  : CPU,   10.040s; elapsed,    8.342s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    9.630s; elapsed,    8.343s, #calls:   1', 'TOTAL                                  : CPU,    9.630s; elapsed,    8.343s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=8: CPU,   10.270s; elapsed,    8.344s, #calls:   1', 'TOTAL                                  : CPU,   10.270s; elapsed,    8.344s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   23.220s; elapsed,   11.636s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.280s; elapsed,    0.164s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   20.790s; elapsed,   20.616s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   16.400s; elapsed,    8.349s, #calls:   1', 'TOTAL                                  : CPU,   60.690s; elapsed,   40.765s']
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Deff-/A_strum_5k_omp1_paramslevmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Deff-/b_strum_5k_omp1_paramslevmar.parameter_flags=Deff-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    4.280s; elapsed,    4.277s', 'individual call time for EIGEN_LDLT: CPU,    2.510s; elapsed,    2.506s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.051s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    4.280s; elapsed,    4.277s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.051s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.510s; elapsed,    2.506s, #calls:   1', 'TOTAL                                  : CPU,    6.840s; elapsed,    6.834s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    5.430s; elapsed,    2.735s', 'individual call time for EIGEN_LDLT: CPU,    2.700s; elapsed,    2.509s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.044s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    5.430s; elapsed,    2.735s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.044s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.700s; elapsed,    2.509s, #calls:   1', 'TOTAL                                  : CPU,    8.200s; elapsed,    5.288s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    7.430s; elapsed,    1.892s', 'individual call time for EIGEN_LDLT: CPU,    3.620s; elapsed,    3.076s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.110s; elapsed,    0.043s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    7.430s; elapsed,    1.892s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.110s; elapsed,    0.043s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.620s; elapsed,    3.076s, #calls:   1', 'TOTAL                                  : CPU,   11.160s; elapsed,    5.011s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   14.060s; elapsed,    1.797s', 'individual call time for EIGEN_LDLT: CPU,    4.550s; elapsed,    3.299s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.140s; elapsed,    0.038s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   14.060s; elapsed,    1.797s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.140s; elapsed,    0.038s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    4.550s; elapsed,    3.299s, #calls:   1', 'TOTAL                                  : CPU,   18.750s; elapsed,    5.133s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 19309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   36.060s; elapsed,    2.315s', 'individual call time for EIGEN_LDLT: CPU,    5.510s; elapsed,    2.809s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.280s; elapsed,    0.039s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   36.060s; elapsed,    2.315s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.280s; elapsed,    0.039s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    5.510s; elapsed,    2.809s, #calls:   1', 'TOTAL                                  : CPU,   41.850s; elapsed,    5.164s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    4.390s; elapsed,    4.397s', 'individual call time for EIGEN_LDLT: CPU,    2.610s; elapsed,    2.604s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.053s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processesCPP -2', '', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 251,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 6,135', '#   - number of levels = 77', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.145689', '#   - matching time = 0.0309079', '#   - symmetrization time = 0.0150032', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 6,135', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0415602', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 51.9326 MB', '#   - factor time = 2.91968', '#   - factor nonzeros = 6,491,577', '#   - factor memory = 51.9326 MB', '#   - total flops = 7.45429e+09, min = 3.29924e+09, max = 4.15505e+09', '#   - flop rate = 2.55312 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.81378e+09', '# --------------------------------------------', '# total                 = 7.81378e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =   3.4489e-11\trel.res =  1.47627e-15\tbw.error =  3.21475e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.039288', '#   - total flops = 1.51891e+07, min = 7.43097e+06, max = 7.75814e+06', '#   - flop rate = 0.386609 GFlop/s', '#   - bytes moved = 97.4472 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.48033 GByte/s', '#   - solve arithmetic intensity = 0.15587 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    3.190s; elapsed,    3.207s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    3.210s; elapsed,    3.219s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    3.190s; elapsed,    3.207s, #calls:   1', 'TOTAL                                  : CPU,    3.190s; elapsed,    3.207s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    4.390s; elapsed,    4.397s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.053s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.610s; elapsed,    2.604s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    3.210s; elapsed,    3.219s, #calls:   1', 'TOTAL                                  : CPU,   10.260s; elapsed,   10.272s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    5.820s; elapsed,    2.937s', 'individual call time for EIGEN_LDLT: CPU,    2.690s; elapsed,    2.493s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.046s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 251,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 6,135', '#   - number of levels = 77', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.149949', '#   - matching time = 0.0315361', '#   - symmetrization time = 0.014894', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 6,135', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.047554', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 51.9326 MB', '#   - factor time = 1.71412', '#   - factor nonzeros = 6,491,577', '#   - factor memory = 51.9326 MB', '#   - total flops = 7.45434e+09, min = 3.29924e+09, max = 4.1551e+09', '#   - flop rate = 4.34879 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.81378e+09', '# --------------------------------------------', '# total                 = 7.81378e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.44115e-11\trel.res =  1.47295e-15\tbw.error =  3.21352e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.032227', '#   - total flops = 1.51917e+07, min = 7.43353e+06, max = 7.75814e+06', '#   - flop rate = 0.471395 GFlop/s', '#   - bytes moved = 97.5548 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 3.02711 GByte/s', '#   - solve arithmetic intensity = 0.155725 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    4.010s; elapsed,    2.009s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    4.040s; elapsed,    2.019s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    4.010s; elapsed,    2.009s, #calls:   1', 'TOTAL                                  : CPU,    4.010s; elapsed,    2.009s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    5.820s; elapsed,    2.937s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.046s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.690s; elapsed,    2.493s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    4.040s; elapsed,    2.019s, #calls:   1', 'TOTAL                                  : CPU,   12.620s; elapsed,    7.495s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    7.190s; elapsed,    1.830s', 'individual call time for EIGEN_LDLT: CPU,    3.110s; elapsed,    2.573s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.080s; elapsed,    0.038s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 251,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 6,135', '#   - number of levels = 77', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.145875', '#   - matching time = 0.0327501', '#   - symmetrization time = 0.016371', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 6,135', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.04703', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 51.9326 MB', '#   - factor time = 1.6067', '#   - factor nonzeros = 6,491,577', '#   - factor memory = 51.9326 MB', '#   - total flops = 7.45434e+09, min = 3.29924e+09, max = 4.1551e+09', '#   - flop rate = 4.63952 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.81378e+09', '# --------------------------------------------', '# total                 = 7.81378e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.43531e-11\trel.res =  1.47045e-15\tbw.error =  3.21352e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.040668', '#   - total flops = 1.51917e+07, min = 7.43353e+06, max = 7.75814e+06', '#   - flop rate = 0.373553 GFlop/s', '#   - bytes moved = 97.5688 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.39915 GByte/s', '#   - solve arithmetic intensity = 0.155702 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    7.620s; elapsed,    1.914s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    7.680s; elapsed,    1.926s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    7.620s; elapsed,    1.914s, #calls:   1', 'TOTAL                                  : CPU,    7.620s; elapsed,    1.914s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    7.190s; elapsed,    1.830s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.080s; elapsed,    0.038s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.110s; elapsed,    2.573s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    7.680s; elapsed,    1.926s, #calls:   1', 'TOTAL                                  : CPU,   18.060s; elapsed,    6.366s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   16.040s; elapsed,    2.038s', 'individual call time for EIGEN_LDLT: CPU,    3.710s; elapsed,    2.425s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.140s; elapsed,    0.035s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 251,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 6,135', '#   - number of levels = 77', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.171064', '#   - matching time = 0.0355849', '#   - symmetrization time = 0.0161989', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 6,135', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.050483', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 51.9326 MB', '#   - factor time = 2.25728', '#   - factor nonzeros = 6,491,577', '#   - factor memory = 51.9326 MB', '#   - total flops = 7.45434e+09, min = 3.29924e+09, max = 4.1551e+09', '#   - flop rate = 3.30236 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.81378e+09', '# --------------------------------------------', '# total                 = 7.81378e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.43303e-11\trel.res =  1.46947e-15\tbw.error =  3.21352e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0449791', '#   - total flops = 1.51917e+07, min = 7.43353e+06, max = 7.75814e+06', '#   - flop rate = 0.33775 GFlop/s', '#   - bytes moved = 97.5975 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.16984 GByte/s', '#   - solve arithmetic intensity = 0.155656 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   15.920s; elapsed,    2.596s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   20.850s; elapsed,    2.608s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   15.920s; elapsed,    2.596s, #calls:   1', 'TOTAL                                  : CPU,   15.920s; elapsed,    2.596s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   16.040s; elapsed,    2.038s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.140s; elapsed,    0.035s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.710s; elapsed,    2.425s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   20.850s; elapsed,    2.608s, #calls:   1', 'TOTAL                                  : CPU,   40.740s; elapsed,    7.106s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 9654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   32.310s; elapsed,    2.063s', 'individual call time for EIGEN_LDLT: CPU,    7.100s; elapsed,    4.389s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.240s; elapsed,    0.038s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 251,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 6,135', '#   - number of levels = 77', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.192469', '#   - matching time = 0.049571', '#   - symmetrization time = 0.01757', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 6,135', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0615411', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 51.9326 MB', '#   - factor time = 1.90784', '#   - factor nonzeros = 6,491,577', '#   - factor memory = 51.9326 MB', '#   - total flops = 7.45449e+09, min = 3.29924e+09, max = 4.15525e+09', '#   - flop rate = 3.90729 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.81378e+09', '# --------------------------------------------', '# total                 = 7.81378e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.41901e-11\trel.res =  1.46347e-15\tbw.error =  3.20616e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0497222', '#   - total flops = 1.51917e+07, min = 7.43353e+06, max = 7.75814e+06', '#   - flop rate = 0.305531 GFlop/s', '#   - bytes moved = 97.6642 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 1.9642 GByte/s', '#   - solve arithmetic intensity = 0.15555 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   33.420s; elapsed,    2.300s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   36.650s; elapsed,    2.314s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   33.420s; elapsed,    2.300s, #calls:   1', 'TOTAL                                  : CPU,   33.420s; elapsed,    2.300s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   32.310s; elapsed,    2.063s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.240s; elapsed,    0.038s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    7.100s; elapsed,    4.389s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   36.650s; elapsed,    2.314s, #calls:   1', 'TOTAL                                  : CPU,   76.300s; elapsed,    8.805s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    4.740s; elapsed,    4.748s', 'individual call time for EIGEN_LDLT: CPU,    2.660s; elapsed,    2.656s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.060s; elapsed,    0.062s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 251,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 6,141', '#   - number of levels = 73', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.166356', '#   - matching time = 0.035315', '#   - symmetrization time = 0.0222821', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 6,141', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.038007', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 51.6883 MB', '#   - factor time = 1.94571', '#   - factor nonzeros = 6,461,037', '#   - factor memory = 51.6883 MB', '#   - total flops = 7.39199e+09, min = 916324, max = 3.29543e+09', '#   - flop rate = 3.79912 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.73217e+09', '# --------------------------------------------', '# total                 = 7.73217e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.78316e-11\trel.res =   1.1913e-15\tbw.error =  2.54495e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.039614', '#   - total flops = 1.65485e+07, min = 39371, max = 7.71852e+06', '#   - flop rate = 0.417744 GFlop/s', '#   - bytes moved = 90.5862 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.28672 GByte/s', '#   - solve arithmetic intensity = 0.182683 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    2.240s; elapsed,    2.253s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    2.250s; elapsed,    2.261s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    2.250s; elapsed,    2.262s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    2.250s; elapsed,    2.266s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    2.240s; elapsed,    2.253s, #calls:   1', 'TOTAL                                  : CPU,    2.240s; elapsed,    2.253s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    2.250s; elapsed,    2.261s, #calls:   1', 'TOTAL                                  : CPU,    2.250s; elapsed,    2.261s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    2.250s; elapsed,    2.262s, #calls:   1', 'TOTAL                                  : CPU,    2.250s; elapsed,    2.262s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    4.740s; elapsed,    4.748s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.060s; elapsed,    0.062s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.660s; elapsed,    2.656s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    2.250s; elapsed,    2.266s, #calls:   1', 'TOTAL                                  : CPU,    9.710s; elapsed,    9.732s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    5.770s; elapsed,    2.912s', 'individual call time for EIGEN_LDLT: CPU,    2.870s; elapsed,    2.697s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.090s; elapsed,    0.051s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 251,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 6,141', '#   - number of levels = 73', '#   - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.167973', '#   - matching time = 0.0343199', '#   - symmetrization time = 0.0224059', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 6,141', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.042094', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 51.6883 MB', '#   - factor time = 1.70353', '#   - factor nonzeros = 6,461,037', '#   - factor memory = 51.6883 MB', '#   - total flops = 7.39201e+09, min = 916324, max = 3.29543e+09', '#   - flop rate = 4.33923 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.73217e+09', '# --------------------------------------------', '# total                 = 7.73217e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.70025e-11\trel.res =  1.15582e-15\tbw.error =  2.66061e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0429471', '#   - total flops = 1.65485e+07, min = 39371, max = 7.71852e+06', '#   - flop rate = 0.385324 GFlop/s', '#   - bytes moved = 90.6001 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.10958 GByte/s', '#   - solve arithmetic intensity = 0.182654 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    2.700s; elapsed,    2.034s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    3.840s; elapsed,    2.034s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    3.830s; elapsed,    2.026s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    4.030s; elapsed,    2.039s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    3.840s; elapsed,    2.034s, #calls:   1', 'TOTAL                                  : CPU,    3.840s; elapsed,    2.034s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    2.700s; elapsed,    2.034s, #calls:   1', 'TOTAL                                  : CPU,    2.700s; elapsed,    2.034s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    3.830s; elapsed,    2.026s, #calls:   1', 'TOTAL                                  : CPU,    3.830s; elapsed,    2.026s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    5.770s; elapsed,    2.912s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.090s; elapsed,    0.051s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.870s; elapsed,    2.697s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    4.030s; elapsed,    2.039s, #calls:   1', 'TOTAL                                  : CPU,   12.760s; elapsed,    7.699s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    8.240s; elapsed,    2.092s', 'individual call time for EIGEN_LDLT: CPU,    3.140s; elapsed,    2.607s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.090s; elapsed,    0.039s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 251,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 6,141', '#   - number of levels = 73', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.168116', '#   - matching time = 0.0348558', '#   - symmetrization time = 0.0209329', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 6,141', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0469329', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 51.6883 MB', '#   - factor time = 1.86214', '#   - factor nonzeros = 6,461,037', '#   - factor memory = 51.6883 MB', '#   - total flops = 7.39202e+09, min = 916780, max = 3.29543e+09', '#   - flop rate = 3.96963 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.73217e+09', '# --------------------------------------------', '# total                 = 7.73217e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.69566e-11\trel.res =  1.15385e-15\tbw.error =  2.65871e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0433009', '#   - total flops = 1.65485e+07, min = 39371, max = 7.71852e+06', '#   - flop rate = 0.382175 GFlop/s', '#   - bytes moved = 90.6178 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.09275 GByte/s', '#   - solve arithmetic intensity = 0.182619 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    4.180s; elapsed,    2.197s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    7.900s; elapsed,    2.197s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    6.700s; elapsed,    2.190s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    8.650s; elapsed,    2.202s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    7.900s; elapsed,    2.197s, #calls:   1', 'TOTAL                                  : CPU,    7.900s; elapsed,    2.197s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    4.180s; elapsed,    2.197s, #calls:   1', 'TOTAL                                  : CPU,    4.180s; elapsed,    2.197s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    6.700s; elapsed,    2.190s, #calls:   1', 'TOTAL                                  : CPU,    6.700s; elapsed,    2.190s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    8.240s; elapsed,    2.092s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.090s; elapsed,    0.039s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.140s; elapsed,    2.607s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    8.650s; elapsed,    2.202s, #calls:   1', 'TOTAL                                  : CPU,   20.120s; elapsed,    6.940s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 4827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   17.010s; elapsed,    2.162s', 'individual call time for EIGEN_LDLT: CPU,    7.200s; elapsed,    5.948s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.150s; elapsed,    0.041s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 251,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 6,141', '#   - number of levels = 73', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.171819', '#   - matching time = 0.0376098', '#   - symmetrization time = 0.0226271', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 6,141', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0495281', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 51.6883 MB', '#   - factor time = 2.11308', '#   - factor nonzeros = 6,461,037', '#   - factor memory = 51.6883 MB', '#   - total flops = 7.39202e+09, min = 916780, max = 3.29543e+09', '#   - flop rate = 3.49823 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.73217e+09', '# --------------------------------------------', '# total                 = 7.73217e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.76568e-11\trel.res =  1.18382e-15\tbw.error =   2.6511e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0517571', '#   - total flops = 1.65485e+07, min = 39371, max = 7.71852e+06', '#   - flop rate = 0.319734 GFlop/s', '#   - bytes moved = 90.6531 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 1.75151 GByte/s', '#   - solve arithmetic intensity = 0.182548 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    7.260s; elapsed,    2.465s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,   16.410s; elapsed,    2.467s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,   13.660s; elapsed,    2.461s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   19.410s; elapsed,    2.471s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,   16.410s; elapsed,    2.467s, #calls:   1', 'TOTAL                                  : CPU,   16.410s; elapsed,    2.467s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,   13.660s; elapsed,    2.461s, #calls:   1', 'TOTAL                                  : CPU,   13.660s; elapsed,    2.461s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    7.260s; elapsed,    2.465s, #calls:   1', 'TOTAL                                  : CPU,    7.260s; elapsed,    2.465s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   17.010s; elapsed,    2.162s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.150s; elapsed,    0.041s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    7.200s; elapsed,    5.948s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   19.410s; elapsed,    2.471s, #calls:   1', 'TOTAL                                  : CPU,   43.770s; elapsed,   10.622s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 2413.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    4.970s; elapsed,    4.977s', 'individual call time for EIGEN_LDLT: CPU,    2.670s; elapsed,    2.670s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.060s; elapsed,    0.058s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', 'CPP -2', 'CPP -2', '1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 251,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 6,141', '#   - number of levels = 72', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.169586', '#   - matching time = 0.0344558', '#   - symmetrization time = 0.0159969', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 6,141', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.035197', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 51.6883 MB', '#   - factor time = 1.66368', '#   - factor nonzeros = 6,461,037', '#   - factor memory = 51.6883 MB', '#   - total flops = 7.39199e+09, min = 730831, max = 3.2947e+09', '#   - flop rate = 4.44315 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.70381e+09', '# --------------------------------------------', '# total                 = 7.70381e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =   2.8199e-11\trel.res =  1.20703e-15\tbw.error =  2.54697e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.037612', '#   - total flops = 2.11372e+07, min = 25719, max = 7.69055e+06', '#   - flop rate = 0.561981 GFlop/s', '#   - bytes moved = 85.1253 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.26325 GByte/s', '#   - solve arithmetic intensity = 0.248307 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    1.960s; elapsed,    1.966s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    1.950s; elapsed,    1.963s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    1.930s; elapsed,    1.966s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    1.950s; elapsed,    1.967s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    1.940s; elapsed,    1.966s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    1.920s; elapsed,    1.967s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    1.940s; elapsed,    1.968s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    1.960s; elapsed,    1.971s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    1.950s; elapsed,    1.963s, #calls:   1', 'TOTAL                                  : CPU,    1.950s; elapsed,    1.963s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    1.930s; elapsed,    1.966s, #calls:   1', 'TOTAL                                  : CPU,    1.930s; elapsed,    1.966s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    1.920s; elapsed,    1.967s, #calls:   1', 'TOTAL                                  : CPU,    1.920s; elapsed,    1.967s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    1.950s; elapsed,    1.967s, #calls:   1', 'TOTAL                                  : CPU,    1.950s; elapsed,    1.967s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    1.940s; elapsed,    1.966s, #calls:   1', 'TOTAL                                  : CPU,    1.940s; elapsed,    1.966s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    1.960s; elapsed,    1.966s, #calls:   1', 'TOTAL                                  : CPU,    1.960s; elapsed,    1.966s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    1.940s; elapsed,    1.968s, #calls:   1', 'TOTAL                                  : CPU,    1.940s; elapsed,    1.968s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    4.970s; elapsed,    4.977s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.060s; elapsed,    0.058s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.670s; elapsed,    2.670s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    1.960s; elapsed,    1.971s, #calls:   1', 'TOTAL                                  : CPU,    9.660s; elapsed,    9.677s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 2413.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    6.150s; elapsed,    3.101s', 'individual call time for EIGEN_LDLT: CPU,    5.250s; elapsed,    5.079s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.080s; elapsed,    0.051s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 251,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 6,141', '#   - number of levels = 72', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.153727', '#   - matching time = 0.0327611', '#   - symmetrization time = 0.0188751', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 6,141', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0416169', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 51.6883 MB', '#   - factor time = 1.86923', '#   - factor nonzeros = 6,461,037', '#   - factor memory = 51.6883 MB', '#   - total flops = 7.392e+09, min = 730831, max = 3.2947e+09', '#   - flop rate = 3.95456 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.70381e+09', '# --------------------------------------------', '# total                 = 7.70381e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.81859e-11\trel.res =  1.20647e-15\tbw.error =  2.54697e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0411358', '#   - total flops = 2.11372e+07, min = 25719, max = 7.69055e+06', '#   - flop rate = 0.51384 GFlop/s', '#   - bytes moved = 85.1328 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.06955 GByte/s', '#   - solve arithmetic intensity = 0.248285 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    2.840s; elapsed,    2.176s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    4.190s; elapsed,    2.176s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    3.620s; elapsed,    2.170s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    3.600s; elapsed,    2.177s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    2.810s; elapsed,    2.177s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    3.110s; elapsed,    2.176s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    3.660s; elapsed,    2.176s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    4.280s; elapsed,    2.181s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    3.110s; elapsed,    2.176s, #calls:   1', 'TOTAL                                  : CPU,    3.110s; elapsed,    2.176s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    2.810s; elapsed,    2.177s, #calls:   1', 'TOTAL                                  : CPU,    2.810s; elapsed,    2.177s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    4.190s; elapsed,    2.176s, #calls:   1', 'TOTAL                                  : CPU,    4.190s; elapsed,    2.176s', 'Exiting profiler', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    3.600s; elapsed,    2.177s, #calls:   1', 'TOTAL                                  : CPU,    3.600s; elapsed,    2.177s', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    3.620s; elapsed,    2.170s, #calls:   1', 'TOTAL                                  : CPU,    3.620s; elapsed,    2.170s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    2.840s; elapsed,    2.176s, #calls:   1', 'TOTAL                                  : CPU,    2.840s; elapsed,    2.176s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    3.660s; elapsed,    2.176s, #calls:   1', 'TOTAL                                  : CPU,    3.660s; elapsed,    2.176s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    6.150s; elapsed,    3.101s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.080s; elapsed,    0.051s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    5.250s; elapsed,    5.079s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    4.280s; elapsed,    2.181s, #calls:   1', 'TOTAL                                  : CPU,   15.760s; elapsed,   10.412s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 2413.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   12.210s; elapsed,    3.084s', 'individual call time for EIGEN_LDLT: CPU,    3.180s; elapsed,    2.624s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.100s; elapsed,    0.043s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 251,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 6,141', '#   - number of levels = 72', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.175243', '#   - matching time = 0.0409341', '#   - symmetrization time = 0.0202', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 6,141', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0456691', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 51.6883 MB', '#   - factor time = 1.64532', '#   - factor nonzeros = 6,461,037', '#   - factor memory = 51.6883 MB', '#   - total flops = 7.392e+09, min = 731259, max = 3.2947e+09', '#   - flop rate = 4.49275 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.70381e+09', '# --------------------------------------------', '# total                 = 7.70381e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.81788e-11\trel.res =  1.20616e-15\tbw.error =  2.54697e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0407469', '#   - total flops = 2.11372e+07, min = 25719, max = 7.69055e+06', '#   - flop rate = 0.518744 GFlop/s', '#   - bytes moved = 85.1418 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.08953 GByte/s', '#   - solve arithmetic intensity = 0.248259 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    6.360s; elapsed,    1.988s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    4.090s; elapsed,    1.991s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    7.540s; elapsed,    1.991s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    6.770s; elapsed,    1.989s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    4.820s; elapsed,    1.988s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    6.470s; elapsed,    1.982s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    4.000s; elapsed,    1.990s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    7.840s; elapsed,    1.993s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    7.540s; elapsed,    1.991s, #calls:   1', 'TOTAL                                  : CPU,    7.540s; elapsed,    1.991s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    4.090s; elapsed,    1.991s, #calls:   1', 'TOTAL                                  : CPU,    4.090s; elapsed,    1.991s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    6.360s; elapsed,    1.988s, #calls:   1', 'TOTAL                                  : CPU,    6.360s; elapsed,    1.988s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    6.770s; elapsed,    1.989s, #calls:   1', 'TOTAL                                  : CPU,    6.770s; elapsed,    1.989s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    6.470s; elapsed,    1.982s, #calls:   1', 'TOTAL                                  : CPU,    6.470s; elapsed,    1.982s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    4.000s; elapsed,    1.990s, #calls:   1', 'TOTAL                                  : CPU,    4.000s; elapsed,    1.990s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    4.820s; elapsed,    1.988s, #calls:   1', 'TOTAL                                  : CPU,    4.820s; elapsed,    1.988s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   12.210s; elapsed,    3.084s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.100s; elapsed,    0.043s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.180s; elapsed,    2.624s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    7.840s; elapsed,    1.993s, #calls:   1', 'TOTAL                                  : CPU,   23.330s; elapsed,    7.743s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 1206.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    4.710s; elapsed,    4.718s', 'individual call time for EIGEN_LDLT: CPU,    2.600s; elapsed,    2.594s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.057s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 251,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 6,141', '#   - number of levels = 71', '#   - nd time = 0.151932', '#   - matching time = 0.0323231', '#   - symmetrization time = 0.013088', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 6,141', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0357971', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 51.6883 MB', '#   - factor time = 1.88278', '#   - factor nonzeros = 6,461,037', '#   - factor memory = 51.6883 MB', '#   - total flops = 7.39199e+09, min = 730831, max = 3.28676e+09', '#   - flop rate = 3.9261 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.64871e+09', '# --------------------------------------------', '# total                 = 7.64871e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =   2.4552e-11\trel.res =  1.05092e-15\tbw.error =  2.21425e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.033186', '#   - total flops = 2.9672e+07, min = 19684, max = 7.58043e+06', '#   - flop rate = 0.894114 GFlop/s', '#   - bytes moved = 82.5245 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.48673 GByte/s', '#   - solve arithmetic intensity = 0.359554 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    2.140s; elapsed,    2.159s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    2.130s; elapsed,    2.159s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    2.130s; elapsed,    2.159s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    2.130s; elapsed,    2.162s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU,    2.130s; elapsed,    2.160s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU,    2.110s; elapsed,    2.160s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    2.120s; elapsed,    2.160s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU,    2.140s; elapsed,    2.159s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU,    2.140s; elapsed,    2.155s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU,    2.110s; elapsed,    2.157s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU,    2.110s; elapsed,    2.159s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    2.090s; elapsed,    2.158s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU,    2.090s; elapsed,    2.158s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    2.140s; elapsed,    2.159s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU,    2.100s; elapsed,    2.159s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    2.170s; elapsed,    2.165s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    2.130s; elapsed,    2.162s, #calls:   1', 'TOTAL                                  : CPU,    2.130s; elapsed,    2.162s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=11: CPU,    2.110s; elapsed,    2.160s, #calls:   1', 'TOTAL                                  : CPU,    2.110s; elapsed,    2.160s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=10: CPU,    2.140s; elapsed,    2.159s, #calls:   1', 'TOTAL                                  : CPU,    2.140s; elapsed,    2.159s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    2.130s; elapsed,    2.159s, #calls:   1', 'TOTAL                                  : CPU,    2.130s; elapsed,    2.159s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=13: CPU,    2.140s; elapsed,    2.155s, #calls:   1', 'TOTAL                                  : CPU,    2.140s; elapsed,    2.155s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    2.140s; elapsed,    2.159s, #calls:   1', 'TOTAL                                  : CPU,    2.140s; elapsed,    2.159s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    2.120s; elapsed,    2.160s, #calls:   1', 'TOTAL                                  : CPU,    2.120s; elapsed,    2.160s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=9: CPU,    2.130s; elapsed,    2.160s, #calls:   1', 'TOTAL                                  : CPU,    2.130s; elapsed,    2.160s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    2.130s; elapsed,    2.159s, #calls:   1', 'TOTAL                                  : CPU,    2.130s; elapsed,    2.159s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    2.090s; elapsed,    2.158s, #calls:   1', 'TOTAL                                  : CPU,    2.090s; elapsed,    2.158s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    2.140s; elapsed,    2.159s, #calls:   1', 'TOTAL                                  : CPU,    2.140s; elapsed,    2.159s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=12: CPU,    2.090s; elapsed,    2.158s, #calls:   1', 'TOTAL                                  : CPU,    2.090s; elapsed,    2.158s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=14: CPU,    2.110s; elapsed,    2.157s, #calls:   1', 'TOTAL                                  : CPU,    2.110s; elapsed,    2.157s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=8: CPU,    2.100s; elapsed,    2.159s, #calls:   1', 'TOTAL                                  : CPU,    2.100s; elapsed,    2.159s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=15: CPU,    2.110s; elapsed,    2.159s, #calls:   1', 'TOTAL                                  : CPU,    2.110s; elapsed,    2.159s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    4.710s; elapsed,    4.718s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.057s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.600s; elapsed,    2.594s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    2.170s; elapsed,    2.165s, #calls:   1', 'TOTAL                                  : CPU,    9.530s; elapsed,    9.534s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '19309 1206.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    6.450s; elapsed,    3.249s', 'individual call time for EIGEN_LDLT: CPU,    2.870s; elapsed,    2.695s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.100s; elapsed,    0.071s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 19,309', '#   - number of nonzeros = 251,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 6,141', '#   - number of levels = 71', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '#   - nd time = # ******************************************************************', '0.173838', '#   - matching time = 0.0362661', '#   - symmetrization time = 0.0147071', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 6,141', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.045975', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 51.6883 MB', '#   - factor time = 1.69256', '#   - factor nonzeros = 6,461,037', '#   - factor memory = 51.6883 MB', '#   - total flops = 7.39199e+09, min = 730831, max = 3.28676e+09', '#   - flop rate = 4.36734 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.64871e+09', '# --------------------------------------------', '# total                 = 7.64871e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =    2.455e-11\trel.res =  1.05084e-15\tbw.error =  2.21425e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0389729', '#   - total flops = 2.9672e+07, min = 19684, max = 7.58043e+06', '#   - flop rate = 0.761351 GFlop/s', '#   - bytes moved = 82.5284 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.11759 GByte/s', '#   - solve arithmetic intensity = 0.359537 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    2.720s; elapsed,    2.024s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    2.690s; elapsed,    2.025s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    2.780s; elapsed,    2.024s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU,    2.900s; elapsed,    2.022s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU,    3.290s; elapsed,    2.024s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    3.000s; elapsed,    2.021s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU,    3.520s; elapsed,    2.017s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU,    3.440s; elapsed,    2.021s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    2.640s; elapsed,    2.022s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    3.930s; elapsed,    2.022s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU,    3.600s; elapsed,    2.021s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    2.650s; elapsed,    2.022s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU,    2.940s; elapsed,    2.022s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU,    2.880s; elapsed,    2.022sindividual call time for STRUMPACK_MPI_DIST_RANK=12: CPU,    3.170s; elapsed,    2.021s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    4.000s; elapsed,    2.027s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    2.690s; elapsed,    2.025s, #calls:   1', 'TOTAL                                  : CPU,    2.690s; elapsed,    2.025s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    2.720s; elapsed,    2.024s, #calls:   1', 'TOTAL                                  : CPU,    2.720s; elapsed,    2.024s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    2.780s; elapsed,    2.024s, #calls:   1', 'TOTAL                                  : CPU,    2.780s; elapsed,    2.024s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=15: CPU,    3.290s; elapsed,    2.024s, #calls:   1', 'TOTAL                                  : CPU,    3.290s; elapsed,    2.024s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=14: CPU,    3.440s; elapsed,    2.021s, #calls:   1', 'TOTAL                                  : CPU,    3.440s; elapsed,    2.021s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=13: CPU,    3.520s; elapsed,    2.017s, #calls:   1', 'TOTAL                                  : CPU,    3.520s; elapsed,    2.017s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    3.930s; elapsed,    2.022s, #calls:   1', 'TOTAL                                  : CPU,    3.930s; elapsed,    2.022s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=8: CPU,    2.940s; elapsed,    2.022s, #calls:   1', 'TOTAL                                  : CPU,    2.940s; elapsed,    2.022s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=9: CPU,    2.900s; elapsed,    2.022s, #calls:   1', 'TOTAL                                  : CPU,    2.900s; elapsed,    2.022s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    2.640s; elapsed,    2.022s, #calls:   1', 'TOTAL                                  : CPU,    2.640s; elapsed,    2.022s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    2.650s; elapsed,    2.022s, #calls:   1', 'TOTAL                                  : CPU,    2.650s; elapsed,    2.022s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=11: CPU,    3.600s; elapsed,    2.021s, #calls:   1', 'TOTAL                                  : CPU,    3.600s; elapsed,    2.021s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=10: CPU,    2.880s; elapsed,    2.022s, #calls:   1', 'TOTAL                                  : CPU,    2.880s; elapsed,    2.022s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    3.000s; elapsed,    2.021s, #calls:   1', 'TOTAL                                  : CPU,    3.000s; elapsed,    2.021s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=12: CPU,    3.170s; elapsed,    2.021s, #calls:   1', 'TOTAL                                  : CPU,    3.170s; elapsed,    2.021s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    6.450s; elapsed,    3.249s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.100s; elapsed,    0.071s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.870s; elapsed,    2.695s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    4.000s; elapsed,    2.027s, #calls:   1', 'TOTAL                                  : CPU,   13.420s; elapsed,    8.041s']
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Eta-/A_strum_5k_omp1_paramslevmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Eta-/b_strum_5k_omp1_paramslevmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '17309 17309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    4.190s; elapsed,    4.200s', 'individual call time for EIGEN_LDLT: CPU,    2.520s; elapsed,    2.509s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.050s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    4.190s; elapsed,    4.200s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.050s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.520s; elapsed,    2.509s, #calls:   1', 'TOTAL                                  : CPU,    6.760s; elapsed,    6.759s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '17309 17309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    5.380s; elapsed,    2.714s', 'individual call time for EIGEN_LDLT: CPU,    2.770s; elapsed,    2.584s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.110s; elapsed,    0.066s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    5.380s; elapsed,    2.714s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.110s; elapsed,    0.066s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.770s; elapsed,    2.584s, #calls:   1', 'TOTAL                                  : CPU,    8.260s; elapsed,    5.364s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '17309 17309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    9.400s; elapsed,    2.384s', 'individual call time for EIGEN_LDLT: CPU,    3.350s; elapsed,    2.806s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.090s; elapsed,    0.037s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    9.400s; elapsed,    2.384s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.090s; elapsed,    0.037s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.350s; elapsed,    2.806s, #calls:   1', 'TOTAL                                  : CPU,   12.840s; elapsed,    5.227s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '17309 17309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   15.870s; elapsed,    2.022s', 'individual call time for EIGEN_LDLT: CPU,    4.910s; elapsed,    3.619s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.140s; elapsed,    0.037s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   15.870s; elapsed,    2.022s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.140s; elapsed,    0.037s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    4.910s; elapsed,    3.619s, #calls:   1', 'TOTAL                                  : CPU,   20.920s; elapsed,    5.678s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '17309 17309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   36.750s; elapsed,    2.347s', 'individual call time for EIGEN_LDLT: CPU,    6.330s; elapsed,    3.595s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.220s; elapsed,    0.036s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   36.750s; elapsed,    2.347s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.220s; elapsed,    0.036s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    6.330s; elapsed,    3.595s, #calls:   1', 'TOTAL                                  : CPU,   43.300s; elapsed,    5.978s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '17309 8654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    4.370s; elapsed,    4.382s', 'individual call time for EIGEN_LDLT: CPU,    2.600s; elapsed,    2.606s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.052s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 17,309', '#   - number of nonzeros = 249,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,087', '#   - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.1261', '#   - matching time = 0.030179', '#   - symmetrization time = 0.0141439', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,087', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.041594', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 53.0181 MB', '#   - factor time = 2.81571', '#   - factor nonzeros = 6,627,267', '#   - factor memory = 53.0181 MB', '#   - total flops = 7.33115e+09, min = 3.62189e+09, max = 3.70926e+09', '#   - flop rate = 2.60366 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.68091e+09', '# --------------------------------------------', '# total                 = 7.68091e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =   3.3352e-11\trel.res =   1.4276e-15\tbw.error =  3.28895e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.035306', '#   - total flops = 1.53008e+07, min = 6.57394e+06, max = 8.72688e+06', '#   - flop rate = 0.433377 GFlop/s', '#   - bytes moved = 92.1272 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.60939 GByte/s', '#   - solve arithmetic intensity = 0.166083 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    3.060s; elapsed,    3.073s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    3.090s; elapsed,    3.085s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    3.060s; elapsed,    3.073s, #calls:   1', 'TOTAL                                  : CPU,    3.060s; elapsed,    3.073s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    4.370s; elapsed,    4.382s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.052s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.600s; elapsed,    2.606s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    3.090s; elapsed,    3.085s, #calls:   1', 'TOTAL                                  : CPU,   10.110s; elapsed,   10.125s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '17309 8654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    5.310s; elapsed,    2.675s', 'individual call time for EIGEN_LDLT: CPU,    2.820s; elapsed,    2.638s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.046s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 17,309', '#   - number of nonzeros = 249,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,087', '#   - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.130841', '#   - matching time = 0.0312269', '#   - symmetrization time = 0.0159578', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,087', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0484161', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 53.0181 MB', '#   - factor time = 1.77197', '#   - factor nonzeros = 6,627,267', '#   - factor memory = 53.0181 MB', '#   - total flops = 7.3312e+09, min = 3.62194e+09, max = 3.70926e+09', '#   - flop rate = 4.13732 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.68091e+09', '# --------------------------------------------', '# total                 = 7.68091e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =   3.3097e-11\trel.res =  1.41669e-15\tbw.error =   3.2689e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0360379', '#   - total flops = 1.53038e+07, min = 6.57691e+06, max = 8.72688e+06', '#   - flop rate = 0.424658 GFlop/s', '#   - bytes moved = 92.2479 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.55974 GByte/s', '#   - solve arithmetic intensity = 0.165899 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    4.020s; elapsed,    2.048s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    4.110s; elapsed,    2.060s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    4.020s; elapsed,    2.048s, #calls:   1', 'TOTAL                                  : CPU,    4.020s; elapsed,    2.048s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    5.310s; elapsed,    2.675s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.046s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.820s; elapsed,    2.638s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    4.110s; elapsed,    2.060s, #calls:   1', 'TOTAL                                  : CPU,   12.310s; elapsed,    7.419s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '17309 8654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   10.950s; elapsed,    2.767s', 'individual call time for EIGEN_LDLT: CPU,    4.100s; elapsed,    3.540s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.090s; elapsed,    0.042s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 17,309', '#   - number of nonzeros = 249,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,087', '#   - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.123603', '#   - matching time = 0.031127', '#   - symmetrization time = 0.0144041', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,087', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0463271', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 53.0181 MB', '#   - factor time = 2.24457', '#   - factor nonzeros = 6,627,267', '#   - factor memory = 53.0181 MB', '#   - total flops = 7.3312e+09, min = 3.62194e+09, max = 3.70926e+09', '#   - flop rate = 3.26619 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.68091e+09', '# --------------------------------------------', '# total                 = 7.68091e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.30224e-11\trel.res =  1.41349e-15\tbw.error =  3.26735e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0390739', '#   - total flops = 1.53038e+07, min = 6.57691e+06, max = 8.72688e+06', '#   - flop rate = 0.391662 GFlop/s', '#   - bytes moved = 92.2597 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.36116 GByte/s', '#   - solve arithmetic intensity = 0.165877 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    8.030s; elapsed,    2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   10.100s; elapsed,    2.525s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    8.030s; elapsed,    2.513s, #calls:   1', 'TOTAL                                  : CPU,    8.030s; elapsed,    2.513s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   10.950s; elapsed,    2.767s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.090s; elapsed,    0.042s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    4.100s; elapsed,    3.540s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   10.100s; elapsed,    2.525s, #calls:   1', 'TOTAL                                  : CPU,   25.240s; elapsed,    8.874s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '17309 8654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   17.540s; elapsed,    2.228s', 'individual call time for EIGEN_LDLT: CPU,    4.430s; elapsed,    3.129s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.140s; elapsed,    0.038s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processesCPP -2', '', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 17,309', '#   - number of nonzeros = 249,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,087', '#   - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.128883', '#   - matching time = 0.0334818', '#   - symmetrization time = 0.0149758', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,087', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0500679', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 53.0181 MB', '#   - factor time = 1.94183', '#   - factor nonzeros = 6,627,267', '#   - factor memory = 53.0181 MB', '#   - total flops = 7.33121e+09, min = 3.62194e+09, max = 3.70926e+09', '#   - flop rate = 3.77541 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.68091e+09', '# --------------------------------------------', '# total                 = 7.68091e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.30087e-11\trel.res =   1.4129e-15\tbw.error =  3.26735e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0372891', '#   - total flops = 1.53038e+07, min = 6.57691e+06, max = 8.72688e+06', '#   - flop rate = 0.410409 GFlop/s', '#   - bytes moved = 92.2841 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.47482 GByte/s', '#   - solve arithmetic intensity = 0.165833 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   14.640s; elapsed,    2.221s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   17.860s; elapsed,    2.234s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   14.640s; elapsed,    2.221s, #calls:   1', 'TOTAL                                  : CPU,   14.640s; elapsed,    2.221s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   17.540s; elapsed,    2.228s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.140s; elapsed,    0.038s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    4.430s; elapsed,    3.129s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   17.860s; elapsed,    2.234s, #calls:   1', 'TOTAL                                  : CPU,   39.970s; elapsed,    7.629s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '17309 8654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   33.660s; elapsed,    2.152s', 'individual call time for EIGEN_LDLT: CPU,    6.880s; elapsed,    4.158s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.250s; elapsed,    0.040s', 'CPP -2', '# Initializing STRUMPACKCPP -2', '', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 17,309', '#   - number of nonzeros = 249,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,087', '#   - number of levels = 68', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.14408', '#   - matching time = 0.0367179', '#   - symmetrization time = 0.0176201', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,087', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0636098', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 53.0181 MB', '#   - factor time = 1.79586', '#   - factor nonzeros = 6,627,267', '#   - factor memory = 53.0181 MB', '#   - total flops = 7.33139e+09, min = 3.62212e+09, max = 3.70926e+09', '#   - flop rate = 4.08239 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.68091e+09', '# --------------------------------------------', '# total                 = 7.68091e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.29008e-11\trel.res =  1.40828e-15\tbw.error =  3.26812e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.050818', '#   - total flops = 1.53038e+07, min = 6.57691e+06, max = 8.72688e+06', '#   - flop rate = 0.301149 GFlop/s', '#   - bytes moved = 92.3391 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 1.81706 GByte/s', '#   - solve arithmetic intensity = 0.165735 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   31.560s; elapsed,    2.128s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   34.110s; elapsed,    2.139s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   31.560s; elapsed,    2.128s, #calls:   1', 'TOTAL                                  : CPU,   31.560s; elapsed,    2.128s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   33.660s; elapsed,    2.152s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.250s; elapsed,    0.040s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    6.880s; elapsed,    4.158s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   34.110s; elapsed,    2.139s, #calls:   1', 'TOTAL                                  : CPU,   74.900s; elapsed,    8.490s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '17309 4327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    4.670s; elapsed,    4.672s', 'individual call time for EIGEN_LDLT: CPU,    2.690s; elapsed,    2.685s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.060s; elapsed,    0.058s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 17,309', '#   - number of nonzeros = 249,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,087', '#   - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.145503', '#   - matching time = 0.0334229', '#   - symmetrization time = 0.0190492', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,087', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.039644', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 53.0181 MB', '#   - factor time = 1.79257', '#   - factor nonzeros = 6,627,267', '#   - factor memory = 53.0181 MB', '#   - total flops = 7.33115e+09, min = 3.72336e+06, max = 3.70554e+09', '#   - flop rate = 4.08973 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.66768e+09', '# --------------------------------------------', '# total                 = 7.66768e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.91374e-11\trel.res =  1.24719e-15\tbw.error =  2.48962e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0350709', '#   - total flops = 1.64897e+07, min = 63729, max = 8.66016e+06', '#   - flop rate = 0.470182 GFlop/s', '#   - bytes moved = 86.6401 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.47043 GByte/s', '#   - solve arithmetic intensity = 0.190324 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    2.070s; elapsed,    2.079s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    2.050s; elapsed,    2.077s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    2.070s; elapsed,    2.070s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    2.080s; elapsed,    2.082s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    2.070s; elapsed,    2.079s, #calls:   1', 'TOTAL                                  : CPU,    2.070s; elapsed,    2.079s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    2.070s; elapsed,    2.070s, #calls:   1', 'TOTAL                                  : CPU,    2.070s; elapsed,    2.070s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    2.050s; elapsed,    2.077s, #calls:   1', 'TOTAL                                  : CPU,    2.050s; elapsed,    2.077s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    4.670s; elapsed,    4.672s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.060s; elapsed,    0.058s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.690s; elapsed,    2.685s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    2.080s; elapsed,    2.082s, #calls:   1', 'TOTAL                                  : CPU,    9.500s; elapsed,    9.497s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '17309 4327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    5.210s; elapsed,    2.628s', 'individual call time for EIGEN_LDLT: CPU,    2.750s; elapsed,    2.569s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.046s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 4 MPI processesCPP -2', 'CPP -2', 'CPP -2', '', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 17,309', '#   - number of nonzeros = 249,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,087', '#   - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.130406', '#   - matching time = 0.0314019', '#   - symmetrization time = 0.0193281', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,087', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0407379', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 53.0181 MB', '#   - factor time = 1.95598', '#   - factor nonzeros = 6,627,267', '#   - factor memory = 53.0181 MB', '#   - total flops = 7.33116e+09, min = 3.72336e+06, max = 3.70554e+09', '#   - flop rate = 3.74808 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.66768e+09', '# --------------------------------------------', '# total                 = 7.66768e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.89755e-11\trel.res =  1.24027e-15\tbw.error =  2.42638e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.041337', '#   - total flops = 1.64897e+07, min = 63729, max = 8.66016e+06', '#   - flop rate = 0.398909 GFlop/s', '#   - bytes moved = 86.6493 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.09617 GByte/s', '#   - solve arithmetic intensity = 0.190304 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    2.870s; elapsed,    2.237s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    4.070s; elapsed,    2.235s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    3.790s; elapsed,    2.228s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    4.420s; elapsed,    2.240s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    4.070s; elapsed,    2.235s, #calls:   1', 'TOTAL                                  : CPU,    4.070s; elapsed,    2.235s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    2.870s; elapsed,    2.237s, #calls:   1', 'TOTAL                                  : CPU,    2.870s; elapsed,    2.237s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    3.790s; elapsed,    2.228s, #calls:   1', 'TOTAL                                  : CPU,    3.790s; elapsed,    2.228s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    5.210s; elapsed,    2.628s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.046s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.750s; elapsed,    2.569s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    4.420s; elapsed,    2.240s, #calls:   1', 'TOTAL                                  : CPU,   12.450s; elapsed,    7.484s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '17309 4327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    7.360s; elapsed,    1.876s', 'individual call time for EIGEN_LDLT: CPU,    6.640s; elapsed,    6.100s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.100s; elapsed,    0.043s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 17,309', '#   - number of nonzeros = 249,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,087', '#   - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.129768', '#   - matching time = 0.032598', '#   - symmetrization time = 0.0193801', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,087', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0464342', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 53.0181 MB', '#   - factor time = 1.72388', '#   - factor nonzeros = 6,627,267', '#   - factor memory = 53.0181 MB', '#   - total flops = 7.33117e+09, min = 3.72505e+06, max = 3.70554e+09', '#   - flop rate = 4.25272 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.66768e+09', '# --------------------------------------------', '# total                 = 7.66768e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.89543e-11\trel.res =  1.23936e-15\tbw.error =  2.42638e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0433669', '#   - total flops = 1.64897e+07, min = 63729, max = 8.66016e+06', '#   - flop rate = 0.380237 GFlop/s', '#   - bytes moved = 86.6618 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 1.99834 GByte/s', '#   - solve arithmetic intensity = 0.190276 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    3.880s; elapsed,    2.013s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    7.420s; elapsed,    2.013s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    6.520s; elapsed,    2.005s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    7.950s; elapsed,    2.017s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    7.420s; elapsed,    2.013s, #calls:   1', 'TOTAL                                  : CPU,    7.420s; elapsed,    2.013s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    3.880s; elapsed,    2.013s, #calls:   1', 'TOTAL                                  : CPU,    3.880s; elapsed,    2.013s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    6.520s; elapsed,    2.005s, #calls:   1', 'TOTAL                                  : CPU,    6.520s; elapsed,    2.005s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    7.360s; elapsed,    1.876s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.100s; elapsed,    0.043s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    6.640s; elapsed,    6.100s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    7.950s; elapsed,    2.017s, #calls:   1', 'TOTAL                                  : CPU,   22.050s; elapsed,   10.035s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '17309 4327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   16.230s; elapsed,    2.068s', 'individual call time for EIGEN_LDLT: CPU,    3.950s; elapsed,    2.649s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.140s; elapsed,    0.039s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using CPP -2', 'CPP -2', 'CPP -2', '4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 17,309', '#   - number of nonzeros = 249,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,087', '#   - number of levels = 67', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.145592', '#   - matching time = 0.0366342', '#   - symmetrization time = 0.023905', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,087', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0573509', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 53.0181 MB', '#   - factor time = 1.78539', '#   - factor nonzeros = 6,627,267', '#   - factor memory = 53.0181 MB', '#   - total flops = 7.33117e+09, min = 3.7252e+06, max = 3.70554e+09', '#   - flop rate = 4.10621 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.66768e+09', '# --------------------------------------------', '# total                 = 7.66768e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.89215e-11\trel.res =  1.23796e-15\tbw.error =  2.42602e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.048136', '#   - total flops = 1.64897e+07, min = 63729, max = 8.66016e+06', '#   - flop rate = 0.342565 GFlop/s', '#   - bytes moved = 86.6868 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 1.80087 GByte/s', '#   - solve arithmetic intensity = 0.190222 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    7.060s; elapsed,    2.117s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,   15.690s; elapsed,    2.117s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,   13.010s; elapsed,    2.110s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   16.650s; elapsed,    2.122s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    7.060s; elapsed,    2.117s, #calls:   1', 'TOTAL                                  : CPU,    7.060s; elapsed,    2.117s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,   15.690s; elapsed,    2.117s, #calls:   1', 'TOTAL                                  : CPU,   15.690s; elapsed,    2.117s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,   13.010s; elapsed,    2.110s, #calls:   1', 'TOTAL                                  : CPU,   13.010s; elapsed,    2.110s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   16.230s; elapsed,    2.068s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.140s; elapsed,    0.039s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.950s; elapsed,    2.649s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   16.650s; elapsed,    2.122s, #calls:   1', 'TOTAL                                  : CPU,   36.970s; elapsed,    6.878s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '17309 2163.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    4.470s; elapsed,    4.469s', 'individual call time for EIGEN_LDLT: CPU,    2.620s; elapsed,    2.626s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.056s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 17,309', '#   - number of nonzeros = 249,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,087', '#   - number of levels = 66', '#   - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.129871', '#   - matching time = 0.0303349', '#   - symmetrization time = 0.02282', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,087', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.035038', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 53.0181 MB', '#   - factor time = 1.91071', '#   - factor nonzeros = 6,627,267', '#   - factor memory = 53.0181 MB', '#   - total flops = 7.33115e+09, min = 3.56378e+06, max = 3.70197e+09', '#   - flop rate = 3.83686 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.63859e+09', '# --------------------------------------------', '# total                 = 7.63859e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.81301e-11\trel.res =  1.20408e-15\tbw.error =  2.38827e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0389578', '#   - total flops = 2.05202e+07, min = 52652, max = 8.60455e+06', '#   - flop rate = 0.526728 GFlop/s', '#   - bytes moved = 81.8271 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.1004 GByte/s', '#   - solve arithmetic intensity = 0.250775 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    2.140s; elapsed,    2.183s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    2.170s; elapsed,    2.184s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    2.170s; elapsed,    2.182s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    2.150s; elapsed,    2.174s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    2.150s; elapsed,    2.183s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    2.180s; elapsed,    2.184s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    2.160s; elapsed,    2.183s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    2.190s; elapsed,    2.187s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    2.140s; elapsed,    2.183s, #calls:   1', 'TOTAL                                  : CPU,    2.140s; elapsed,    2.183s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    2.170s; elapsed,    2.182s, #calls:   1', 'TOTAL                                  : CPU,    2.170s; elapsed,    2.182s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    2.180s; elapsed,    2.184s, #calls:   1', 'TOTAL                                  : CPU,    2.180s; elapsed,    2.184s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    2.170s; elapsed,    2.184s, #calls:   1', 'TOTAL                                  : CPU,    2.170s; elapsed,    2.184s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    2.150s; elapsed,    2.174s, #calls:   1', 'TOTAL                                  : CPU,    2.150s; elapsed,    2.174s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    2.150s; elapsed,    2.183s, #calls:   1', 'TOTAL                                  : CPU,    2.150s; elapsed,    2.183s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    2.160s; elapsed,    2.183s, #calls:   1', 'TOTAL                                  : CPU,    2.160s; elapsed,    2.183s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    4.470s; elapsed,    4.469s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.056s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.620s; elapsed,    2.626s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    2.190s; elapsed,    2.187s, #calls:   1', 'TOTAL                                  : CPU,    9.330s; elapsed,    9.338s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '17309 2163.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    5.450s; elapsed,    2.758s', 'individual call time for EIGEN_LDLT: CPU,    3.030s; elapsed,    2.834s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.048s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 17,309', '#   - number of nonzeros = 249,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,087', '#   - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.134082', '#   - matching time = 0.0329502', '#   - symmetrization time = 0.0245631', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,087', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.044343', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 53.0181 MB', '#   - factor time = 1.83034', '#   - factor nonzeros = 6,627,267', '#   - factor memory = 53.0181 MB', '#   - total flops = 7.33116e+09, min = 3.56378e+06, max = 3.70197e+09', '#   - flop rate = 4.00535 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.63859e+09', '# --------------------------------------------', '# total                 = 7.63859e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.81094e-11\trel.res =  1.20319e-15\tbw.error =  2.38936e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0450761', '#   - total flops = 2.05202e+07, min = 52652, max = 8.60455e+06', '#   - flop rate = 0.455234 GFlop/s', '#   - bytes moved = 81.8342 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 1.81547 GByte/s', '#   - solve arithmetic intensity = 0.250753 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    2.800s; elapsed,    2.130s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    3.520s; elapsed,    2.120s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    3.590s; elapsed,    2.129s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    3.980s; elapsed,    2.129s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    3.050s; elapsed,    2.129s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    3.670s; elapsed,    2.129s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    2.770s; elapsed,    2.129s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    4.210s; elapsed,    2.134s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    3.520s; elapsed,    2.120s, #calls:   1', 'TOTAL                                  : CPU,    3.520s; elapsed,    2.120s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    2.800s; elapsed,    2.130s, #calls:   1', 'TOTAL                                  : CPU,    2.800s; elapsed,    2.130s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    3.590s; elapsed,    2.129s, #calls:   1', 'TOTAL                                  : CPU,    3.590s; elapsed,    2.129s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    3.050s; elapsed,    2.129s, #calls:   1', 'TOTAL                                  : CPU,    3.050s; elapsed,    2.129s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    2.770s; elapsed,    2.129s, #calls:   1', 'TOTAL                                  : CPU,    2.770s; elapsed,    2.129s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    3.980s; elapsed,    2.129s, #calls:   1', 'TOTAL                                  : CPU,    3.980s; elapsed,    2.129s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    3.670s; elapsed,    2.129s, #calls:   1', 'TOTAL                                  : CPU,    3.670s; elapsed,    2.129s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    5.450s; elapsed,    2.758s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.048s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.030s; elapsed,    2.834s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    4.210s; elapsed,    2.134s, #calls:   1', 'TOTAL                                  : CPU,   12.760s; elapsed,    7.774s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '17309 2163.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    8.580s; elapsed,    2.176s', 'individual call time for EIGEN_LDLT: CPU,    3.020s; elapsed,    2.469s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.100s; elapsed,    0.040s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using 8 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 17,309', '#   - number of nonzeros = 249,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,087', '#   - number of levels = 66', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.147006', '#   - matching time = 0.0353072', '#   - symmetrization time = 0.023737', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,087', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0469329', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 53.0181 MB', '#   - factor time = 1.92219', '#   - factor nonzeros = 6,627,267', '#   - factor memory = 53.0181 MB', '#   - total flops = 7.33116e+09, min = 3.56586e+06, max = 3.70197e+09', '#   - flop rate = 3.81397 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.63859e+09', '# --------------------------------------------', '# total                 = 7.63859e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.14314e-11\trel.res =  1.34539e-15\tbw.error =  2.39921e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0487611', '#   - total flops = 2.05202e+07, min = 52652, max = 8.60455e+06', '#   - flop rate = 0.420831 GFlop/s', '#   - bytes moved = 81.8434 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 1.67846 GByte/s', '#   - solve arithmetic intensity = 0.250725 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    4.310s; elapsed,    2.241s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    4.160s; elapsed,    2.241s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    7.690s; elapsed,    2.242s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    6.650s; elapsed,    2.242s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    5.030s; elapsed,    2.241s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    6.490s; elapsed,    2.242s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    6.400s; elapsed,    2.233s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    8.880s; elapsed,    2.245s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    4.310s; elapsed,    2.241s, #calls:   1', 'TOTAL                                  : CPU,    4.310s; elapsed,    2.241s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    7.690s; elapsed,    2.242s, #calls:   1', 'TOTAL                                  : CPU,    7.690s; elapsed,    2.242s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    5.030s; elapsed,    2.241s, #calls:   1', 'TOTAL                                  : CPU,    5.030s; elapsed,    2.241s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    4.160s; elapsed,    2.241s, #calls:   1', 'TOTAL                                  : CPU,    4.160s; elapsed,    2.241s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    6.490s; elapsed,    2.242s, #calls:   1', 'TOTAL                                  : CPU,    6.490s; elapsed,    2.242s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    6.650s; elapsed,    2.242s, #calls:   1', 'TOTAL                                  : CPU,    6.650s; elapsed,    2.242s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    6.400s; elapsed,    2.233s, #calls:   1', 'TOTAL                                  : CPU,    6.400s; elapsed,    2.233s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    8.580s; elapsed,    2.176s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.100s; elapsed,    0.040s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.020s; elapsed,    2.469s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    8.880s; elapsed,    2.245s, #calls:   1', 'TOTAL                                  : CPU,   20.580s; elapsed,    6.931s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '17309 1081.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    4.940s; elapsed,    4.942s', 'individual call time for EIGEN_LDLT: CPU,    3.220s; elapsed,    3.219s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.060s; elapsed,    0.063s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 17,309', '#   - number of nonzeros = 249,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,087', '#   - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.144272', '#   - matching time = 0.0349131', '#   - symmetrization time = 0.0130441', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,087', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0379288', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 53.0181 MB', '#   - factor time = 1.88906', '#   - factor nonzeros = 6,627,267', '#   - factor memory = 53.0181 MB', '#   - total flops = 7.33115e+09, min = 3.56378e+06, max = 3.68612e+09', '#   - flop rate = 3.88084 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.58349e+09', '# --------------------------------------------', '# total                 = 7.58349e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.40797e-11\trel.res =  1.03071e-15\tbw.error =  2.36644e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0317309', '#   - total flops = 2.77955e+07, min = 47242, max = 8.4104e+06', '#   - flop rate = 0.875975 GFlop/s', '#   - bytes moved = 78.9378 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.48773 GByte/s', '#   - solve arithmetic intensity = 0.352118 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    2.100s; elapsed,    2.162s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU,    2.150s; elapsed,    2.164s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU,    2.120s; elapsed,    2.164s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU,    2.150s; elapsed,    2.165s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU,    2.150s; elapsed,    2.164s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    2.140s; elapsed,    2.163s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    2.120s; elapsed,    2.164s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    2.100s; elapsed,    2.163s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    2.160s; elapsed,    2.163s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU,    2.100s; elapsed,    2.160s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU,    2.120s; elapsed,    2.158s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    2.160s; elapsed,    2.163s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    2.130s; elapsed,    2.164s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU,    2.130s; elapsed,    2.163s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU,    2.110s; elapsed,    2.164s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    2.160s; elapsed,    2.169s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=11: CPU,    2.150s; elapsed,    2.165s, #calls:   1', 'TOTAL                                  : CPU,    2.150s; elapsed,    2.165s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=9: CPU,    2.150s; elapsed,    2.164s, #calls:   1', 'TOTAL                                  : CPU,    2.150s; elapsed,    2.164s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=13: CPU,    2.150s; elapsed,    2.164s, #calls:   1', 'TOTAL                                  : CPU,    2.150s; elapsed,    2.164s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    2.120s; elapsed,    2.164s, #calls:   1', 'TOTAL                                  : CPU,    2.120s; elapsed,    2.164s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    2.100s; elapsed,    2.163s, #calls:   1', 'TOTAL                                  : CPU,    2.100s; elapsed,    2.163s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=14: CPU,    2.100s; elapsed,    2.160s, #calls:   1', 'TOTAL                                  : CPU,    2.100s; elapsed,    2.160s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=15: CPU,    2.120s; elapsed,    2.158s, #calls:   1', 'TOTAL                                  : CPU,    2.120s; elapsed,    2.158s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=8: CPU,    2.110s; elapsed,    2.164s, #calls:   1', 'TOTAL                                  : CPU,    2.110s; elapsed,    2.164s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    2.160s; elapsed,    2.163s, #calls:   1', 'TOTAL                                  : CPU,    2.160s; elapsed,    2.163s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    2.130s; elapsed,    2.164s, #calls:   1', 'TOTAL                                  : CPU,    2.130s; elapsed,    2.164s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    2.140s; elapsed,    2.163s, #calls:   1', 'TOTAL                                  : CPU,    2.140s; elapsed,    2.163s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    2.160s; elapsed,    2.163s, #calls:   1', 'TOTAL                                  : CPU,    2.160s; elapsed,    2.163s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    2.100s; elapsed,    2.162s, #calls:   1', 'TOTAL                                  : CPU,    2.100s; elapsed,    2.162s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=10: CPU,    2.120s; elapsed,    2.164s, #calls:   1', 'TOTAL                                  : CPU,    2.120s; elapsed,    2.164s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=12: CPU,    2.130s; elapsed,    2.163s, #calls:   1', 'TOTAL                                  : CPU,    2.130s; elapsed,    2.163s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    4.940s; elapsed,    4.942s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.060s; elapsed,    0.063s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.220s; elapsed,    3.219s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    2.160s; elapsed,    2.169s, #calls:   1', 'TOTAL                                  : CPU,   10.380s; elapsed,   10.393s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '17309 1081.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    5.780s; elapsed,    2.916s', 'individual call time for EIGEN_LDLT: CPU,    2.710s; elapsed,    2.530s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.080s; elapsed,    0.048s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threadsCPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 16 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 17,309', '#   - number of nonzeros = 249,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 4,087', '#   - number of levels = 65', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.149898', '#   - matching time = 0.0356231', '#   - symmetrization time = 0.0138478', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 4,087', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.045547', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 53.0181 MB', '#   - factor time = 1.70579', '#   - factor nonzeros = 6,627,267', '#   - factor memory = 53.0181 MB', '#   - total flops = 7.33116e+09, min = 3.56378e+06, max = 3.68612e+09', '#   - flop rate = 4.29781 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.58349e+09', '# --------------------------------------------', '# total                 = 7.58349e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.63605e-11\trel.res =  1.12833e-15\tbw.error =  2.10167e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.033108', '#   - total flops = 2.77955e+07, min = 47242, max = 8.4104e+06', '#   - flop rate = 0.839539 GFlop/s', '#   - bytes moved = 78.9432 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.38442 GByte/s', '#   - solve arithmetic intensity = 0.352094 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    2.750s; elapsed,    2.000s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    2.670s; elapsed,    1.999s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    2.620s; elapsed,    1.997sindividual call time for STRUMPACK_MPI_DIST_RANK=9: CPU,    2.860s; elapsed,    1.997s', '', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    2.590s; elapsed,    1.997s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU,    2.830s; elapsed,    1.999s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU,    3.440s; elapsed,    1.998s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU,    3.420s; elapsed,    1.993s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    2.710s; elapsed,    1.998s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    2.620s; elapsed,    1.998s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    3.810s; elapsed,    1.997s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU,    3.090s; elapsed,    1.998s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU,    3.380s; elapsed,    1.994s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU,    2.940s; elapsed,    1.998s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU,    3.430s; elapsed,    1.998s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    3.950s; elapsed,    2.005s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    2.750s; elapsed,    2.000s, #calls:   1', 'TOTAL                                  : CPU,    2.750s; elapsed,    2.000s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=8: CPU,    2.940s; elapsed,    1.998s, #calls:   1', 'TOTAL                                  : CPU,    2.940s; elapsed,    1.998s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    2.670s; elapsed,    1.999s, #calls:   1', 'TOTAL                                  : CPU,    2.670s; elapsed,    1.999s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    2.710s; elapsed,    1.998s, #calls:   1', 'TOTAL                                  : CPU,    2.710s; elapsed,    1.998s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=9: CPU,    2.860s; elapsed,    1.997s, #calls:   1', 'TOTAL                                  : CPU,    2.860s; elapsed,    1.997s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=14: CPU,    3.380s; elapsed,    1.994s, #calls:   1', 'TOTAL                                  : CPU,    3.380s; elapsed,    1.994s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=10: CPU,    2.830s; elapsed,    1.999s, #calls:   1', 'TOTAL                                  : CPU,    2.830s; elapsed,    1.999s', 'Exiting profiler', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=11: CPU,    3.440s; elapsed,    1.998s, #calls:   1', 'TOTAL                                  : CPU,    3.440s; elapsed,    1.998s', 'time for     STRUMPACK_MPI_DIST_RANK=13: CPU,    3.430s; elapsed,    1.998s, #calls:   1', 'TOTAL                                  : CPU,    3.430s; elapsed,    1.998s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    3.810s; elapsed,    1.997s, #calls:   1', 'TOTAL                                  : CPU,    3.810s; elapsed,    1.997s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=15: CPU,    3.420s; elapsed,    1.993s, #calls:   1', 'TOTAL                                  : CPU,    3.420s; elapsed,    1.993s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    2.620s; elapsed,    1.998s, #calls:   1', 'TOTAL                                  : CPU,    2.620s; elapsed,    1.998s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    2.590s; elapsed,    1.997s, #calls:   1', 'TOTAL                                  : CPU,    2.590s; elapsed,    1.997s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    2.620s; elapsed,    1.997s, #calls:   1', 'TOTAL                                  : CPU,    2.620s; elapsed,    1.997s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=12: CPU,    3.090s; elapsed,    1.998s, #calls:   1', 'TOTAL                                  : CPU,    3.090s; elapsed,    1.998s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    5.780s; elapsed,    2.916s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.080s; elapsed,    0.048s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.710s; elapsed,    2.530s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    3.950s; elapsed,    2.005s, #calls:   1', 'TOTAL                                  : CPU,   12.520s; elapsed,    7.499s']
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-/A_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-/b_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Bfactor-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   24.060s; elapsed,   24.074s', 'individual call time for EIGEN_LDLT: CPU,   18.820s; elapsed,   18.838s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.170s; elapsed,    0.169s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   24.060s; elapsed,   24.074s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.170s; elapsed,    0.169s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   18.820s; elapsed,   18.838s, #calls:   1', 'TOTAL                                  : CPU,   43.050s; elapsed,   43.082s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   30.320s; elapsed,   15.219s', 'individual call time for EIGEN_LDLT: CPU,   19.210s; elapsed,   19.031s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.270s; elapsed,    0.162s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   30.320s; elapsed,   15.219s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.270s; elapsed,    0.162s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   19.210s; elapsed,   19.031s, #calls:   1', 'TOTAL                                  : CPU,   49.800s; elapsed,   34.412s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   38.720s; elapsed,    9.720s', 'individual call time for EIGEN_LDLT: CPU,   19.430s; elapsed,   18.877s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.300s; elapsed,    0.117s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   38.720s; elapsed,    9.720s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.300s; elapsed,    0.117s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   19.430s; elapsed,   18.877s, #calls:   1', 'TOTAL                                  : CPU,   58.450s; elapsed,   28.715s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   80.890s; elapsed,   10.165s', 'individual call time for EIGEN_LDLT: CPU,   20.280s; elapsed,   19.021s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.390s; elapsed,    0.097s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   80.890s; elapsed,   10.165s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.390s; elapsed,    0.097s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   20.280s; elapsed,   19.021s, #calls:   1', 'TOTAL                                  : CPU,  101.560s; elapsed,   29.283s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,  166.980s; elapsed,   10.497s', 'individual call time for EIGEN_LDLT: CPU,   21.380s; elapsed,   18.683s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.740s; elapsed,    0.088s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,  166.980s; elapsed,   10.497s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.740s; elapsed,    0.088s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   21.380s; elapsed,   18.683s, #calls:   1', 'TOTAL                                  : CPU,  189.100s; elapsed,   29.267s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   24.790s; elapsed,   24.814s', 'individual call time for EIGEN_LDLT: CPU,   19.390s; elapsed,   19.405s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.180s; elapsed,    0.179s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 491,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,781', '#   - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.226218', '#   - matching time = 0.0563948', '#   - symmetrization time = 0.0344679', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,781', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.074439', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 160.156 MB', '#   - factor time = 16.616', '#   - factor nonzeros = 20,019,517', '#   - factor memory = 160.156 MB', '#   - total flops = 5.00254e+10, min = 2.29344e+10, max = 2.7091e+10', '#   - flop rate = 3.01068 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.23886e+10', '# --------------------------------------------', '# total                 = 5.23886e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  4.51073e-11\trel.res =  1.84455e-15\tbw.error =  4.16326e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0946271', '#   - total flops = 4.47923e+07, min = 2.05301e+07, max = 2.42622e+07', '#   - flop rate = 0.473356 GFlop/s', '#   - bytes moved = 229.689 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.42731 GByte/s', '#   - solve arithmetic intensity = 0.195013 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   17.110s; elapsed,   17.132s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   17.130s; elapsed,   17.147s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   17.110s; elapsed,   17.132s, #calls:   1', 'TOTAL                                  : CPU,   17.110s; elapsed,   17.132s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   24.790s; elapsed,   24.814s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.180s; elapsed,    0.179s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   19.390s; elapsed,   19.405s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   17.130s; elapsed,   17.147s, #calls:   1', 'TOTAL                                  : CPU,   61.490s; elapsed,   61.544s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   29.670s; elapsed,   14.881s', 'individual call time for EIGEN_LDLT: CPU,   32.380s; elapsed,   32.222s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.290s; elapsed,    0.176s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 491,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,781', '#   - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,#   - nd time = ', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.217317', '#   - matching time = 0.0619159', '#   - symmetrization time = 0.0342321', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,781', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.076571', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 160.156 MB', '#   - factor time = 9.66358', '#   - factor nonzeros = 20,019,517', '#   - factor memory = 160.156 MB', '#   - total flops = 5.00256e+10, min = 2.29344e+10, max = 2.70912e+10', '#   - flop rate = 5.17672 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.23886e+10', '# --------------------------------------------', '# total                 = 5.23886e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  4.46879e-11\trel.res =   1.8274e-15\tbw.error =  4.16242e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0888779', '#   - total flops = 4.481e+07, min = 2.05478e+07, max = 2.42622e+07', '#   - flop rate = 0.504175 GFlop/s', '#   - bytes moved = 230.398 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.59229 GByte/s', '#   - solve arithmetic intensity = 0.19449 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   19.650s; elapsed,   10.176s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   20.240s; elapsed,   10.189s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   19.650s; elapsed,   10.176s, #calls:   1', 'TOTAL                                  : CPU,   19.650s; elapsed,   10.176s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   29.670s; elapsed,   14.881s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.290s; elapsed,    0.176s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   32.380s; elapsed,   32.222s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   20.240s; elapsed,   10.189s, #calls:   1', 'TOTAL                                  : CPU,   82.580s; elapsed,   57.469s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   38.960s; elapsed,    9.794s', 'individual call time for EIGEN_LDLT: CPU,   19.690s; elapsed,   19.168s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.290s; elapsed,    0.113s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 491,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,781', '#   - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.211068', '#   - matching time = 0.0545471', '#   - symmetrization time = 0.0341558', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,781', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0801709', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 160.156 MB', '#   - factor time = 9.53991', '#   - factor nonzeros = 20,019,517', '#   - factor memory = 160.156 MB', '#   - total flops = 5.00256e+10, min = 2.29344e+10, max = 2.70912e+10', '#   - flop rate = 5.24383 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.23886e+10', '# --------------------------------------------', '# total                 = 5.23886e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  4.46826e-11\trel.res =  1.82718e-15\tbw.error =  4.16509e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.096158', '#   - total flops = 4.481e+07, min = 2.05478e+07, max = 2.42622e+07', '#   - flop rate = 0.466004 GFlop/s', '#   - bytes moved = 230.416 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.39622 GByte/s', '#   - solve arithmetic intensity = 0.194475 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   34.310s; elapsed,   10.050s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   39.940s; elapsed,   10.065s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   34.310s; elapsed,   10.050s, #calls:   1', 'TOTAL                                  : CPU,   34.310s; elapsed,   10.050s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   38.960s; elapsed,    9.794s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.290s; elapsed,    0.113s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   19.690s; elapsed,   19.168s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   39.940s; elapsed,   10.065s, #calls:   1', 'TOTAL                                  : CPU,   98.880s; elapsed,   39.140s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   76.380s; elapsed,    9.621s', 'individual call time for EIGEN_LDLT: CPU,   20.560s; elapsed,   19.329s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.410s; elapsed,    0.093s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 491,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,781', '#   - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.218543', '#   - matching time = 0.057996', '#   - symmetrization time = 0.040771', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,781', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0839651', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 160.156 MB', '#   - factor time = 9.67384', '#   - factor nonzeros = 20,019,517', '#   - factor memory = 160.156 MB', '#   - total flops = 5.00256e+10, min = 2.29344e+10, max = 2.70912e+10', '#   - flop rate = 5.17123 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.23886e+10', '# --------------------------------------------', '# total                 = 5.23886e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  4.47005e-11\trel.res =  1.82791e-15\tbw.error =  4.16543e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.121101', '#   - total flops = 4.481e+07, min = 2.05478e+07, max = 2.42622e+07', '#   - flop rate = 0.370022 GFlop/s', '#   - bytes moved = 230.453 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 1.90298 GByte/s', '#   - solve arithmetic intensity = 0.194443 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   67.660s; elapsed,   10.232s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   81.200s; elapsed,   10.251s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   67.660s; elapsed,   10.232s, #calls:   1', 'TOTAL                                  : CPU,   67.660s; elapsed,   10.232s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   76.380s; elapsed,    9.621s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.410s; elapsed,    0.093s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   20.560s; elapsed,   19.329s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   81.200s; elapsed,   10.251s, #calls:   1', 'TOTAL                                  : CPU,  178.550s; elapsed,   39.294s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,  189.190s; elapsed,   11.888s', 'individual call time for EIGEN_LDLT: CPU,   22.050s; elapsed,   19.352s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.710s; elapsed,    0.089s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 491,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,781', '#   - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.24396', '#   - matching time = 0.068167', '#   - symmetrization time = 0.0411301', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,781', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0925391', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 160.156 MB', '#   - factor time = 10.9859', '#   - factor nonzeros = 20,019,517', '#   - factor memory = 160.156 MB', '#   - total flops = 5.00263e+10, min = 2.29344e+10, max = 2.70919e+10', '#   - flop rate = 4.55368 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.23886e+10', '# --------------------------------------------', '# total                 = 5.23886e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  4.46636e-11\trel.res =  1.82641e-15\tbw.error =  4.16509e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0951681', '#   - total flops = 4.48137e+07, min = 2.05515e+07, max = 2.42622e+07', '#   - flop rate = 0.47089 GFlop/s', '#   - bytes moved = 230.586 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.42293 GByte/s', '#   - solve arithmetic intensity = 0.194347 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,  155.330s; elapsed,   11.577s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,  183.140s; elapsed,   11.592s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,  155.330s; elapsed,   11.577s, #calls:   1', 'TOTAL                                  : CPU,  155.330s; elapsed,   11.577s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,  189.190s; elapsed,   11.888s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.710s; elapsed,    0.089s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   22.050s; elapsed,   19.352s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,  183.140s; elapsed,   11.592s, #calls:   1', 'TOTAL                                  : CPU,  395.090s; elapsed,   42.921s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   25.620s; elapsed,   25.647s', 'individual call time for EIGEN_LDLT: CPU,   19.380s; elapsed,   19.390s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.180s; elapsed,    0.188s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 491,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,763', '#   - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.230615', '#   - matching time = 0.0568919', '#   - symmetrization time = 0.022572', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,763', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0617738', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 162.795 MB', '#   - factor time = 9.74542', '#   - factor nonzeros = 20,349,413', '#   - factor memory = 162.795 MB', '#   - total flops = 5.13294e+10, min = 2.45595e+06, max = 2.28929e+10', '#   - flop rate = 5.26703 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.36486e+10', '# --------------------------------------------', '# total                 = 5.36486e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  4.31158e-11\trel.res =  1.76311e-15\tbw.error =  4.50443e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0648479', '#   - total flops = 5.20842e+07, min = 54847, max = 2.42155e+07', '#   - flop rate = 0.803175 GFlop/s', '#   - bytes moved = 202.404 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 3.1212 GByte/s', '#   - solve arithmetic intensity = 0.257329 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,   10.170s; elapsed,   10.199s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   10.190s; elapsed,   10.201s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,   10.180s; elapsed,   10.193s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   10.200s; elapsed,   10.207s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,   10.180s; elapsed,   10.193s, #calls:   1', 'TOTAL                                  : CPU,   10.180s; elapsed,   10.193s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   10.190s; elapsed,   10.201s, #calls:   1', 'TOTAL                                  : CPU,   10.190s; elapsed,   10.201s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,   10.170s; elapsed,   10.199s, #calls:   1', 'TOTAL                                  : CPU,   10.170s; elapsed,   10.199s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   25.620s; elapsed,   25.647s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.180s; elapsed,    0.188s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   19.380s; elapsed,   19.390s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   10.200s; elapsed,   10.207s, #calls:   1', 'TOTAL                                  : CPU,   55.380s; elapsed,   55.432s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   29.330s; elapsed,   14.733s', 'individual call time for EIGEN_LDLT: CPU,   19.220s; elapsed,   19.045s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.210s; elapsed,    0.132s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 491,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,763', '#   - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.229134', '#   - matching time = 0.0568249', '#   - symmetrization time = 0.0270479', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,763', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.073278', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 162.795 MB', '#   - factor time = 9.63242', '#   - factor nonzeros = 20,349,413', '#   - factor memory = 162.795 MB', '#   - total flops = 5.13295e+10, min = 2.45595e+06, max = 2.28929e+10', '#   - flop rate = 5.32882 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.36486e+10', '# --------------------------------------------', '# total                 = 5.36486e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =   4.1948e-11\trel.res =  1.71536e-15\tbw.error =  4.50638e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.066623', '#   - total flops = 5.20895e+07, min = 54847, max = 2.42155e+07', '#   - flop rate = 0.781856 GFlop/s', '#   - bytes moved = 202.615 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 3.04122 GByte/s', '#   - solve arithmetic intensity = 0.257087 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,   15.870s; elapsed,   10.106s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   11.420s; elapsed,   10.109s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,   18.970s; elapsed,   10.102s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   19.970s; elapsed,   10.116s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   11.420s; elapsed,   10.109s, #calls:   1', 'TOTAL                                  : CPU,   11.420s; elapsed,   10.109s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,   18.970s; elapsed,   10.102s, #calls:   1', 'TOTAL                                  : CPU,   18.970s; elapsed,   10.102s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,   15.870s; elapsed,   10.106s, #calls:   1', 'TOTAL                                  : CPU,   15.870s; elapsed,   10.106s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   29.330s; elapsed,   14.733s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.210s; elapsed,    0.132s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   19.220s; elapsed,   19.045s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   19.970s; elapsed,   10.116s, #calls:   1', 'TOTAL                                  : CPU,   68.730s; elapsed,   44.026s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   39.460s; elapsed,    9.925s', 'individual call time for EIGEN_LDLT: CPU,   19.830s; elapsed,   19.291s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.300s; elapsed,    0.115s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 491,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,763', '#   - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.231708', '#   - matching time = 0.058223', '#   - symmetrization time = 0.0290191', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,763', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0720448', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 162.795 MB', '#   - factor time = 9.5864', '#   - factor nonzeros = 20,349,413', '#   - factor memory = 162.795 MB', '#   - total flops = 5.13295e+10, min = 2.45658e+06, max = 2.28929e+10', '#   - flop rate = 5.35441 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.36486e+10', '# --------------------------------------------', '# total                 = 5.36486e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  4.19742e-11\trel.res =  1.71643e-15\tbw.error =  4.50833e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0679989', '#   - total flops = 5.20895e+07, min = 54847, max = 2.42155e+07', '#   - flop rate = 0.766035 GFlop/s', '#   - bytes moved = 202.628 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.97987 GByte/s', '#   - solve arithmetic intensity = 0.25707 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,   33.240s; elapsed,   10.061s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,   24.920s; elapsed,   10.065s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   13.690s; elapsed,   10.066s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   39.610s; elapsed,   10.073s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,   33.240s; elapsed,   10.061s, #calls:   1', 'TOTAL                                  : CPU,   33.240s; elapsed,   10.061s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   13.690s; elapsed,   10.066s, #calls:   1', 'TOTAL                                  : CPU,   13.690s; elapsed,   10.066s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,   24.920s; elapsed,   10.065s, #calls:   1', 'TOTAL                                  : CPU,   24.920s; elapsed,   10.065s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   39.460s; elapsed,    9.925s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.300s; elapsed,    0.115s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   19.830s; elapsed,   19.291s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   39.610s; elapsed,   10.073s, #calls:   1', 'TOTAL                                  : CPU,   99.200s; elapsed,   39.404s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   88.360s; elapsed,   11.199s', 'individual call time for EIGEN_LDLT: CPU,   20.130s; elapsed,   18.904s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.470s; elapsed,    0.107s', 'CPP -2', '# Initializing STRUMPACK', '# using CPP -2', 'CPP -2', 'CPP -2', '8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 491,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,763', '#   - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.251237', '#   - matching time = 0.067874', '#   - symmetrization time = 0.028312', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,763', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.083693', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 162.795 MB', '#   - factor time = 10.3587', '#   - factor nonzeros = 20,349,413', '#   - factor memory = 162.795 MB', '#   - total flops = 5.13295e+10, min = 2.45658e+06, max = 2.28929e+10', '#   - flop rate = 4.95521 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.36486e+10', '# --------------------------------------------', '# total                 = 5.36486e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  4.19328e-11\trel.res =  1.71474e-15\tbw.error =  4.50833e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0798702', '#   - total flops = 5.20895e+07, min = 54847, max = 2.42155e+07', '#   - flop rate = 0.652177 GFlop/s', '#   - bytes moved = 202.654 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.5373 GByte/s', '#   - solve arithmetic intensity = 0.257036 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,   48.270s; elapsed,   10.892s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,   68.930s; elapsed,   10.888s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   19.640s; elapsed,   10.894s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   85.580s; elapsed,   10.901s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,   68.930s; elapsed,   10.888s, #calls:   1', 'TOTAL                                  : CPU,   68.930s; elapsed,   10.888s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,   48.270s; elapsed,   10.892s, #calls:   1', 'TOTAL                                  : CPU,   48.270s; elapsed,   10.892s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   19.640s; elapsed,   10.894s, #calls:   1', 'TOTAL                                  : CPU,   19.640s; elapsed,   10.894s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   88.360s; elapsed,   11.199s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.470s; elapsed,    0.107s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   20.130s; elapsed,   18.904s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   85.580s; elapsed,   10.901s, #calls:   1', 'TOTAL                                  : CPU,  194.540s; elapsed,   41.110s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 2913.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   25.820s; elapsed,   25.845s', 'individual call time for EIGEN_LDLT: CPU,   18.870s; elapsed,   18.878s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.190s; elapsed,    0.189s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 491,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,761', '#   - number of levels = 130', '#   - nd time = 0.226691# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '', '#   - matching time = 0.0579801', '#   - symmetrization time = 0.0214732', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,761', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0543609', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 163.769 MB', '#   - factor time = 9.98133', '#   - factor nonzeros = 20,471,129', '#   - factor memory = 163.769 MB', '#   - total flops = 5.19135e+10, min = 2.03243e+06, max = 2.28029e+10', '#   - flop rate = 5.20106 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.41139e+10', '# --------------------------------------------', '# total                 = 5.41139e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.69221e-11\trel.res =  1.50984e-15\tbw.error =  3.89873e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0675311', '#   - total flops = 6.8532e+07, min = 38492, max = 2.40848e+07', '#   - flop rate = 1.01482 GFlop/s', '#   - bytes moved = 190.4 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.81945 GByte/s', '#   - solve arithmetic intensity = 0.359936 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,   10.390s; elapsed,   10.426s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   10.390s; elapsed,   10.423s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,   10.400s; elapsed,   10.418s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,   10.400s; elapsed,   10.424s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,   10.380s; elapsed,   10.424s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,   10.400s; elapsed,   10.424s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,   10.410s; elapsed,   10.419s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   10.420s; elapsed,   10.430s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,   10.400s; elapsed,   10.424s, #calls:   1', 'TOTAL                                  : CPU,   10.400s; elapsed,   10.424s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,   10.390s; elapsed,   10.426s, #calls:   1', 'TOTAL                                  : CPU,   10.390s; elapsed,   10.426s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,   10.400s; elapsed,   10.418s, #calls:   1', 'TOTAL                                  : CPU,   10.400s; elapsed,   10.418s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,   10.410s; elapsed,   10.419s, #calls:   1', 'TOTAL                                  : CPU,   10.410s; elapsed,   10.419s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,   10.400s; elapsed,   10.424s, #calls:   1', 'TOTAL                                  : CPU,   10.400s; elapsed,   10.424s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   10.390s; elapsed,   10.423s, #calls:   1', 'TOTAL                                  : CPU,   10.390s; elapsed,   10.423s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,   10.380s; elapsed,   10.424s, #calls:   1', 'TOTAL                                  : CPU,   10.380s; elapsed,   10.424s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   25.820s; elapsed,   25.845s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.190s; elapsed,    0.189s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   18.870s; elapsed,   18.878s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   10.420s; elapsed,   10.430s, #calls:   1', 'TOTAL                                  : CPU,   55.300s; elapsed,   55.340s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 2913.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   32.600s; elapsed,   16.339s', 'individual call time for EIGEN_LDLT: CPU,   19.430s; elapsed,   19.249s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.210s; elapsed,    0.134s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 491,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,761', '#   - number of levels = 130', '#   - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.251934', '#   - matching time = 0.064342', '#   - symmetrization time = 0.026175', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,761', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0743341', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 163.769 MB', '#   - factor time = 10.1306', '#   - factor nonzeros = 20,471,129', '#   - factor memory = 163.769 MB', '#   - total flops = 5.19135e+10, min = 2.03243e+06, max = 2.28029e+10', '#   - flop rate = 5.12445 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.41139e+10', '# --------------------------------------------', '# total                 = 5.41139e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.71507e-11\trel.res =  1.51918e-15\tbw.error =  3.89716e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0607901', '#   - total flops = 6.8532e+07, min = 38492, max = 2.40848e+07', '#   - flop rate = 1.12736 GFlop/s', '#   - bytes moved = 190.413 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 3.13231 GByte/s', '#   - solve arithmetic intensity = 0.359911 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,   12.290s; elapsed,   10.621s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,   14.530s; elapsed,   10.632s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,   15.630s; elapsed,   10.625s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   11.860s; elapsed,   10.630s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,   18.810s; elapsed,   10.630s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,   12.460s; elapsed,   10.630s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,   11.690s; elapsed,   10.625s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   20.970s; elapsed,   10.635s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,   12.290s; elapsed,   10.621s, #calls:   1', 'TOTAL                                  : CPU,   12.290s; elapsed,   10.621s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,   15.630s; elapsed,   10.625s, #calls:   1', 'TOTAL                                  : CPU,   15.630s; elapsed,   10.625s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,   14.530s; elapsed,   10.632s, #calls:   1', 'TOTAL                                  : CPU,   14.530s; elapsed,   10.632s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,   18.810s; elapsed,   10.630s, #calls:   1', 'TOTAL                                  : CPU,   18.810s; elapsed,   10.630s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,   12.460s; elapsed,   10.630s, #calls:   1', 'TOTAL                                  : CPU,   12.460s; elapsed,   10.630s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,   11.690s; elapsed,   10.625s, #calls:   1', 'TOTAL                                  : CPU,   11.690s; elapsed,   10.625s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   11.860s; elapsed,   10.630s, #calls:   1', 'TOTAL                                  : CPU,   11.860s; elapsed,   10.630s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   32.600s; elapsed,   16.339s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.210s; elapsed,    0.134s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   19.430s; elapsed,   19.249s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   20.970s; elapsed,   10.635s, #calls:   1', 'TOTAL                                  : CPU,   73.210s; elapsed,   46.358s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 2913.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   42.160s; elapsed,   10.650s', 'individual call time for EIGEN_LDLT: CPU,   21.940s; elapsed,   21.469s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.350s; elapsed,    0.131s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 491,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,761', '#   - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.250029', '#   - matching time = 0.066143', '#   - symmetrization time = 0.032233', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,761', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0823579', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 163.769 MB', '#   - factor time = 10.4576', '#   - factor nonzeros = 20,471,129', '#   - factor memory = 163.769 MB', '#   - total flops = 5.19135e+10, min = 2.03308e+06, max = 2.28029e+10', '#   - flop rate = 4.96419 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.41139e+10', '# --------------------------------------------', '# total                 = 5.41139e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.71433e-11\trel.res =  1.51888e-15\tbw.error =  3.89716e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0783091', '#   - total flops = 6.8532e+07, min = 38492, max = 2.40848e+07', '#   - flop rate = 0.875148 GFlop/s', '#   - bytes moved = 190.43 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.43178 GByte/s', '#   - solve arithmetic intensity = 0.35988 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,   26.260s; elapsed,   10.984s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,   16.670s; elapsed,   10.991s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,   16.060s; elapsed,   10.982s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,   21.080s; elapsed,   10.993s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   14.840s; elapsed,   10.991s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,   14.370s; elapsed,   10.991s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,   36.750s; elapsed,   10.991s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   43.170s; elapsed,   10.997s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,   16.670s; elapsed,   10.991s, #calls:   1', 'TOTAL                                  : CPU,   16.670s; elapsed,   10.991s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,   26.260s; elapsed,   10.984s, #calls:   1', 'TOTAL                                  : CPU,   26.260s; elapsed,   10.984s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,   16.060s; elapsed,   10.982s, #calls:   1', 'TOTAL                                  : CPU,   16.060s; elapsed,   10.982s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   14.840s; elapsed,   10.991s, #calls:   1', 'TOTAL                                  : CPU,   14.840s; elapsed,   10.991s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,   21.080s; elapsed,   10.993s, #calls:   1', 'TOTAL                                  : CPU,   21.080s; elapsed,   10.993s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,   14.370s; elapsed,   10.991s, #calls:   1', 'TOTAL                                  : CPU,   14.370s; elapsed,   10.991s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,   36.750s; elapsed,   10.991s, #calls:   1', 'TOTAL                                  : CPU,   36.750s; elapsed,   10.991s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   42.160s; elapsed,   10.650s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.350s; elapsed,    0.131s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   21.940s; elapsed,   21.469s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   43.170s; elapsed,   10.997s, #calls:   1', 'TOTAL                                  : CPU,  107.620s; elapsed,   43.246s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 1456.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   26.610s; elapsed,   26.637s', 'individual call time for EIGEN_LDLT: CPU,   20.100s; elapsed,   20.121s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.180s; elapsed,    0.191s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 491,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,761', '#   - number of levels = 129', '#   - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.247728', '#   - matching time = 0.063144', '#   - symmetrization time = 0.0182321', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,761', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0620339', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 163.769 MB', '#   - factor time = 10.3851', '#   - factor nonzeros = 20,471,129', '#   - factor memory = 163.769 MB', '#   - total flops = 5.19135e+10, min = 2.03243e+06, max = 2.27913e+10', '#   - flop rate = 4.99882 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.39135e+10', '# --------------------------------------------', '# total                 = 5.39135e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.04218e-11\trel.res =  1.24402e-15\tbw.error =  3.17516e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0472679', '#   - total flops = 1.09026e+08, min = 31207, max = 2.39472e+07', '#   - flop rate = 2.30654 GFlop/s', '#   - bytes moved = 181.908 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 3.84844 GByte/s', '#   - solve arithmetic intensity = 0.599346 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,   10.780s; elapsed,   10.835s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,   10.810s; elapsed,   10.838s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU,   10.780s; elapsed,   10.838s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU,   10.800s; elapsed,   10.838s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,   10.830s; elapsed,   10.838s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU,   10.800s; elapsed,   10.831s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU,   10.770s; elapsed,   10.837s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,   10.810s; elapsed,   10.837s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU,   10.810s; elapsed,   10.832s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,   10.780s; elapsed,   10.837s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU,   10.770s; elapsed,   10.839s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU,   10.810s; elapsed,   10.837s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU,   10.820s; elapsed,   10.834s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   10.810s; elapsed,   10.838s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,   10.770s; elapsed,   10.837s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   10.820s; elapsed,   10.843s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,   10.810s; elapsed,   10.838s, #calls:   1', 'TOTAL                                  : CPU,   10.810s; elapsed,   10.838s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=12: CPU,   10.800s; elapsed,   10.831s, #calls:   1', 'TOTAL                                  : CPU,   10.800s; elapsed,   10.831s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=15: CPU,   10.800s; elapsed,   10.838s, #calls:   1', 'TOTAL                                  : CPU,   10.800s; elapsed,   10.838s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,   10.830s; elapsed,   10.838s, #calls:   1', 'TOTAL                                  : CPU,   10.830s; elapsed,   10.838s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=8: CPU,   10.770s; elapsed,   10.839s, #calls:   1', 'TOTAL                                  : CPU,   10.770s; elapsed,   10.839s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,   10.780s; elapsed,   10.837s, #calls:   1', 'TOTAL                                  : CPU,   10.780s; elapsed,   10.837s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,   10.810s; elapsed,   10.837s, #calls:   1', 'TOTAL                                  : CPU,   10.810s; elapsed,   10.837s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=9: CPU,   10.810s; elapsed,   10.837s, #calls:   1', 'TOTAL                                  : CPU,   10.810s; elapsed,   10.837s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=13: CPU,   10.770s; elapsed,   10.837s, #calls:   1', 'TOTAL                                  : CPU,   10.770s; elapsed,   10.837s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,   10.770s; elapsed,   10.837s, #calls:   1', 'TOTAL                                  : CPU,   10.770s; elapsed,   10.837s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,   10.780s; elapsed,   10.835s, #calls:   1', 'TOTAL                                  : CPU,   10.780s; elapsed,   10.835s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=11: CPU,   10.810s; elapsed,   10.832s, #calls:   1', 'TOTAL                                  : CPU,   10.810s; elapsed,   10.832s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=14: CPU,   10.780s; elapsed,   10.838s, #calls:   1', 'TOTAL                                  : CPU,   10.780s; elapsed,   10.838s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=10: CPU,   10.820s; elapsed,   10.834s, #calls:   1', 'TOTAL                                  : CPU,   10.820s; elapsed,   10.834s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   10.810s; elapsed,   10.838s, #calls:   1', 'TOTAL                                  : CPU,   10.810s; elapsed,   10.838s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   26.610s; elapsed,   26.637s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.180s; elapsed,    0.191s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   20.100s; elapsed,   20.121s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   10.820s; elapsed,   10.843s, #calls:   1', 'TOTAL                                  : CPU,   57.710s; elapsed,   57.792s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 1456.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   30.310s; elapsed,   15.226s', 'individual call time for EIGEN_LDLT: CPU,   19.680s; elapsed,   19.512s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.230s; elapsed,    0.142s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 16 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 491,337', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,761', '#   - number of levels = 129', '#   - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.266644', '#   - matching time = 0.073559', '#   - symmetrization time = 0.018481', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,761', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.074456', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 163.769 MB', '#   - factor time = 10.0739', '#   - factor nonzeros = 20,471,129', '#   - factor memory = 163.769 MB', '#   - total flops = 5.19135e+10, min = 2.03243e+06, max = 2.27913e+10', '#   - flop rate = 5.15326 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 5.39135e+10', '# --------------------------------------------', '# total                 = 5.39135e+10', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      24454.4\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.01963e-11\trel.res =   1.2348e-15\tbw.error =  3.17394e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.053853', '#   - total flops = 1.09026e+08, min = 31207, max = 2.39472e+07', '#   - flop rate = 2.0245 GFlop/s', '#   - bytes moved = 181.917 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 3.37803 GByte/s', '#   - solve arithmetic intensity = 0.599314 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU,   13.830s; elapsed,   10.587s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU,   12.370s; elapsed,   10.583s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,   11.620s; elapsed,   10.584s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU,   11.960s; elapsed,   10.578s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU,   15.020s; elapsed,   10.579s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU,   12.210s; elapsed,   10.584s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU,   12.020s; elapsed,   10.579s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU,   13.720s; elapsed,   10.585s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,   11.770s; elapsed,   10.582s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,   11.880s; elapsed,   10.583s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,   11.550s; elapsed,   10.584s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   12.070s; elapsed,   10.584s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,   18.500s; elapsed,   10.582s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU,   12.440s; elapsed,   10.584s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,   11.620s; elapsed,   10.582s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   20.860s; elapsed,   10.589s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=15: CPU,   13.830s; elapsed,   10.587s, #calls:   1', 'TOTAL                                  : CPU,   13.830s; elapsed,   10.587s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=12: CPU,   15.020s; elapsed,   10.579s, #calls:   1', 'TOTAL                                  : CPU,   15.020s; elapsed,   10.579s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,   11.550s; elapsed,   10.584s, #calls:   1', 'TOTAL                                  : CPU,   11.550s; elapsed,   10.584s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,   11.620s; elapsed,   10.584s, #calls:   1', 'TOTAL                                  : CPU,   11.620s; elapsed,   10.584s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=13: CPU,   12.370s; elapsed,   10.583s, #calls:   1', 'TOTAL                                  : CPU,   12.370s; elapsed,   10.583s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=10: CPU,   12.020s; elapsed,   10.579s, #calls:   1', 'TOTAL                                  : CPU,   12.020s; elapsed,   10.579s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   12.070s; elapsed,   10.584s, #calls:   1', 'TOTAL                                  : CPU,   12.070s; elapsed,   10.584s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,   11.880s; elapsed,   10.583s, #calls:   1', 'TOTAL                                  : CPU,   11.880s; elapsed,   10.583s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=9: CPU,   12.210s; elapsed,   10.584s, #calls:   1', 'TOTAL                                  : CPU,   12.210s; elapsed,   10.584s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=11: CPU,   11.960s; elapsed,   10.578s, #calls:   1', 'TOTAL                                  : CPU,   11.960s; elapsed,   10.578s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=8: CPU,   12.440s; elapsed,   10.584s, #calls:   1', 'TOTAL                                  : CPU,   12.440s; elapsed,   10.584s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=14: CPU,   13.720s; elapsed,   10.585s, #calls:   1', 'TOTAL                                  : CPU,   13.720s; elapsed,   10.585s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,   11.620s; elapsed,   10.582s, #calls:   1', 'TOTAL                                  : CPU,   11.620s; elapsed,   10.582s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,   11.770s; elapsed,   10.582s, #calls:   1', 'TOTAL                                  : CPU,   11.770s; elapsed,   10.582sExiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,   18.500s; elapsed,   10.582s, #calls:   1', 'TOTAL                                  : CPU,   18.500s; elapsed,   10.582s', '', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   30.310s; elapsed,   15.226s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.230s; elapsed,    0.142s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,   19.680s; elapsed,   19.512s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   20.860s; elapsed,   10.589s, #calls:   1', 'TOTAL                                  : CPU,   71.080s; elapsed,   45.468s']
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-/A_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-/b_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Deff-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    5.660s; elapsed,    5.667s', 'individual call time for EIGEN_LDLT: CPU,    2.470s; elapsed,    2.472s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.053s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    5.660s; elapsed,    5.667s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.053s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.470s; elapsed,    2.472s, #calls:   1', 'TOTAL                                  : CPU,    8.180s; elapsed,    8.191s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    7.620s; elapsed,    3.811s', 'individual call time for EIGEN_LDLT: CPU,    2.620s; elapsed,    2.446s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.080s; elapsed,    0.049s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    7.620s; elapsed,    3.811s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.080s; elapsed,    0.049s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.620s; elapsed,    2.446s, #calls:   1', 'TOTAL                                  : CPU,   10.320s; elapsed,    6.306s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    9.250s; elapsed,    2.317s', 'individual call time for EIGEN_LDLT: CPU,    5.290s; elapsed,    4.744s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.110s; elapsed,    0.046s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    9.250s; elapsed,    2.317s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.110s; elapsed,    0.046s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    5.290s; elapsed,    4.744s, #calls:   1', 'TOTAL                                  : CPU,   14.650s; elapsed,    7.107s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   21.680s; elapsed,    2.719s', 'individual call time for EIGEN_LDLT: CPU,    6.450s; elapsed,    5.228s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.170s; elapsed,    0.041s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   21.680s; elapsed,    2.719s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.170s; elapsed,    0.041s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    6.450s; elapsed,    5.228s, #calls:   1', 'TOTAL                                  : CPU,   28.300s; elapsed,    7.987s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 23309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   36.630s; elapsed,    2.297s', 'individual call time for EIGEN_LDLT: CPU,    8.480s; elapsed,    5.838s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.290s; elapsed,    0.041s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   36.630s; elapsed,    2.297s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.290s; elapsed,    0.041s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    8.480s; elapsed,    5.838s, #calls:   1', 'TOTAL                                  : CPU,   45.400s; elapsed,    8.175s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    5.930s; elapsed,    5.926s', 'individual call time for EIGEN_LDLT: CPU,    2.460s; elapsed,    2.467s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.053s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 255,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 8,139', '#   - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.183663', '#   - matching time = 0.032547', '#   - symmetrization time = 0.016258', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 8,139', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.043304', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 48.2791 MB', '#   - factor time = 4.05603', '#   - factor nonzeros = 6,034,893', '#   - factor memory = 48.2791 MB', '#   - total flops = 7.11168e+09, min = 2.51306e+09, max = 4.59862e+09', '#   - flop rate = 1.75336 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.7315e+09', '# --------------------------------------------', '# total                 = 7.7315e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.18684e-11\trel.res =  1.36409e-15\tbw.error =  3.68017e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0481272', '#   - total flops = 1.45529e+07, min = 6.25352e+06, max = 8.29941e+06', '#   - flop rate = 0.302385 GFlop/s', '#   - bytes moved = 107.14 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.22619 GByte/s', '#   - solve arithmetic intensity = 0.13583 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    4.390s; elapsed,    4.402s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    4.410s; elapsed,    4.413s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    4.390s; elapsed,    4.402s, #calls:   1', 'TOTAL                                  : CPU,    4.390s; elapsed,    4.402s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    5.930s; elapsed,    5.926s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.053s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.460s; elapsed,    2.467s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    4.410s; elapsed,    4.413s, #calls:   1', 'TOTAL                                  : CPU,   12.850s; elapsed,   12.860s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    7.720s; elapsed,    3.860s', 'individual call time for EIGEN_LDLT: CPU,    2.720s; elapsed,    2.550s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.080s; elapsed,    0.049s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 255,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 8,139', '#   - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.189001', '#   - matching time = 0.0323069', '#   - symmetrization time = 0.0165331', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 8,139', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0420849', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 48.2791 MB', '#   - factor time = 2.23998', '#   - factor nonzeros = 6,034,893', '#   - factor memory = 48.2791 MB', '#   - total flops = 7.11174e+09, min = 2.51306e+09, max = 4.59868e+09', '#   - flop rate = 3.17492 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.7315e+09', '# --------------------------------------------', '# total                 = 7.7315e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.16628e-11\trel.res =   1.3553e-15\tbw.error =  3.67931e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0438221', '#   - total flops = 1.45583e+07, min = 6.25352e+06, max = 8.30482e+06', '#   - flop rate = 0.332215 GFlop/s', '#   - bytes moved = 107.353 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.44976 GByte/s', '#   - solve arithmetic intensity = 0.135611 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    5.150s; elapsed,    2.594s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    5.120s; elapsed,    2.604s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    5.150s; elapsed,    2.594s, #calls:   1', 'TOTAL                                  : CPU,    5.150s; elapsed,    2.594s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    7.720s; elapsed,    3.860s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.080s; elapsed,    0.049s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.720s; elapsed,    2.550s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    5.120s; elapsed,    2.604s, #calls:   1', 'TOTAL                                  : CPU,   15.640s; elapsed,    9.063s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    9.880s; elapsed,    2.476s', 'individual call time for EIGEN_LDLT: CPU,    3.110s; elapsed,    2.585s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.110s; elapsed,    0.044s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 255,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 8,139', '#   - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.189412', '#   - matching time = 0.032264', '#   - symmetrization time = 0.0180559', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 8,139', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0430522', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 48.2791 MB', '#   - factor time = 2.12122', '#   - factor nonzeros = 6,034,893', '#   - factor memory = 48.2791 MB', '#   - total flops = 7.11174e+09, min = 2.51306e+09, max = 4.59868e+09', '#   - flop rate = 3.35266 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.7315e+09', '# --------------------------------------------', '# total                 = 7.7315e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.16386e-11\trel.res =  1.35426e-15\tbw.error =  3.67931e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.048625', '#   - total flops = 1.45583e+07, min = 6.25352e+06, max = 8.30482e+06', '#   - flop rate = 0.2994 GFlop/s', '#   - bytes moved = 107.363 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.20798 GByte/s', '#   - solve arithmetic intensity = 0.135599 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    9.840s; elapsed,    2.485s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    9.290s; elapsed,    2.495s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    9.840s; elapsed,    2.485s, #calls:   1', 'TOTAL                                  : CPU,    9.840s; elapsed,    2.485s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    9.880s; elapsed,    2.476s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.110s; elapsed,    0.044s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.110s; elapsed,    2.585s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    9.290s; elapsed,    2.495s, #calls:   1', 'TOTAL                                  : CPU,   22.390s; elapsed,    7.600s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   18.690s; elapsed,    2.344s', 'individual call time for EIGEN_LDLT: CPU,    4.850s; elapsed,    3.577s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.150s; elapsed,    0.043s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 255,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 8,139', '#   - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.177027', '#   - matching time = 0.03299', '#   - symmetrization time = 0.020232', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 8,139', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0554481', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 48.2791 MB', '#   - factor time = 2.16919', '#   - factor nonzeros = 6,034,893', '#   - factor memory = 48.2791 MB', '#   - total flops = 7.11175e+09, min = 2.51306e+09, max = 4.59868e+09', '#   - flop rate = 3.27853 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.7315e+09', '# --------------------------------------------', '# total                 = 7.7315e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =    3.167e-11\trel.res =   1.3556e-15\tbw.error =  3.67931e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.058609', '#   - total flops = 1.45583e+07, min = 6.25352e+06, max = 8.30482e+06', '#   - flop rate = 0.248398 GFlop/s', '#   - bytes moved = 107.383 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 1.8322 GByte/s', '#   - solve arithmetic intensity = 0.135573 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   20.290s; elapsed,    2.552s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   20.480s; elapsed,    2.564s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   20.290s; elapsed,    2.552s, #calls:   1', 'TOTAL                                  : CPU,   20.290s; elapsed,    2.552s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   18.690s; elapsed,    2.344s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.150s; elapsed,    0.043s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    4.850s; elapsed,    3.577s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   20.480s; elapsed,    2.564s, #calls:   1', 'TOTAL                                  : CPU,   44.170s; elapsed,    8.527s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 11654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   38.320s; elapsed,    2.409s', 'individual call time for EIGEN_LDLT: CPU,    5.580s; elapsed,    3.002s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.310s; elapsed,    0.044s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 255,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 8,139', '#   - number of levels = 132', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.206053', '#   - matching time = 0.0501931', '#   - symmetrization time = 0.021385', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 8,139', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0629201', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 48.2791 MB', '#   - factor time = 2.13356', '#   - factor nonzeros = 6,034,893', '#   - factor memory = 48.2791 MB', '#   - total flops = 7.11193e+09, min = 2.51306e+09, max = 4.59887e+09', '#   - flop rate = 3.33337 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.7315e+09', '# --------------------------------------------', '# total                 = 7.7315e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.18243e-11\trel.res =  1.36221e-15\tbw.error =  3.74487e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0608232', '#   - total flops = 1.45583e+07, min = 6.25352e+06, max = 8.30482e+06', '#   - flop rate = 0.239355 GFlop/s', '#   - bytes moved = 107.429 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 1.76625 GByte/s', '#   - solve arithmetic intensity = 0.135516 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   40.120s; elapsed,    2.572s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   40.380s; elapsed,    2.576s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   40.120s; elapsed,    2.572s, #calls:   1', 'TOTAL                                  : CPU,   40.120s; elapsed,    2.572s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   38.320s; elapsed,    2.409s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.310s; elapsed,    0.044s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    5.580s; elapsed,    3.002s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   40.380s; elapsed,    2.576s, #calls:   1', 'TOTAL                                  : CPU,   84.590s; elapsed,    8.031s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    6.300s; elapsed,    6.301s', 'individual call time for EIGEN_LDLT: CPU,    4.790s; elapsed,    4.791s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.060s; elapsed,    0.056s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 255,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 8,139', '#   - number of levels = 131', '#   - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.178712', '#   - matching time = 0.0305891', '#   - symmetrization time = 0.0179451', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 8,139', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0355392', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 48.2791 MB', '#   - factor time = 3.02934', '#   - factor nonzeros = 6,034,893', '#   - factor memory = 48.2791 MB', '#   - total flops = 7.11168e+09, min = 8.93654e+08, max = 2.51306e+09', '#   - flop rate = 2.3476 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.71747e+09', '# --------------------------------------------', '# total                 = 7.71747e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.72392e-11\trel.res =  1.16595e-15\tbw.error =  2.55499e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.041023', '#   - total flops = 1.9286e+07, min = 3.82026e+06, max = 6.22438e+06', '#   - flop rate = 0.470126 GFlop/s', '#   - bytes moved = 94.649 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.30722 GByte/s', '#   - solve arithmetic intensity = 0.203763 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    3.340s; elapsed,    3.348s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    3.340s; elapsed,    3.343s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    3.350s; elapsed,    3.353s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    3.350s; elapsed,    3.356s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    3.340s; elapsed,    3.348s, #calls:   1', 'TOTAL                                  : CPU,    3.340s; elapsed,    3.348s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    3.350s; elapsed,    3.353s, #calls:   1', 'TOTAL                                  : CPU,    3.350s; elapsed,    3.353s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    3.340s; elapsed,    3.343s, #calls:   1', 'TOTAL                                  : CPU,    3.340s; elapsed,    3.343s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    6.300s; elapsed,    6.301s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.060s; elapsed,    0.056s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    4.790s; elapsed,    4.791s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    3.350s; elapsed,    3.356s, #calls:   1', 'TOTAL                                  : CPU,   14.500s; elapsed,   14.504s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    7.750s; elapsed,    3.881s', 'individual call time for EIGEN_LDLT: CPU,    2.750s; elapsed,    2.568s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.080s; elapsed,    0.050s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 255,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 8,139', '#   - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.185792', '#   - matching time = 0.033582', '#   - symmetrization time = 0.018688', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 8,139', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0437679', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 48.2791 MB', '#   - factor time = 1.90523', '#   - factor nonzeros = 6,034,893', '#   - factor memory = 48.2791 MB', '#   - total flops = 7.11169e+09, min = 8.93654e+08, max = 2.51306e+09', '#   - flop rate = 3.73271 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.71747e+09', '# --------------------------------------------', '# total                 = 7.71747e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.71856e-11\trel.res =  1.16365e-15\tbw.error =  2.55499e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0351601', '#   - total flops = 1.9286e+07, min = 3.82026e+06, max = 6.22438e+06', '#   - flop rate = 0.548519 GFlop/s', '#   - bytes moved = 94.654 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.69209 GByte/s', '#   - solve arithmetic intensity = 0.203752 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    3.720s; elapsed,    2.245s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    4.450s; elapsed,    2.243s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    3.960s; elapsed,    2.235s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    4.470s; elapsed,    2.248s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    4.450s; elapsed,    2.243s, #calls:   1', 'TOTAL                                  : CPU,    4.450s; elapsed,    2.243s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    3.960s; elapsed,    2.235s, #calls:   1', 'TOTAL                                  : CPU,    3.960s; elapsed,    2.235s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    3.720s; elapsed,    2.245s, #calls:   1', 'TOTAL                                  : CPU,    3.720s; elapsed,    2.245s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    7.750s; elapsed,    3.881s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.080s; elapsed,    0.050s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.750s; elapsed,    2.568s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    4.470s; elapsed,    2.248s, #calls:   1', 'TOTAL                                  : CPU,   15.050s; elapsed,    8.747s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    9.470s; elapsed,    2.387s', 'individual call time for EIGEN_LDLT: CPU,    3.740s; elapsed,    3.220s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.120s; elapsed,    0.054s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 255,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 8,139', '#   - number of levels = 131', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.211134', '#   - matching time = 0.0458491', '#   - symmetrization time = 0.0200331', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 8,139', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0436361', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 48.2791 MB', '#   - factor time = 2.11703', '#   - factor nonzeros = 6,034,893', '#   - factor memory = 48.2791 MB', '#   - total flops = 7.11169e+09, min = 8.93654e+08, max = 2.51306e+09', '#   - flop rate = 3.35927 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.71747e+09', '# --------------------------------------------', '# total                 = 7.71747e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.71718e-11\trel.res =  1.16306e-15\tbw.error =  2.55499e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0419059', '#   - total flops = 1.9286e+07, min = 3.82026e+06, max = 6.22438e+06', '#   - flop rate = 0.460221 GFlop/s', '#   - bytes moved = 94.6611 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.2589 GByte/s', '#   - solve arithmetic intensity = 0.203737 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    6.650s; elapsed,    2.507s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    8.360s; elapsed,    2.497s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    9.850s; elapsed,    2.504s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    9.720s; elapsed,    2.510s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    8.360s; elapsed,    2.497s, #calls:   1', 'TOTAL                                  : CPU,    8.360s; elapsed,    2.497s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    6.650s; elapsed,    2.507s, #calls:   1', 'TOTAL                                  : CPU,    6.650s; elapsed,    2.507s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    9.850s; elapsed,    2.504s, #calls:   1', 'TOTAL                                  : CPU,    9.850s; elapsed,    2.504s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    9.470s; elapsed,    2.387s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.120s; elapsed,    0.054s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.740s; elapsed,    3.220s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    9.720s; elapsed,    2.510s, #calls:   1', 'TOTAL                                  : CPU,   23.050s; elapsed,    8.170s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 5827.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   18.120s; elapsed,    2.270s', 'individual call time for EIGEN_LDLT: CPU,    3.720s; elapsed,    2.486s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.160s; elapsed,    0.042s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 255,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 8,139', '#   - number of levels = 131', '#   - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.205088', '#   - matching time = 0.0374961', '#   - symmetrization time = 0.020402', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 8,139', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0452731', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 48.2791 MB', '#   - factor time = 2.12288', '#   - factor nonzeros = 6,034,893', '#   - factor memory = 48.2791 MB', '#   - total flops = 7.11169e+09, min = 8.93654e+08, max = 2.51306e+09', '#   - flop rate = 3.35002 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.71747e+09', '# --------------------------------------------', '# total                 = 7.71747e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.71495e-11\trel.res =   1.1621e-15\tbw.error =  2.55499e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0447309', '#   - total flops = 1.9286e+07, min = 3.82026e+06, max = 6.22438e+06', '#   - flop rate = 0.431155 GFlop/s', '#   - bytes moved = 94.6754 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.11655 GByte/s', '#   - solve arithmetic intensity = 0.203706 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,   12.370s; elapsed,    2.500s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,   15.620s; elapsed,    2.492s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   19.630s; elapsed,    2.498s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   19.750s; elapsed,    2.504s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,   15.620s; elapsed,    2.492s, #calls:   1', 'TOTAL                                  : CPU,   15.620s; elapsed,    2.492s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   19.630s; elapsed,    2.498s, #calls:   1', 'TOTAL                                  : CPU,   19.630s; elapsed,    2.498s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,   12.370s; elapsed,    2.500s, #calls:   1', 'TOTAL                                  : CPU,   12.370s; elapsed,    2.500s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   18.120s; elapsed,    2.270s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.160s; elapsed,    0.042s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.720s; elapsed,    2.486s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   19.750s; elapsed,    2.504s, #calls:   1', 'TOTAL                                  : CPU,   41.750s; elapsed,    7.302s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 2913.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    6.290s; elapsed,    6.306s', 'individual call time for EIGEN_LDLT: CPU,    2.640s; elapsed,    2.634s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.066s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', '# using 8 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 255,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 8,141', '#   - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.210765', '#   - matching time = 0.035332', '#   - symmetrization time = 0.0226789', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 8,141', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0359058', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.8475 MB', '#   - factor time = 2.63446', '#   - factor nonzeros = 6,230,935', '#   - factor memory = 49.8475 MB', '#   - total flops = 7.56445e+09, min = 227903, max = 2.51361e+09', '#   - flop rate = 2.87135 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.14808e+09', '# --------------------------------------------', '# total                 = 8.14808e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.58238e-11\trel.res =   1.5334e-15\tbw.error =  2.79918e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0461409', '#   - total flops = 2.27636e+07, min = 23029, max = 6.19462e+06', '#   - flop rate = 0.493349 GFlop/s', '#   - bytes moved = 95.0808 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.06066 GByte/s', '#   - solve arithmetic intensity = 0.239413 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    2.970s; elapsed,    3.006s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    2.970s; elapsed,    3.004s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    2.970s; elapsed,    3.005s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    2.960s; elapsed,    3.007s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    2.990s; elapsed,    3.003s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    2.980s; elapsed,    2.995s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    2.970s; elapsed,    3.005s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    3.010s; elapsed,    3.009s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    2.970s; elapsed,    3.005s, #calls:   1', 'TOTAL                                  : CPU,    2.970s; elapsed,    3.005s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    2.960s; elapsed,    3.007s, #calls:   1', 'TOTAL                                  : CPU,    2.960s; elapsed,    3.007s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    2.990s; elapsed,    3.003s, #calls:   1', 'TOTAL                                  : CPU,    2.990s; elapsed,    3.003s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    2.980s; elapsed,    2.995s, #calls:   1', 'TOTAL                                  : CPU,    2.980s; elapsed,    2.995s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    2.970s; elapsed,    3.006s, #calls:   1', 'TOTAL                                  : CPU,    2.970s; elapsed,    3.006s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    2.970s; elapsed,    3.005s, #calls:   1', 'TOTAL                                  : CPU,    2.970s; elapsed,    3.005s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    2.970s; elapsed,    3.004s, #calls:   1', 'TOTAL                                  : CPU,    2.970s; elapsed,    3.004s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    6.290s; elapsed,    6.306s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.066s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.640s; elapsed,    2.634s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    3.010s; elapsed,    3.009s, #calls:   1', 'TOTAL                                  : CPU,   12.010s; elapsed,   12.015s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 2913.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    7.670s; elapsed,    3.840s', 'individual call time for EIGEN_LDLT: CPU,    2.770s; elapsed,    2.580s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.049s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 255,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 8,141', '#   - number of levels = 130', '#   - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.186872', '#   - matching time = 0.0331202', '#   - symmetrization time = 0.0211761', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 8,141', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.039819', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.8475 MB', '#   - factor time = 2.29259', '#   - factor nonzeros = 6,230,935', '#   - factor memory = 49.8475 MB', '#   - total flops = 7.56446e+09, min = 227903, max = 2.51361e+09', '#   - flop rate = 3.29953 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.14808e+09', '# --------------------------------------------', '# total                 = 8.14808e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.57235e-11\trel.res =  1.52911e-15\tbw.error =  2.80249e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0422561', '#   - total flops = 2.27636e+07, min = 23029, max = 6.19462e+06', '#   - flop rate = 0.538705 GFlop/s', '#   - bytes moved = 95.089 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.2503 GByte/s', '#   - solve arithmetic intensity = 0.239392 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    3.450s; elapsed,    2.640s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    5.160s; elapsed,    2.640s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    3.670s; elapsed,    2.641s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    4.230s; elapsed,    2.641s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    3.410s; elapsed,    2.640s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    3.410s; elapsed,    2.641s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    4.390s; elapsed,    2.632s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    5.230s; elapsed,    2.645s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    3.410s; elapsed,    2.641s, #calls:   1', 'TOTAL                                  : CPU,    3.410s; elapsed,    2.641s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    5.160s; elapsed,    2.640s, #calls:   1', 'TOTAL                                  : CPU,    5.160s; elapsed,    2.640s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    4.230s; elapsed,    2.641s, #calls:   1', 'TOTAL                                  : CPU,    4.230s; elapsed,    2.641s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    3.410s; elapsed,    2.640s, #calls:   1', 'TOTAL                                  : CPU,    3.410s; elapsed,    2.640s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    3.670s; elapsed,    2.641s, #calls:   1', 'TOTAL                                  : CPU,    3.670s; elapsed,    2.641s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    4.390s; elapsed,    2.632s, #calls:   1', 'TOTAL                                  : CPU,    4.390s; elapsed,    2.632s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    3.450s; elapsed,    2.640s, #calls:   1', 'TOTAL                                  : CPU,    3.450s; elapsed,    2.640s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    7.670s; elapsed,    3.840s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.049s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.770s; elapsed,    2.580s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    5.230s; elapsed,    2.645s, #calls:   1', 'TOTAL                                  : CPU,   15.740s; elapsed,    9.114s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 2913.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   15.160s; elapsed,    3.819s', 'individual call time for EIGEN_LDLT: CPU,    3.370s; elapsed,    2.847s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.120s; elapsed,    0.048s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3CPP -2', 'CPP -2', 'CPP -2', '', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 255,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 8,141', '#   - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.234132', '#   - matching time = 0.0391309', '#   - symmetrization time = 0.0235999', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 8,141', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0450461', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.8475 MB', '#   - factor time = 2.41375', '#   - factor nonzeros = 6,230,935', '#   - factor memory = 49.8475 MB', '#   - total flops = 7.56446e+09, min = 228737, max = 2.51361e+09', '#   - flop rate = 3.13391 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.14808e+09', '# --------------------------------------------', '# total                 = 8.14808e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =   3.5714e-11\trel.res =   1.5287e-15\tbw.error =  2.80816e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.049768', '#   - total flops = 2.27636e+07, min = 23029, max = 6.19462e+06', '#   - flop rate = 0.457394 GFlop/s', '#   - bytes moved = 95.0992 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 1.91085 GByte/s', '#   - solve arithmetic intensity = 0.239366 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    7.160s; elapsed,    2.833s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    5.950s; elapsed,    2.832s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    5.170s; elapsed,    2.831s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    8.520s; elapsed,    2.822s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    5.370s; elapsed,    2.831s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    5.280s; elapsed,    2.831s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,   10.820s; elapsed,    2.831s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   11.010s; elapsed,    2.836s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    5.170s; elapsed,    2.831s, #calls:   1', 'TOTAL                                  : CPU,    5.170s; elapsed,    2.831s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    8.520s; elapsed,    2.822s, #calls:   1', 'TOTAL                                  : CPU,    8.520s; elapsed,    2.822s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    7.160s; elapsed,    2.833s, #calls:   1', 'TOTAL                                  : CPU,    7.160s; elapsed,    2.833s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    5.950s; elapsed,    2.832s, #calls:   1', 'TOTAL                                  : CPU,    5.950s; elapsed,    2.832s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    5.280s; elapsed,    2.831s, #calls:   1', 'TOTAL                                  : CPU,    5.280s; elapsed,    2.831s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,   10.820s; elapsed,    2.831s, #calls:   1', 'TOTAL                                  : CPU,   10.820s; elapsed,    2.831s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    5.370s; elapsed,    2.831s, #calls:   1', 'TOTAL                                  : CPU,    5.370s; elapsed,    2.831s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   15.160s; elapsed,    3.819s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.120s; elapsed,    0.048s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.370s; elapsed,    2.847s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   11.010s; elapsed,    2.836s, #calls:   1', 'TOTAL                                  : CPU,   29.660s; elapsed,    9.550s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 1456.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    6.440s; elapsed,    6.444s', 'individual call time for EIGEN_LDLT: CPU,    2.660s; elapsed,    2.661s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.067s', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 255,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 8,135', '#   - number of levels = 129', '#   - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.20999', '#   - matching time = 0.0366721', '#   - symmetrization time = 0.0154288', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 8,135', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0342059', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 48.9364 MB', '#   - factor time = 2.07525', '#   - factor nonzeros = 6,117,051', '#   - factor memory = 48.9364 MB', '#   - total flops = 7.3195e+09, min = 381751, max = 2.51086e+09', '#   - flop rate = 3.52705 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.85078e+09', '# --------------------------------------------', '# total                 = 7.85078e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.63666e-11\trel.res =  1.12859e-15\tbw.error =  2.13979e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.037606', '#   - total flops = 3.59797e+07, min = 18684, max = 6.128e+06', '#   - flop rate = 0.956754 GFlop/s', '#   - bytes moved = 90.6956 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.41173 GByte/s', '#   - solve arithmetic intensity = 0.396708 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU,    2.410s; elapsed,    2.426s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    2.390s; elapsed,    2.424s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    2.380s; elapsed,    2.425s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    2.410s; elapsed,    2.423s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU,    2.360s; elapsed,    2.418s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU,    2.370s; elapsed,    2.426s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU,    2.380s; elapsed,    2.424s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    2.360s; elapsed,    2.424s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    2.380s; elapsed,    2.423s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    2.350s; elapsed,    2.425s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU,    2.360s; elapsed,    2.424s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU,    2.390s; elapsed,    2.421s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU,    2.420s; elapsed,    2.426s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU,    2.390s; elapsed,    2.424s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    2.410s; elapsed,    2.424s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    2.410s; elapsed,    2.430s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=12: CPU,    2.370s; elapsed,    2.426s, #calls:   1', 'TOTAL                                  : CPU,    2.370s; elapsed,    2.426s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=9: CPU,    2.410s; elapsed,    2.426s, #calls:   1', 'TOTAL                                  : CPU,    2.410s; elapsed,    2.426s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    2.380s; elapsed,    2.425s, #calls:   1', 'TOTAL                                  : CPU,    2.380s; elapsed,    2.425s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    2.350s; elapsed,    2.425s, #calls:   1', 'TOTAL                                  : CPU,    2.350s; elapsed,    2.425s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    2.360s; elapsed,    2.424s, #calls:   1', 'TOTAL                                  : CPU,    2.360s; elapsed,    2.424s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=13: CPU,    2.380s; elapsed,    2.424s, #calls:   1', 'TOTAL                                  : CPU,    2.380s; elapsed,    2.424s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=14: CPU,    2.420s; elapsed,    2.426s, #calls:   1', 'TOTAL                                  : CPU,    2.420s; elapsed,    2.426s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=8: CPU,    2.360s; elapsed,    2.424s, #calls:   1', 'TOTAL                                  : CPU,    2.360s; elapsed,    2.424s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    2.390s; elapsed,    2.424s, #calls:   1', 'TOTAL                                  : CPU,    2.390s; elapsed,    2.424s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=11: CPU,    2.360s; elapsed,    2.418s, #calls:   1', 'TOTAL                                  : CPU,    2.360s; elapsed,    2.418s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=10: CPU,    2.390s; elapsed,    2.421s, #calls:   1', 'TOTAL                                  : CPU,    2.390s; elapsed,    2.421s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    2.410s; elapsed,    2.423s, #calls:   1', 'TOTAL                                  : CPU,    2.410s; elapsed,    2.423s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=15: CPU,    2.390s; elapsed,    2.424s, #calls:   1', 'TOTAL                                  : CPU,    2.390s; elapsed,    2.424s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    2.380s; elapsed,    2.423s, #calls:   1', 'TOTAL                                  : CPU,    2.380s; elapsed,    2.423s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    2.410s; elapsed,    2.424s, #calls:   1', 'TOTAL                                  : CPU,    2.410s; elapsed,    2.424s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    6.440s; elapsed,    6.444s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.067s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.660s; elapsed,    2.661s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    2.410s; elapsed,    2.430s, #calls:   1', 'TOTAL                                  : CPU,   11.580s; elapsed,   11.601s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '23309 1456.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    8.240s; elapsed,    4.132s', 'individual call time for EIGEN_LDLT: CPU,    2.780s; elapsed,    2.604s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.080s; elapsed,    0.061s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 23,309', '#   - number of nonzeros = 255,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 8,135', '#   - number of levels = 129', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '#   - nd time = # ******************************************************************', '0.220307', '#   - matching time = 0.045186', '#   - symmetrization time = 0.0156379', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 8,135', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0388739', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 48.9364 MB', '#   - factor time = 2.13793', '#   - factor nonzeros = 6,117,051', '#   - factor memory = 48.9364 MB', '#   - total flops = 7.3195e+09, min = 381751, max = 2.51086e+09', '#   - flop rate = 3.42364 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.85078e+09', '# --------------------------------------------', '# total                 = 7.85078e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =   2.6397e-11\trel.res =   1.1299e-15\tbw.error =  2.13979e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0360532', '#   - total flops = 3.59797e+07, min = 18684, max = 6.128e+06', '#   - flop rate = 0.997962 GFlop/s', '#   - bytes moved = 90.7028 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.51581 GByte/s', '#   - solve arithmetic intensity = 0.396677 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    3.380s; elapsed,    2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU,    3.220s; elapsed,    2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU,    3.490s; elapsed,    2.509s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU,    3.970s; elapsed,    2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU,    3.980s; elapsed,    2.514s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    3.340s; elapsed,    2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    3.310s; elapsed,    2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    4.880s; elapsed,    2.515s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU,    3.450s; elapsed,    2.514s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU,    4.010s; elapsed,    2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU,    3.240s; elapsed,    2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU,    4.300s; elapsed,    2.511s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    3.310s; elapsed,    2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    3.330s; elapsed,    2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    3.280s; elapsed,    2.513s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    4.890s; elapsed,    2.519s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    4.880s; elapsed,    2.515s, #calls:   1', 'TOTAL                                  : CPU,    4.880s; elapsed,    2.515s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=14: CPU,    3.970s; elapsed,    2.513s, #calls:   1', 'TOTAL                                  : CPU,    3.970s; elapsed,    2.513s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=9: CPU,    3.220s; elapsed,    2.513s, #calls:   1', 'TOTAL                                  : CPU,    3.220s; elapsed,    2.513s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=15: CPU,    3.980s; elapsed,    2.514s, #calls:   1', 'TOTAL                                  : CPU,    3.980s; elapsed,    2.514s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    3.310s; elapsed,    2.513s, #calls:   1', 'TOTAL                                  : CPU,    3.310s; elapsed,    2.513s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    3.380s; elapsed,    2.513s, #calls:   1', 'TOTAL                                  : CPU,    3.380s; elapsed,    2.513s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    3.340s; elapsed,    2.513s, #calls:   1', 'TOTAL                                  : CPU,    3.340s; elapsed,    2.513s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    3.330s; elapsed,    2.513s, #calls:   1', 'TOTAL                                  : CPU,    3.330s; elapsed,    2.513s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=12: CPU,    3.450s; elapsed,    2.514s, #calls:   1', 'TOTAL                                  : CPU,    3.450s; elapsed,    2.514s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=8: CPU,    3.240s; elapsed,    2.513s, #calls:   1', 'TOTAL                                  : CPU,    3.240s; elapsed,    2.513s', 'Exiting profiler', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=10: CPU,    4.300s; elapsed,    2.511s, #calls:   1', 'TOTAL                                  : CPU,    4.300s; elapsed,    2.511s', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    3.310s; elapsed,    2.513s, #calls:   1', 'TOTAL                                  : CPU,    3.310s; elapsed,    2.513s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=13: CPU,    4.010s; elapsed,    2.513s, #calls:   1', 'TOTAL                                  : CPU,    4.010s; elapsed,    2.513s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=11: CPU,    3.490s; elapsed,    2.509s, #calls:   1', 'TOTAL                                  : CPU,    3.490s; elapsed,    2.509s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    3.280s; elapsed,    2.513s, #calls:   1', 'TOTAL                                  : CPU,    3.280s; elapsed,    2.513s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    8.240s; elapsed,    4.132s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.080s; elapsed,    0.061s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.780s; elapsed,    2.604s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    4.890s; elapsed,    2.519s, #calls:   1', 'TOTAL                                  : CPU,   15.990s; elapsed,    9.316s']
Data Set Name:=strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-
set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-/A_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-.csv']) set(['/net/dials/raid1/mlxd/InDev/feb_sprint/sam/samosa/step4K_samosa_debug_5k/out_strumpack_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-/b_strum_5k_omp1_paramslevmar.parameter_flags=Rxy-levmar.parameter_flags=Eta-.csv'])
mpirun -n 1
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    5.620s; elapsed,    5.636s', 'individual call time for EIGEN_LDLT: CPU,    2.510s; elapsed,    2.499s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.056s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    5.620s; elapsed,    5.636s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.056s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.510s; elapsed,    2.499s, #calls:   1', 'TOTAL                                  : CPU,    8.180s; elapsed,    8.191s']
mpirun -n 1
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    6.840s; elapsed,    3.420s', 'individual call time for EIGEN_LDLT: CPU,    2.640s; elapsed,    2.463s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.080s; elapsed,    0.045s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    6.840s; elapsed,    3.420s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.080s; elapsed,    0.045s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.640s; elapsed,    2.463s, #calls:   1', 'TOTAL                                  : CPU,    9.560s; elapsed,    5.928s']
mpirun -n 1
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    9.130s; elapsed,    2.289s', 'individual call time for EIGEN_LDLT: CPU,    3.130s; elapsed,    2.598s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.100s; elapsed,    0.042s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    9.130s; elapsed,    2.289s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.100s; elapsed,    0.042s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.130s; elapsed,    2.598s, #calls:   1', 'TOTAL                                  : CPU,   12.360s; elapsed,    4.929s']
mpirun -n 1
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   19.060s; elapsed,    2.387s', 'individual call time for EIGEN_LDLT: CPU,    6.800s; elapsed,    5.530s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.140s; elapsed,    0.042s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   19.060s; elapsed,    2.387s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.140s; elapsed,    0.042s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    6.800s; elapsed,    5.530s, #calls:   1', 'TOTAL                                  : CPU,   26.000s; elapsed,    7.959s']
mpirun -n 1
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 21309.0', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   43.340s; elapsed,    2.718s', 'individual call time for EIGEN_LDLT: CPU,    7.950s; elapsed,    5.299s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.260s; elapsed,    0.038s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   43.340s; elapsed,    2.718s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.260s; elapsed,    0.038s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    7.950s; elapsed,    5.299s, #calls:   1', 'TOTAL                                  : CPU,   51.550s; elapsed,    8.054s']
mpirun -n 2
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    5.630s; elapsed,    5.627s', 'individual call time for EIGEN_LDLT: CPU,    2.420s; elapsed,    2.423s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.055s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,927', '#   - number of levels = 130', '#   - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.161156', '#   - matching time = 0.029036', '#   - symmetrization time = 0.015054', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,927', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0424871', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.1376 MB', '#   - factor time = 3.92202', '#   - factor nonzeros = 6,142,201', '#   - factor memory = 49.1376 MB', '#   - total flops = 7.46369e+09, min = 2.94438e+09, max = 4.51931e+09', '#   - flop rate = 1.90302 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.06944e+09', '# --------------------------------------------', '# total                 = 8.06944e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =   3.6413e-11\trel.res =  1.55862e-15\tbw.error =  3.62098e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0474091', '#   - total flops = 1.47533e+07, min = 6.98046e+06, max = 7.77288e+06', '#   - flop rate = 0.311193 GFlop/s', '#   - bytes moved = 105.81 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.23185 GByte/s', '#   - solve arithmetic intensity = 0.139433 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    4.230s; elapsed,    4.237s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    4.250s; elapsed,    4.247s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    4.230s; elapsed,    4.237s, #calls:   1', 'TOTAL                                  : CPU,    4.230s; elapsed,    4.237s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    5.630s; elapsed,    5.627s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.055s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.420s; elapsed,    2.423s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    4.250s; elapsed,    4.247s, #calls:   1', 'TOTAL                                  : CPU,   12.350s; elapsed,   12.352s']
mpirun -n 2
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    7.090s; elapsed,    3.549s', 'individual call time for EIGEN_LDLT: CPU,    2.760s; elapsed,    2.577s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.046s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 2 MPI processesCPP -2', '', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,927', '#   - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.173755', '#   - matching time = 0.031755', '#   - symmetrization time = 0.0164971', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,927', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0449631', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.1376 MB', '#   - factor time = 3.3272', '#   - factor nonzeros = 6,142,201', '#   - factor memory = 49.1376 MB', '#   - total flops = 7.46376e+09, min = 2.94438e+09, max = 4.51938e+09', '#   - flop rate = 2.24325 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.06944e+09', '# --------------------------------------------', '# total                 = 8.06944e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.67124e-11\trel.res =  1.57144e-15\tbw.error =  3.63192e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0456979', '#   - total flops = 1.47589e+07, min = 6.98046e+06, max = 7.77839e+06', '#   - flop rate = 0.322965 GFlop/s', '#   - bytes moved = 106.027 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.32017 GByte/s', '#   - solve arithmetic intensity = 0.139199 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    6.790s; elapsed,    3.666s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    7.350s; elapsed,    3.676s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    6.790s; elapsed,    3.666s, #calls:   1', 'TOTAL                                  : CPU,    6.790s; elapsed,    3.666s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    7.090s; elapsed,    3.549s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.046s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.760s; elapsed,    2.577s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    7.350s; elapsed,    3.676s, #calls:   1', 'TOTAL                                  : CPU,   17.270s; elapsed,    9.848s']
mpirun -n 2
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    9.840s; elapsed,    2.468s', 'individual call time for EIGEN_LDLT: CPU,    3.070s; elapsed,    2.529s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.100s; elapsed,    0.043s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', '# using CPP -2', '2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,927', '#   - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.177125', '#   - matching time = 0.032584', '#   - symmetrization time = 0.0162432', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,927', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0469749', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.1376 MB', '#   - factor time = 2.17574', '#   - factor nonzeros = 6,142,201', '#   - factor memory = 49.1376 MB', '#   - total flops = 7.46376e+09, min = 2.94438e+09, max = 4.51938e+09', '#   - flop rate = 3.43045 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.06944e+09', '# --------------------------------------------', '# total                 = 8.06944e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =   3.6699e-11\trel.res =  1.57086e-15\tbw.error =  3.63192e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.053381', '#   - total flops = 1.47589e+07, min = 6.98046e+06, max = 7.77839e+06', '#   - flop rate = 0.276482 GFlop/s', '#   - bytes moved = 106.038 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 1.98644 GByte/s', '#   - solve arithmetic intensity = 0.139184 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   10.090s; elapsed,    2.531s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   10.090s; elapsed,    2.541s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   10.090s; elapsed,    2.531s, #calls:   1', 'TOTAL                                  : CPU,   10.090s; elapsed,    2.531s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    9.840s; elapsed,    2.468s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.100s; elapsed,    0.043s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.070s; elapsed,    2.529s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   10.090s; elapsed,    2.541s, #calls:   1', 'TOTAL                                  : CPU,   23.100s; elapsed,    7.582s']
mpirun -n 2
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   19.660s; elapsed,    2.462s', 'individual call time for EIGEN_LDLT: CPU,    7.270s; elapsed,    6.028s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.160s; elapsed,    0.042s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', '# using 2 MPI processes', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,927', '#   - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.19543', '#   - matching time = 0.036809', '#   - symmetrization time = 0.019227', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,927', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0593681', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.1376 MB', '#   - factor time = 2.39285', '#   - factor nonzeros = 6,142,201', '#   - factor memory = 49.1376 MB', '#   - total flops = 7.46376e+09, min = 2.94438e+09, max = 4.51938e+09', '#   - flop rate = 3.1192 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.06944e+09', '# --------------------------------------------', '# total                 = 8.06944e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.66517e-11\trel.res =  1.56884e-15\tbw.error =  3.63192e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0481601', '#   - total flops = 1.47589e+07, min = 6.98046e+06, max = 7.77839e+06', '#   - flop rate = 0.306454 GFlop/s', '#   - bytes moved = 106.061 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.20225 GByte/s', '#   - solve arithmetic intensity = 0.139155 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   18.970s; elapsed,    2.790s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   22.140s; elapsed,    2.801s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   18.970s; elapsed,    2.790s, #calls:   1', 'TOTAL                                  : CPU,   18.970s; elapsed,    2.790s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   19.660s; elapsed,    2.462s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.160s; elapsed,    0.042s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    7.270s; elapsed,    6.028s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   22.140s; elapsed,    2.801s, #calls:   1', 'TOTAL                                  : CPU,   49.230s; elapsed,   11.333s']
mpirun -n 2
OMP_NUM_THREADS:=16
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 10654.5', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   41.400s; elapsed,    2.597s', 'individual call time for EIGEN_LDLT: CPU,    7.940s; elapsed,    5.317s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.270s; elapsed,    0.040s', 'CPP -2', '# Initializing STRUMPACK', '# using 16 OpenMP threads', '# number of tasking levels = 7 = log_2(#threads) + 3', 'CPP -2', '# using 2 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,927', '#   - number of levels = 130', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.215598', '#   - matching time = 0.0398321', '#   - symmetrization time = 0.0203738', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,927', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0602891', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.1376 MB', '#   - factor time = 2.16204', '#   - factor nonzeros = 6,142,201', '#   - factor memory = 49.1376 MB', '#   - total flops = 7.46395e+09, min = 2.94438e+09, max = 4.51957e+09', '#   - flop rate = 3.45227 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.06944e+09', '# --------------------------------------------', '# total                 = 8.06944e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.65568e-11\trel.res =  1.56478e-15\tbw.error =  3.57131e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.061873', '#   - total flops = 1.47589e+07, min = 6.98046e+06, max = 7.77839e+06', '#   - flop rate = 0.238535 GFlop/s', '#   - bytes moved = 106.109 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 1.71494 GByte/s', '#   - solve arithmetic intensity = 0.139092 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,   40.160s; elapsed,    2.593s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   40.890s; elapsed,    2.603s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,   40.160s; elapsed,    2.593s, #calls:   1', 'TOTAL                                  : CPU,   40.160s; elapsed,    2.593s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   41.400s; elapsed,    2.597s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.270s; elapsed,    0.040s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    7.940s; elapsed,    5.317s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   40.890s; elapsed,    2.603s, #calls:   1', 'TOTAL                                  : CPU,   90.500s; elapsed,   10.557s']
mpirun -n 4
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    6.250s; elapsed,    6.255s', 'individual call time for EIGEN_LDLT: CPU,    2.640s; elapsed,    2.643s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.060s; elapsed,    0.060s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,931', '#   - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.191609', '#   - matching time = 0.0349221', '#   - symmetrization time = 0.0150561', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,931', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.035259', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.3058 MB', '#   - factor time = 2.74773', '#   - factor nonzeros = 6,163,223', '#   - factor memory = 49.3058 MB', '#   - total flops = 7.53221e+09, min = 865688, max = 2.95202e+09', '#   - flop rate = 2.74125 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.13207e+09', '# --------------------------------------------', '# total                 = 8.13207e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.02608e-11\trel.res =  1.29528e-15\tbw.error =  2.43099e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0377111', '#   - total flops = 1.6493e+07, min = 43055, max = 6.92318e+06', '#   - flop rate = 0.43735 GFlop/s', '#   - bytes moved = 98.2063 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.60417 GByte/s', '#   - solve arithmetic intensity = 0.167942 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    3.050s; elapsed,    3.070s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    3.060s; elapsed,    3.074s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    3.050s; elapsed,    3.072s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    3.070s; elapsed,    3.079s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    3.050s; elapsed,    3.072s, #calls:   1', 'TOTAL                                  : CPU,    3.050s; elapsed,    3.072s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    3.050s; elapsed,    3.070s, #calls:   1', 'TOTAL                                  : CPU,    3.050s; elapsed,    3.070s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    3.060s; elapsed,    3.074s, #calls:   1', 'TOTAL                                  : CPU,    3.060s; elapsed,    3.074s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    6.250s; elapsed,    6.255s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.060s; elapsed,    0.060s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.640s; elapsed,    2.643s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    3.070s; elapsed,    3.079s, #calls:   1', 'TOTAL                                  : CPU,   12.020s; elapsed,   12.037s']
mpirun -n 4
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    7.760s; elapsed,    3.888s', 'individual call time for EIGEN_LDLT: CPU,    2.900s; elapsed,    2.718s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.050s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', '# using 4 MPI processes', 'CPP -2', 'CPP -2', 'CPP -2', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,931', '#   - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.#   - nd time = ', '# ******************************************************************', '0.187901', '#   - matching time = 0.0358179', '#   - symmetrization time = 0.0150061', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,931', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0474441', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.3058 MB', '#   - factor time = 2.06362', '#   - factor nonzeros = 6,163,223', '#   - factor memory = 49.3058 MB', '#   - total flops = 7.53223e+09, min = 865688, max = 2.95202e+09', '#   - flop rate = 3.65001 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.13207e+09', '# --------------------------------------------', '# total                 = 8.13207e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.01024e-11\trel.res =   1.2885e-15\tbw.error =  2.41537e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0345149', '#   - total flops = 1.64951e+07, min = 43055, max = 6.92318e+06', '#   - flop rate = 0.477913 GFlop/s', '#   - bytes moved = 98.2807 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.84749 GByte/s', '#   - solve arithmetic intensity = 0.167837 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    4.430s; elapsed,    2.405s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    4.760s; elapsed,    2.406s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    3.090s; elapsed,    2.410s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    4.760s; elapsed,    2.414s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    4.430s; elapsed,    2.405s, #calls:   1', 'TOTAL                                  : CPU,    4.430s; elapsed,    2.405s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    4.760s; elapsed,    2.406s, #calls:   1', 'TOTAL                                  : CPU,    4.760s; elapsed,    2.406s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    3.090s; elapsed,    2.410s, #calls:   1', 'TOTAL                                  : CPU,    3.090s; elapsed,    2.410s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    7.760s; elapsed,    3.888s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.050s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.900s; elapsed,    2.718s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    4.760s; elapsed,    2.414s, #calls:   1', 'TOTAL                                  : CPU,   15.490s; elapsed,    9.070s']
mpirun -n 4
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   12.170s; elapsed,    3.051s', 'individual call time for EIGEN_LDLT: CPU,    3.270s; elapsed,    2.743s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.110s; elapsed,    0.047s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,931', '#   - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.20429', '#   - matching time = 0.0361559', '#   - symmetrization time = 0.015136', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,931', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0423629', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.3058 MB', '#   - factor time = 3.18081', '#   - factor nonzeros = 6,163,223', '#   - factor memory = 49.3058 MB', '#   - total flops = 7.53223e+09, min = 866384, max = 2.95202e+09', '#   - flop rate = 2.36803 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.13207e+09', '# --------------------------------------------', '# total                 = 8.13207e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.01136e-11\trel.res =  1.28898e-15\tbw.error =  2.43099e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.04076', '#   - total flops = 1.64951e+07, min = 43055, max = 6.92318e+06', '#   - flop rate = 0.404689 GFlop/s', '#   - bytes moved = 98.2891 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.41141 GByte/s', '#   - solve arithmetic intensity = 0.167822 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    9.240s; elapsed,    3.535s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    5.600s; elapsed,    3.539s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,   11.360s; elapsed,    3.536s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   14.010s; elapsed,    3.544s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    5.600s; elapsed,    3.539s, #calls:   1', 'TOTAL                                  : CPU,    5.600s; elapsed,    3.539s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    9.240s; elapsed,    3.535s, #calls:   1', 'TOTAL                                  : CPU,    9.240s; elapsed,    3.535s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,   11.360s; elapsed,    3.536s, #calls:   1', 'TOTAL                                  : CPU,   11.360s; elapsed,    3.536s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   12.170s; elapsed,    3.051s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.110s; elapsed,    0.047s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.270s; elapsed,    2.743s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   14.010s; elapsed,    3.544s, #calls:   1', 'TOTAL                                  : CPU,   29.560s; elapsed,    9.385s']
mpirun -n 4
OMP_NUM_THREADS:=8
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 5327.25', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   18.260s; elapsed,    2.288s', 'individual call time for EIGEN_LDLT: CPU,    3.810s; elapsed,    2.632s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.190s; elapsed,    0.045s', 'CPP -2', '# Initializing STRUMPACK', '# using 8 OpenMP threads', '# number of tasking levels = 6 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', '# using 4 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,931', '#   - number of levels = 128', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.195347', '#   - matching time = 0.039413', '#   - symmetrization time = 0.026557', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,931', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.045253', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.3058 MB', '#   - factor time = 2.50829', '#   - factor nonzeros = 6,163,223', '#   - factor memory = 49.3058 MB', '#   - total flops = 7.53224e+09, min = 866384, max = 2.95202e+09', '#   - flop rate = 3.00294 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.13207e+09', '# --------------------------------------------', '# total                 = 8.13207e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  3.01093e-11\trel.res =   1.2888e-15\tbw.error =  2.43099e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.048475', '#   - total flops = 1.64951e+07, min = 43055, max = 6.92318e+06', '#   - flop rate = 0.340281 GFlop/s', '#   - bytes moved = 98.3065 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.02798 GByte/s', '#   - solve arithmetic intensity = 0.167793 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    7.600s; elapsed,    2.883s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,   20.670s; elapsed,    2.882s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,   16.630s; elapsed,    2.883s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   22.500s; elapsed,    2.890s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    7.600s; elapsed,    2.883s, #calls:   1', 'TOTAL                                  : CPU,    7.600s; elapsed,    2.883s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,   16.630s; elapsed,    2.883s, #calls:   1', 'TOTAL                                  : CPU,   16.630s; elapsed,    2.883s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,   20.670s; elapsed,    2.882s, #calls:   1', 'TOTAL                                  : CPU,   20.670s; elapsed,    2.882s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   18.260s; elapsed,    2.288s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.190s; elapsed,    0.045s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.810s; elapsed,    2.632s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   22.500s; elapsed,    2.890s, #calls:   1', 'TOTAL                                  : CPU,   44.760s; elapsed,    7.854s']
mpirun -n 8
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 2663.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    6.100s; elapsed,    6.102s', 'individual call time for EIGEN_LDLT: CPU,    2.580s; elapsed,    2.580s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.055s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,945', '#   - number of levels = 126', '#   - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.177324', '#   - matching time = 0.0320971', '#   - symmetrization time = 0.0155351', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,945', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.033618', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.269 MB', '#   - factor time = 2.34061', '#   - factor nonzeros = 6,158,621', '#   - factor memory = 49.269 MB', '#   - total flops = 7.49292e+09, min = 698726, max = 2.95246e+09', '#   - flop rate = 3.20126 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.05389e+09', '# --------------------------------------------', '# total                 = 8.05389e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.57236e-11\trel.res =  1.10107e-15\tbw.error =  2.56229e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0352139', '#   - total flops = 2.07556e+07, min = 27135, max = 6.90876e+06', '#   - flop rate = 0.589414 GFlop/s', '#   - bytes moved = 93.7947 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.66357 GByte/s', '#   - solve arithmetic intensity = 0.221288 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    2.600s; elapsed,    2.647s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    2.610s; elapsed,    2.644s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    2.620s; elapsed,    2.640s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    2.610s; elapsed,    2.646s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    2.630s; elapsed,    2.647s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    2.630s; elapsed,    2.646s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    2.640s; elapsed,    2.647s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    2.650s; elapsed,    2.652s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    2.610s; elapsed,    2.644s, #calls:   1', 'TOTAL                                  : CPU,    2.610s; elapsed,    2.644s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    2.620s; elapsed,    2.640s, #calls:   1', 'TOTAL                                  : CPU,    2.620s; elapsed,    2.640s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    2.630s; elapsed,    2.647s, #calls:   1', 'TOTAL                                  : CPU,    2.630s; elapsed,    2.647s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    2.600s; elapsed,    2.647s, #calls:   1', 'TOTAL                                  : CPU,    2.600s; elapsed,    2.647s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    2.610s; elapsed,    2.646s, #calls:   1', 'TOTAL                                  : CPU,    2.610s; elapsed,    2.646s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    2.630s; elapsed,    2.646s, #calls:   1', 'TOTAL                                  : CPU,    2.630s; elapsed,    2.646s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    2.640s; elapsed,    2.647s, #calls:   1', 'TOTAL                                  : CPU,    2.640s; elapsed,    2.647s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    6.100s; elapsed,    6.102s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.050s; elapsed,    0.055s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.580s; elapsed,    2.580s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    2.650s; elapsed,    2.652s, #calls:   1', 'TOTAL                                  : CPU,   11.380s; elapsed,   11.389s']
mpirun -n 8
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 2663.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    7.080s; elapsed,    3.546s', 'individual call time for EIGEN_LDLT: CPU,    2.730s; elapsed,    2.552s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.090s; elapsed,    0.055s', 'CPP -2', '# Initializing STRUMPACK', '# using 2 OpenMP threads', '# number of tasking levels = 4 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,945', '#   - number of levels = 126', '#   - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.196749', '#   - matching time = 0.0356281', '#   - symmetrization time = 0.01385', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,945', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0362802', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.269 MB', '#   - factor time = 2.1096', '#   - factor nonzeros = 6,158,621', '#   - factor memory = 49.269 MB', '#   - total flops = 7.49293e+09, min = 698726, max = 2.95246e+09', '#   - flop rate = 3.55183 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.05389e+09', '# --------------------------------------------', '# total                 = 8.05389e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.55859e-11\trel.res =  1.09518e-15\tbw.error =  2.50926e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.034878', '#   - total flops = 2.07556e+07, min = 27135, max = 6.90876e+06', '#   - flop rate = 0.595091 GFlop/s', '#   - bytes moved = 93.8014 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.68941 GByte/s', '#   - solve arithmetic intensity = 0.221272 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    3.960s; elapsed,    2.446s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    3.300s; elapsed,    2.446s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    3.230s; elapsed,    2.443s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    4.230s; elapsed,    2.441s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    3.070s; elapsed,    2.446s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    3.070s; elapsed,    2.445s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    4.770s; elapsed,    2.445s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    4.840s; elapsed,    2.450s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    3.960s; elapsed,    2.446s, #calls:   1', 'TOTAL                                  : CPU,    3.960s; elapsed,    2.446s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    3.300s; elapsed,    2.446s, #calls:   1', 'TOTAL                                  : CPU,    3.300s; elapsed,    2.446s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    4.230s; elapsed,    2.441s, #calls:   1', 'TOTAL                                  : CPU,    4.230s; elapsed,    2.441s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    3.070s; elapsed,    2.446s, #calls:   1', 'TOTAL                                  : CPU,    3.070s; elapsed,    2.446s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    3.070s; elapsed,    2.445s, #calls:   1', 'TOTAL                                  : CPU,    3.070s; elapsed,    2.445s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    3.230s; elapsed,    2.443s, #calls:   1', 'TOTAL                                  : CPU,    3.230s; elapsed,    2.443s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    4.770s; elapsed,    2.445s, #calls:   1', 'TOTAL                                  : CPU,    4.770s; elapsed,    2.445s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    7.080s; elapsed,    3.546s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.090s; elapsed,    0.055s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.730s; elapsed,    2.552s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    4.840s; elapsed,    2.450s, #calls:   1', 'TOTAL                                  : CPU,   14.740s; elapsed,    8.604s']
mpirun -n 8
OMP_NUM_THREADS:=4
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 2663.625', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,   11.350s; elapsed,    2.840s', 'individual call time for EIGEN_LDLT: CPU,    3.660s; elapsed,    3.133s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.110s; elapsed,    0.059s', 'CPP -2', '# Initializing STRUMPACK', '# using 4 OpenMP threads', '# number of tasking levels = 5 = log_2(#threads) + 3', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 8 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,945', '#   - number of levels = 126', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '#   - nd time = 0.212622', '#   - matching time = 0.0406101', '#   - symmetrization time = 0.018033', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,945', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0448229', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.269 MB', '#   - factor time = 2.62631', '#   - factor nonzeros = 6,158,621', '#   - factor memory = 49.269 MB', '#   - total flops = 7.49293e+09, min = 699330, max = 2.95246e+09', '#   - flop rate = 2.85303 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 8.05389e+09', '# --------------------------------------------', '# total                 = 8.05389e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.55917e-11\trel.res =  1.09543e-15\tbw.error =  2.50926e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.039459', '#   - total flops = 2.07556e+07, min = 27135, max = 6.90876e+06', '#   - flop rate = 0.526004 GFlop/s', '#   - bytes moved = 93.8099 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.3774 GByte/s', '#   - solve arithmetic intensity = 0.221252 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    7.380s; elapsed,    3.003s', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    5.020s; elapsed,    3.003s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    5.930s; elapsed,    3.004s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    5.770s; elapsed,    3.000s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    8.870s; elapsed,    2.998s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,   10.500s; elapsed,    3.003s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    4.940s; elapsed,    3.003s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,   11.820s; elapsed,    3.008s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    7.380s; elapsed,    3.003s, #calls:   1', 'TOTAL                                  : CPU,    7.380s; elapsed,    3.003s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    5.020s; elapsed,    3.003s, #calls:   1', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    5.770s; elapsed,    3.000s, #calls:   1', 'TOTAL                                  : CPU,    5.020s; elapsed,    3.003s', 'TOTAL                                  : CPU,    5.770s; elapsed,    3.000s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    5.930s; elapsed,    3.004s, #calls:   1', 'TOTAL                                  : CPU,    5.930s; elapsed,    3.004s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,   10.500s; elapsed,    3.003s, #calls:   1', 'TOTAL                                  : CPU,   10.500s; elapsed,    3.003s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    8.870s; elapsed,    2.998s, #calls:   1', 'TOTAL                                  : CPU,    8.870s; elapsed,    2.998s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    4.940s; elapsed,    3.003s, #calls:   1', 'TOTAL                                  : CPU,    4.940s; elapsed,    3.003s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,   11.350s; elapsed,    2.840s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.110s; elapsed,    0.059s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    3.660s; elapsed,    3.133s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,   11.820s; elapsed,    3.008s, #calls:   1', 'TOTAL                                  : CPU,   26.940s; elapsed,    9.040s']
mpirun -n 16
OMP_NUM_THREADS:=1
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '21309 1331.8125', '# ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', 'individual call time for STRUMPACK_OMP: CPU,    6.570s; elapsed,    6.577s', 'individual call time for EIGEN_LDLT: CPU,    2.570s; elapsed,    2.576s', 'individual call time for EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.066s', 'CPP -2', '# Initializing STRUMPACK', '# using 1 OpenMP thread', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', 'CPP -2', '# using 16 MPI processes', '# matching job: maximum matching with row and column scaling', '# initial matrix:', '#   - number of unknowns = 21,309', '#   - number of nonzeros = 253,323', '# nested dissection reordering:', '#   - Scotch reordering', '#   - strategy parameter = 8', '#   - number of separators = 7,953', '#   - number of levels = 125', '#   - nd time = # ***** WARNING ****************************************************', '# Detected a large number of levels in the frontal/elimination tree.', '# STRUMPACK currently does not handle this safely, which', '# could lead to segmentation faults due to stack overflows.', '# As a remedy, you can try to increase the stack size,', '# or try a different ordering (metis, scotch, ..).', '# When using metis, it often helps to use --sp_enable_METIS_NodeNDP,', '# (enabled by default) iso --sp_enable_METIS_NodeND.', '# ******************************************************************', '0.178816', '#   - matching time = 0.0337219', '#   - symmetrization time = 0.0145042', '# symbolic factorization:', '#   - nr of dense Frontal matrices = 7,953', '#   - nr of HSS Frontal matrices = 0', '#   - symb-factor time = 0.0358319', '# multifrontal factorization:', '#   - estimated memory usage (exact solver) = 49.1334 MB', '#   - factor time = 2.17968', '#   - factor nonzeros = 6,141,669', '#   - factor memory = 49.1334 MB', '#   - total flops = 7.46606e+09, min = 582404, max = 2.95054e+09', '#   - flop rate = 3.42531 GFlop/s', '#   - factor memory/nonzeros = 100 % of multifrontal', '#   - maximum HSS rank = 0', '#   - HSS compression = false', '#   - relative compression tolerance = 0.01', '#   - absolute compression tolerance = 1e-08', '#   - normal(0,1) distribution with minstd_rand engine', '', '# ----- FLOP BREAKDOWN ---------------------', '# compression           = 0', '#    random             = 0', '#    ID                 = 0', '#    QR                 = 0', '#    ortho              = 0', '#    reduce_samples     = 0', '#    update_samples     = 0', '#    extraction         = 0', '#    sampling           = 0', '#       CB_sample       = 0', '#       sparse_sampling = 0', '# ULV_factor            = 0', '# Schur                 = 0', '# full_rank             = 7.98609e+09', '# --------------------------------------------', '# total                 = 7.98609e+09', '# --------------------------------------------', '', 'REFINEMENT it. 0\tres =      23362.3\trel.res =            1\tbw.error =            1', 'REFINEMENT it. 1\tres =  2.31462e-11\trel.res =  9.90751e-16\tbw.error =  2.15469e-15', '# DIRECT/GMRES solve:', '#   - abs_tol = 1e-10, rel_tol = 1e-06, restart = 30, maxit = 5000', '#   - number of Krylov iterations = 1', '#   - solve time = 0.0373371', '#   - total flops = 3.13629e+07, min = 19616, max = 6.85515e+06', '#   - flop rate = 0.839994 GFlop/s', '#   - bytes moved = 90.1412 MB, min = 0 MB, max = 0 MB', '#   - byte rate = 2.41426 GByte/s', '#   - solve arithmetic intensity = 0.347931 flop/byte', 'individual call time for STRUMPACK_MPI_DIST_RANK=1: CPU,    2.480s; elapsed,    2.491s', 'individual call time for STRUMPACK_MPI_DIST_RANK=6: CPU,    2.460s; elapsed,    2.494s', 'individual call time for STRUMPACK_MPI_DIST_RANK=7: CPU,    2.430s; elapsed,    2.493s', 'individual call time for STRUMPACK_MPI_DIST_RANK=15: CPU,    2.460s; elapsed,    2.491s', 'individual call time for STRUMPACK_MPI_DIST_RANK=4: CPU,    2.460s; elapsed,    2.493s', 'individual call time for STRUMPACK_MPI_DIST_RANK=5: CPU,    2.450s; elapsed,    2.491s', 'individual call time for STRUMPACK_MPI_DIST_RANK=10: CPU,    2.480s; elapsed,    2.490s', 'individual call time for STRUMPACK_MPI_DIST_RANK=11: CPU,    2.450s; elapsed,    2.489s', 'individual call time for STRUMPACK_MPI_DIST_RANK=12: CPU,    2.470s; elapsed,    2.487s', 'individual call time for STRUMPACK_MPI_DIST_RANK=13: CPU,    2.440s; elapsed,    2.492s', 'individual call time for STRUMPACK_MPI_DIST_RANK=14: CPU,    2.460s; elapsed,    2.493s', 'individual call time for STRUMPACK_MPI_DIST_RANK=2: CPU,    2.440s; elapsed,    2.492s', 'individual call time for STRUMPACK_MPI_DIST_RANK=8: CPU,    2.490s; elapsed,    2.495s', 'individual call time for STRUMPACK_MPI_DIST_RANK=9: CPU,    2.490s; elapsed,    2.492s', 'individual call time for STRUMPACK_MPI_DIST_RANK=3: CPU,    2.440s; elapsed,    2.491s', 'individual call time for STRUMPACK_MPI_DIST_RANK=0: CPU,    2.490s; elapsed,    2.497s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=6: CPU,    2.460s; elapsed,    2.494s, #calls:   1', 'TOTAL                                  : CPU,    2.460s; elapsed,    2.494s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=8: CPU,    2.490s; elapsed,    2.495s, #calls:   1', 'TOTAL                                  : CPU,    2.490s; elapsed,    2.495s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=4: CPU,    2.460s; elapsed,    2.493s, #calls:   1', 'TOTAL                                  : CPU,    2.460s; elapsed,    2.493s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=7: CPU,    2.430s; elapsed,    2.493s, #calls:   1', 'TOTAL                                  : CPU,    2.430s; elapsed,    2.493s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=12: CPU,    2.470s; elapsed,    2.487s, #calls:   1', 'TOTAL                                  : CPU,    2.470s; elapsed,    2.487s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=15: CPU,    2.460s; elapsed,    2.491s, #calls:   1', 'TOTAL                                  : CPU,    2.460s; elapsed,    2.491s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=1: CPU,    2.480s; elapsed,    2.491s, #calls:   1', 'TOTAL                                  : CPU,    2.480s; elapsed,    2.491s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=11: CPU,    2.450s; elapsed,    2.489s, #calls:   1', 'TOTAL                                  : CPU,    2.450s; elapsed,    2.489s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=5: CPU,    2.450s; elapsed,    2.491s, #calls:   1', 'TOTAL                                  : CPU,    2.450s; elapsed,    2.491s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=2: CPU,    2.440s; elapsed,    2.492s, #calls:   1', 'TOTAL                                  : CPU,    2.440s; elapsed,    2.492s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=10: CPU,    2.480s; elapsed,    2.490s, #calls:   1', 'TOTAL                                  : CPU,    2.480s; elapsed,    2.490s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=3: CPU,    2.440s; elapsed,    2.491s, #calls:   1', 'TOTAL                                  : CPU,    2.440s; elapsed,    2.491s', 'Exiting profiler', 'time for      STRUMPACK_MPI_DIST_RANK=9: CPU,    2.490s; elapsed,    2.492s, #calls:   1', 'TOTAL                                  : CPU,    2.490s; elapsed,    2.492s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=13: CPU,    2.440s; elapsed,    2.492s, #calls:   1', 'TOTAL                                  : CPU,    2.440s; elapsed,    2.492s', 'Exiting profiler', 'time for     STRUMPACK_MPI_DIST_RANK=14: CPU,    2.460s; elapsed,    2.493s, #calls:   1', 'TOTAL                                  : CPU,    2.460s; elapsed,    2.493s', 'Strumpack solutions agree with Eigen', 'Exiting profiler', 'time for                  STRUMPACK_OMP: CPU,    6.570s; elapsed,    6.577s, #calls:   1', 'time for                 EIGEN_BICGSTAB: CPU,    0.070s; elapsed,    0.066s, #calls:   1', 'time for                     EIGEN_LDLT: CPU,    2.570s; elapsed,    2.576s, #calls:   1', 'time for      STRUMPACK_MPI_DIST_RANK=0: CPU,    2.490s; elapsed,    2.497s, #calls:   1', 'TOTAL                                  : CPU,   11.700s; elapsed,   11.716s']
mpirun -n 16
OMP_NUM_THREADS:=2
['/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::ReorderingStrategy already registered; second conversion method ignored.', '  try: mod = __import__(name)', '/net/dials/raid1/mlxd/InDev/STRUMPACK_MPI_DIST/modules/cctbx_project/boost/python.py:25: RuntimeWarning: to-Python converter for strumpack::KrylovSolver already registered; second conver