petsc-3.7.1 2016-05-15
Report Typos and Errors

MatMPIAIJSetPreallocation

Preallocates memory for a sparse parallel matrix in AIJ format (the default parallel PETSc format). For good matrix assembly performance the user should preallocate the matrix storage by setting the parameters d_nz (or d_nnz) and o_nz (or o_nnz). By setting these parameters accurately, performance can be increased by more than a factor of 50.

Synopsis

#include "petscmat.h" 
PetscErrorCode  MatMPIAIJSetPreallocation(Mat B,PetscInt d_nz,const PetscInt d_nnz[],PetscInt o_nz,const PetscInt o_nnz[])
Collective on MPI_Comm Many br

Input Parameters

B - the matrix Many br
d_nz - number of nonzeros per row in DIAGONAL portion of local submatrix Many br(same value is used for all local rows) Many br
d_nnz - array containing the number of nonzeros in the various rows of the Many brDIAGONAL portion of the local submatrix (possibly different for each row) Many bror NULL (PETSC_NULL_INTEGER in Fortran), if d_nz is used to specify the nonzero structure. Many brThe size of this array is equal to the number of local rows, i.e 'm'. Many brFor matrices that will be factored, you must leave room for (and set) Many brthe diagonal entry even if it is zero. Many br
o_nz - number of nonzeros per row in the OFF-DIAGONAL portion of local Many brsubmatrix (same value is used for all local rows). Many br
o_nnz - array containing the number of nonzeros in the various rows of the Many brOFF-DIAGONAL portion of the local submatrix (possibly different for Many breach row) or NULL (PETSC_NULL_INTEGER in Fortran), if o_nz is used to specify the nonzero Many brstructure. The size of this array is equal to the number Many brof local rows, i.e 'm'. Many br

If the *_nnz parameter is given then the *_nz parameter is ignored Many br

The AIJ format (also called the Yale sparse matrix format or Many brcompressed row storage (CSR)), is fully compatible with standard Fortran 77 Many brstorage. The stored row and column indices begin with zero. Many brSee Users-Manual: ch_mat for details. Many br

The parallel matrix is partitioned such that the first m0 rows belong to Many brprocess 0, the next m1 rows belong to process 1, the next m2 rows belong Many brto process 2 etc.. where m0,m1,m2... are the input parameter 'm'. Many br

The DIAGONAL portion of the local submatrix of a processor can be defined Many bras the submatrix which is obtained by extraction the part corresponding to Many brthe rows r1-r2 and columns c1-c2 of the global matrix, where r1 is the Many brfirst row that belongs to the processor, r2 is the last row belonging to Many brthe this processor, and c1-c2 is range of indices of the local part of a Many brvector suitable for applying the matrix to. This is an mxn matrix. In the Many brcommon case of a square matrix, the row and column ranges are the same and Many brthe DIAGONAL part is also square. The remaining portion of the local Many brsubmatrix (mxN) constitute the OFF-DIAGONAL portion. Many br

If o_nnz, d_nnz are specified, then o_nz, and d_nz are ignored. Many br

You can call MatGetInfo() to get information on how effective the preallocation was; Many brfor example the fields mallocs,nz_allocated,nz_used,nz_unneeded; Many brYou can also run with the option -info and look for messages with the string Many brmalloc in them to see if additional memory allocation was needed. Many br

Example usage

Consider the following 8x8 matrix with 34 non-zero values, that is Many brassembled across 3 processors. Lets assume that proc0 owns 3 rows, Many brproc1 owns 3 rows, proc2 owns 2 rows. This division can be shown Many br

as follows

            1  2  0  |  0  3  0  |  0  4
    Proc0   0  5  6  |  7  0  0  |  8  0
            9  0 10  | 11  0  0  | 12  0
    -------------------------------------
           13  0 14  | 15 16 17  |  0  0
    Proc1   0 18  0  | 19 20 21  |  0  0
            0  0  0  | 22 23  0  | 24  0
    -------------------------------------
    Proc2  25 26 27  |  0  0 28  | 29  0
           30  0  0  | 31 32 33  |  0 34
Many br

This can be represented as a collection of submatrices as

      A B C
      D E F
      G H I
Many br

Where the submatrices A,B,C are owned by proc0, D,E,F are Many browned by proc1, G,H,I are owned by proc2. Many br

The 'm' parameters for proc0,proc1,proc2 are 3,3,2 respectively. Many brThe 'n' parameters for proc0,proc1,proc2 are 3,3,2 respectively. Many brThe 'M','N' parameters are 8,8, and have the same values on all procs. Many br

The DIAGONAL submatrices corresponding to proc0,proc1,proc2 are Many brsubmatrices [A], [E], [I] respectively. The OFF-DIAGONAL submatrices Many brcorresponding to proc0,proc1,proc2 are [BC], [DF], [GH] respectively. Many brInternally, each processor stores the DIAGONAL part, and the OFF-DIAGONAL Many brpart as SeqAIJ matrices. for eg: proc1 will store [E] as a SeqAIJ Many brmatrix, ans [DF] as another SeqAIJ matrix. Many br

When d_nz, o_nz parameters are specified, d_nz storage elements are Many brallocated for every row of the local diagonal submatrix, and o_nz Many brstorage locations are allocated for every row of the OFF-DIAGONAL submat. Many brOne way to choose d_nz and o_nz is to use the max nonzerors per local Many brrows for each of the local DIAGONAL, and the OFF-DIAGONAL submatrices. Many br

In this case, the values of d_nz,o_nz are

     proc0 : dnz = 2, o_nz = 2
     proc1 : dnz = 3, o_nz = 2
     proc2 : dnz = 1, o_nz = 4
Many brWe are allocating m*(d_nz+o_nz) storage locations for every proc. This Many brtranslates to 3*(2+2)=12 for proc0, 3*(3+2)=15 for proc1, 2*(1+4)=10 Many brfor proc3. i.e we are using 12+15+10=37 storage locations to store Many br34 values. Many br

When d_nnz, o_nnz parameters are specified, the storage is specified Many brfor every row, coresponding to both DIAGONAL and OFF-DIAGONAL submatrices. Many br

In the above case the values for d_nnz,o_nnz are

     proc0: d_nnz = [2,2,2] and o_nnz = [2,2,2]
     proc1: d_nnz = [3,3,2] and o_nnz = [2,1,1]
     proc2: d_nnz = [1,1]   and o_nnz = [4,4]
Many brHere the space allocated is sum of all the above values i.e 34, and Many brhence pre-allocation is perfect. Many br

Many br

Keywords

matrix, aij, compressed row, sparse, parallel

See Also

MatCreate(), MatCreateSeqAIJ(), MatSetValues(), MatCreateAIJ(), MatMPIAIJSetPreallocationCSR(),
MPIAIJ, MatGetInfo(), PetscSplitOwnership() Many br

Level:intermediate
Location:
src/mat/impls/aij/mpi/mpiaij.c
Index of all Mat routines
Table of Contents for all manual pages
Index of all manual pages

Examples

src/mat/examples/tutorials/ex5.c.html
src/mat/examples/tutorials/ex15.c.html
src/mat/examples/tutorials/ex16.c.html
src/mat/examples/tutorials/ex17.c.html
src/ksp/ksp/examples/tutorials/ex2.c.html
src/ksp/ksp/examples/tutorials/ex3.c.html
src/ksp/ksp/examples/tutorials/ex4.c.html
src/ksp/ksp/examples/tutorials/ex7.c.html
src/ksp/ksp/examples/tutorials/ex18.c.html
src/ksp/ksp/examples/tutorials/ex52.c.html
src/ksp/ksp/examples/tutorials/ex55.c.html