Prof. Thomas Sterling
Dr. Hartmut Kaiser
Department of Computer Science
Louisiana State University
April 7th , 2011
HIGH PERFORMANCE COMPUTING: MODELS, METHODS, &
MEANS
INTRODUCTION TO LIBRARIES 1
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
1
Outline
•
•
•
•
•
•
Why libraries
What is a library
How to use a library
Standard library support
Linear Algebra Library : BLAS
Summary - Materials for Test
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
2
Puzzle of the Day
What's the expected output for the following program and why?
Hint: What's the effect of the continue statement in a do-while loop?
enum { false = 0, true = 1 };
int main()
{
int i = 1;
do {
printf("%d\n",i);
if(++i < 15)
continue;
} while (false);
return 0;
}
References: [3]
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
3
Outline
•
•
•
•
•
•
•
Why libraries
What is a library
How to use a library
Standard library support
Application domains
Linear Algebra Library: BLAS
Summary - Materials for Test
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
4
Why libraries?
• How can we dare to assume to be able to write correct
code?
• Reuse, reuse, reuse!
– Allows to concentrate on the science
– Leverage knowledge and skills of others
– Offload part of your work to library maintainers
• But: used libraries should be
–
–
–
–
High quality
Flexible and generic
Combinable
Preferrably have access to source code
• Use the right tool for the right job
– Having a new and shiny hammer doesn‘t mean everything
is a nail
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
5
Outline
•
•
•
•
•
•
•
Why libraries
What is a library
How to use a library
Standard library support
Application domains
Linear Algebra Library: BLAS
Summary - Materials for Test
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
6
What is a library?
• Short history of software libraries
• Different perspectives
• Classification of libraries
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
7
Short history of software libraries
Bouchons Loom (1725)
Basile Bouchon, Jean Falcon, Jacques Vaucanson,
Joseph Marie Jacquard (1801):
An automated loom that transformed the 18th
century textile industry and became the inspiration
for future calculating and tabulating machines.
The binary principle embodied in the punched-card
operation of the loom was inspiration for the data
processing machines to come.
Picture of Jacquard loom (1830)
References: [1]
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2010
8
Short history of software libraries
Charles Babbage’s Analytical Machine (1830)
Every set of cards made for any formula will at any future time recalculate that formula with whatever
constants may be required. Thus the Analytical Engine will possess a library of its own. Every set of cards
once made will at any future time reproduce the calculations for which it was first arranged.
— Passages from the Life of a Philosopher, Charles Babbage (1864)
References: [1]
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
9
Short history of software libraries
Harvard Mark I: Grace Murray Hopper and Howard Aiken (1944)
Some sequences that were used again and again were permanently wired into the Mark I’s
circuits... Since the Mark I was not a stored-program computer, Hopper had no choice for other
sequences than to code the same pattern in successive pieces of tape. It did not take long for her to
realize that if a way could be found to reuse the pieces of tape already coded for another problem, a lot
of effort would be saved. The Mark I did not allow that to be easily done, but the idea had taken root and
later modifications did permit multiple tape loops to be mounted.
References: [1]
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
10
Short history of software libraries
Alan Turing and the ACE (1946)
Subroutine, call stack, jump and return: When we wish to start on a
subsidiary operation [subroutine] we need only make a note of where
we left off the major operation [return address] and then apply the first
instruction of the subsidiary. When the subsidiary is over we look up the
note and continue with the major operation. Each subsidiary operation
can end with instructions for this recovery of the note. How is the
burying and disinterring [push and pop] of the note to be done? There
are of course many ways. One is to keep a list of these notes in one or
more standard size delay lines (1024), with the most recent last [a
stack]... the burying being done through a standard instruction table
BURY, and the disinterring by the table UNBURY.
— Proposals for the development in the Mathematics Division of an
Automatic Computing Engine (ACE), Alan M. Turing (1946)
[1]
CSC 7600 Lecture 19 : HPC LibrariesReferences:
1
Spring 2011
11
Calling subroutines
• Hardware support for stack
operations (LIFO: last in first
out)
• Special stack pointer or
general register
Grows downward
– Call:
• Put parameters on top of stack
• Put return address on top of
stack
• Jump to subroutine
– Return
• Retrieve address from top of
stack
• Jump to this address
• Modern compilers use stack
for local data as well
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
12
Calling conventions
FORTRAN
C
• Parameters put on stack from
left to right by reference
• Called code is responsible for
cleaning parameters from
stack (unwinding)
• Local data on stack is handled
by called code
• Caller and callee must agree
on number of arguments
• Parameters put on stack from
right to left by value
• Calling code is responsible for
cleaning parameters from
stack (unwinding)
• Local data on stack is handled
by called code
• Subroutines may have variable
parameter count (printf)
• Names are not changed
(sometimes all capital)
• Modern compilers have
various options
• Prepended ‘_‘ for names
(sometimes appended)
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
13
Short history of software libraries
EDSAC - Electronic Delay Storage Automatic Calculator (1951)
“The library of tapes on which subroutines are punched is contained in the steel cabinet shown on
the left. The operator is punching a program tape on a keyboard perforator. She can copy mechanically
tapes taken from the library on to the tape she is preparing by placing them in the tape reader shown in
the center of the photograph.”
References: [1]
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
14
Short history of software libraries
• Key ideas of the EDSAC library (David
Wheeler, Maurice Wilkes)
– Library of subroutines
– Reuse of reliable components to shorten development time and
reduce defects
– Linking, relocatable objects
– Multiple versions of a subroutine, each with a clearly indicated
tradeoff of time, space, accuracy
– Unit testing
– Open subroutines (inline/intrinsic functions), nonstandard semantics
(‘interpretive subroutines’)
– Pure vs. impure functions
– Debugging interpreter
– Passing functions as parameters (e.g., to integration subroutines)
[1]
CSC 7600 Lecture 19 : HPC LibrariesReferences:
1
Spring 2011
15
What is a library?
• No clear answer, could be many things:
– A library is a reuse repository
• “A library is a bunch of code I don’t have to write.”
– A library is a knowledge base
• A library is a knowledge base about a problem domain.
– A library is a language extension
• Different languages have differently sharp borders between
language and libraries
• “In effect, designing a class library is like designing part of a
programming language, and should be approached with
commensurate respect.”
– A library is a notation
References: [1]
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
16
What is a library?
– A library is an expert-in-a-box
• Allows to concentrate on the science without having to worry about
implementation details
– A library is an abstraction
• APIs hide the details; we can use libraries knowing what they do but
don’t need to know how they do it.
– A library is a de-facto standard
• Widely used, open source, well tested
• Full standards: C99 Standard library, C++ Standard template library
– A library is a defect management strategy
• “ The only error free code I ever write is the code I do not have to
write“
References: [1]
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
17
What is a library?
– A library is a tool for software compression
• Especially in the context of shared libraries
– A library is a stable platform
• Implementation can change without breaking code relying on it
– A library is a vehicle for technology adoption
• A certain technology (way of doing things) may be encapsulated
behind a API, simplifying it‘s adoption
• New technologies may be encapsulated behind old APIs
– A library is a communication medium
• Allows to communicate on higher levels using conepts
References: [1]
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
18
Libraries (by locality)
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
19
Libraries (by domain)
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
20
Application domains
• General parallelization, load balancing
– MPI, Charm++
• Mesh manipulation and management
– METIS, ParMETIS, Jostle
• Graph manipulation
– (Parallel) Boost.Graph library, MTGL
• Vector/Signal/Image processing
– VSIPL, PSSL
• Linear algebra
– BLAS, ATLAS, LAPACK, LINPACK, Slatec, pim
• Ordinary and partial Differential Equations
– PETSc
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
21
Outline
•
•
•
•
•
•
•
Why libraries
What is a library
How to use a library
Standard library support
Application domains
Linear Algebra Library: BLAS
Summary - Materials for Test
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
22
How to use a library?
•
•
•
•
Compile single source file
Compile multiple source files
Create a library
Compile multiple source files written using different
languages
• Combining different languages
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
23
Compile single source file
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
24
Compile single source file
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
25
Compile multiple source files
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
26
Create a library
• Static library (.a)
– Created using ar (archiver)
– Just collection of object
modules and a table of
entry points
– Used by linker to add
referred code to created
executable
• Dynamic library (.so)
– Created by ld (linker)
– Executable binary code
with resolved externals
– Used by linker to add
reference to created
executable
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
27
Create a library
• Static library (.a)
• Dynamic library (.so)
– No additional runtime
dependencies
– Beneficial in simple
scenario‘s
– Code loaded only once
– Beneficial in complex
binary applications
– If used in more than one
module code will be
duplicated
– Additional runtime
dependency
– Difficult version control
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
28
Compile multiple source files
• Interface of subroutines must be known
• Different languages have different means of
interface specification for modules,
subroutines and functions
main.c
say_hello.c
int main()
{
return say_hello(“Hello“);
}
int say_hello(char const* msg)
{
return puts(msg);
}
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
29
Compile multiple source files
say_hello.h
int say_hello(char const* msg);
main.c
say_hello.c
#include “say_hello.h“
int main()
{
return say_hello(“Hello“);
}
#include <stdio.h>
#include “say_hello.h“
int say_hello(char const* msg)
{
return puts(msg);
}
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
30
Multi language programming
• Need to account for
– Different calling conventions
• C, FORTRAN calling conventions
• Parameter passing (by value/by reference)
• Parameter types (strings)
– Naming conventions
• FORTRAN: all uppercase, C: case is significant
– Data types
• Memory layout (row major, column major, strings)
• Most of this is done by providing a correct interface
description to the FORTRAN and/or C compilers
• All of this is highly compiler specific, but GNU compiler
suite (gcc, f77 etc.) are well suited
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
31
Why C++
• Multiparadigm language
– Object orientation, functional programming, template meta-programming
•
•
•
•
Better maintainability of programs
More frequent code re-use
More efficient software development in groups
Higher adaptability of software to new demands
– Huge amount of libraries, from simple data structures and algorithms to
modules in highly specialized domains
– But you don’t pay for what you don’t use
• C++ is available and supported by vendors on almost all
Supercomputers like Cray, NEC SX, Hitachi SR8000 …
• With a few minor exceptions C++ is a better C: this allows a smooth
migration from C to C++
– C is a full subset of C++
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
32
C/C++
• Since C and C++ are ‚siblings‘ interfacing is easy:
extern ”C” {…}
– Adjusts naming and calling conventions
• C data types are generally compatible with C++
• C++ data types (classes) are not generally compatible
with C except POD (plain old data) types
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
33
Calling C from C++
say_hello.h
int say_hello(char const* msg);
main.cpp
say_hello.c
extern “C“ {
#include “say_hello.h“
}
int main()
{
return say_hello(“Hello“);
}
#include <stdio.h>
#include “say_hello.h“
int say_hello(char const* msg)
{
return puts(msg);
}
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
34
Calling C++ from C
say_hello.hpp
#ifdef __cplusplus
extern “C“ int say_hello(char const* msg);
#else
int say_hello(char const* msg);
#endif
main.c
say_hello.cpp
#include “say_hello.hpp“
int main()
{
return say_hello(“Hello“);
}
#include <stdio.h>
#include “say_hello.hpp“
int say_hello(char const* msg)
{
return puts(msg);
}
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
35
Calling C++ from C
say_hello.hpp
#ifdef __cplusplus
extern “C“
#endif
int say_hello(char const* msg);
main.c
say_hello.cpp
#include “say_hello.hpp“
int main()
{
return say_hello(“Hello“);
}
#include <stdio.h>
#include “say_hello.hpp“
int say_hello(char const* msg)
{
return puts(msg);
}
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
36
FORTRAN/C
• Data types:
FORTRAN
C/C++
integer*2
short int
integer
long int or int
integer iabc(2,3)
int iabc[3][2];
logical
long int or int
logical*1
bool (C++, one byte)
Real
float
real*8
double
complex
C: struct { float r, i; }
C++: std::complex<float>
double complex
C: struct { double dr, di; },
C++: std::complex<double>
character*6 abc
char abc[6];
parameter
#define PARAMETER value
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
37
Multi language programming
•
•
Interface of subroutines must be declared in a language specific way
• Tooling support available
• SWIG: http://www.swig.org
• FLIB2C: http://www.mycplus.com/utilitiesdetail.asp?iPro=7
More information:
http://arnholm.org/software/cppf77/cppf77.htm
main.f
say_hello.c
INTERFACE TO SUBROUTINE SAY_HELLO
[C.ALIAS: '_say_hello'] (msg)
CHARACTER(*) msg
END
PROGRAM MAIN
CALL SAY_HELLO(“Hello“)
END
int say_hello(
char const* msg, int len)
{
return printf(“%*d”, len, msg);
}
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
38
Outline
•
•
•
•
•
•
•
Why libraries
What is a library
How to use a library
Standard library support
Application domains
Linear Algebra Library: BLAS
Summary - Materials for Test
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
39
Standard library support
• Standard (runtime) libraries
• C++ Standard template library
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
40
Standard (runtime) libraries
• Libraries needed by almost any application
– Provide often used functions (data types, algorithms, I/O, support
routines)
– No need to explicitly specify library
• Compiler and linker usually ‚know’ what runtime libraries to use
• Different languages have different level and amount of
standard library support (system level and support)
– F77: very small standards library, F90: more available
• Filesystem, math, auxiliary
– C99: large standards library aimed at portability over wide amount
of platforms
• Operating system, filesystem, math, string handling, basic data types
(complex, integer types)
– C++: everything in C99 plus Standards Template Library
• Adds data structures, algorithms
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
41
C++ Standard (Template) Library
• C++0x: library ~700 pages description in Standard
–
–
–
–
–
–
Language support (exception handling, memory allocation, etc.)
Diagnostics (assertions, system errors, etc.)
General utilities (tuples, metaprogramming, type traits)
Strings (string classes, numeric conversions)
Localization (locales)
STL (Standard template Library)
• Containers: sequence, associative , unordered associative containers
– (vector, list, set, map, deque etc.)
• Iterators
– (forward, bidirectional, random_access etc.)
• Algorithms: non-modifying, modifying, sorting algorithms
– (foreach, transform, copy, sort, uniq etc.)
–
–
–
–
–
Numerics: complex numbers, random number generators, etc.
Input/output: streams, file I/O
Regex
Atomic operations
Thread support: threads, mutual exclusion, futures
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
42
C++ Standard Template Library
• Algorithms and data structures are generic and
orthogonal
– Each algorithm usable with any data structure
– Algorithms usable with your data structures
– Your algorithms can use standard data structures
• Iterators connect the two
– General pointer concept, i.e. used by algorithms to
refer to the data items
– Allow to abstract the algorithms from the data
structures
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
43
Simple example
• Copy a vector of integer’s into a list
std::vector<int> v; // v = 1, 2, 3, 4;
std::list<int> l;
std::copy(v.begin(), v.end(), std::back_inserter(l));
• Or v.v.:
std::copy(l.begin(), l.end(), std::back_inserter(v));
• How is it implemented
template <typename InIter, typename OutIter>
void copy(InIter f, InIter l, OutIter o)
{
for (/**/; f != l; ++f, ++o)
*o = *f;
}
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
44
Genericity leads to Concepts
• No strict interface anymore (as known from Fortran or C)
• Rather components required to expose concepts, i.e.
satisfy set of requirements
• In the copy example:
– First two parameters must be at least input iterators
• implement operator++(), operator*()
– Last parameter must be at least an output iterator
• Implement operator++(), operator*()
• Advantage:
– Generic, simple, uniform, optimized
• Drawback:
– Long compiler error messages if something is wrong
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
45
Iterator categories (concepts)
Read one item at a
time, in forward
direction only
Read and write one
item at a time, in
forward direction only
Input
Output
forward
bi-directional
Write one item at a
time, either in forward
or backward direction
can jump any distance
Write one item at a
time, in forward
direction only
Write one item at a
time, either in forward
or backward direction
Random access
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
46
Iterator behavior
•
Common operations
++i: Advance one element and return reference to i
i++: Advance one element and return the previous value of i
•
Input iterator operations
*i: Return a read-only reference to the element at i‘s current position
i == j: Return true if i and j positioned at the same element
(i != j: at different elements)
•
Output iterator operations
*i: Return a write-only reference to the element at i‘s current position
i = j: set i‘s position to the same as j‘s
•
Bidirectional iterator operations
--i: Retreat one element and returns i‘s new value
i--: Retreat one element and returns i‘s previous value
•
Random access iterator operations
i + n: Return an iterator positioned n elements ahead i‘s current position
i – n: Return an iterator positioned n elements behind i‘s current position
i[n]: return a reference to the n‘th element from i‘s current position
•
A plain C pointer is a random access iterator
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
47
Containers and iterators
Container
Iterator
Container
Iterator
vector
random access
map
bidirectional
deque
random access
multimap
bidirectional
list
bidirectional
stack
none
set
bidirectional
queue
none
multiset
bidirectional
priority_queue
none
• Every container
– Has typedefs for this (no need to remember above):
• iterator, const_iterator, reverse_iterator, const_reverse_iterator
– Exposes functions returning iterators:
• begin(), end() (non-const and const variants)
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
48
What‘s that all about?
• Orthogonality:
std::vector<int> v; // v = 3, 1, 4, 2;
std::sort(v.begin(), v.end());
std::list<int> l;
// l = 3, 1, 4, 2;
std::sort(v.begin(), v.end());
• Any algorithm is usable with any container
– Still optimal code, because STL contains optimal implementation
for each iterator type
– Optimal code with your data structures as well
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
49
Outline
•
•
•
•
•
•
•
Why libraries
What is a library
How to use a library
Standard library support
Application Domains
Linear Algebra Library: BLAS
Summary - Materials for Test
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
50
Application domains
• Linear algebra
– BLAS, ATLAS, LAPACK, ScaLAPACK, Slatec, pim
• Ordinary and Partial Differential Equations
– PETSc
• Mesh Manipulation and Load Balancing
– METIS, ParMETIS, CHACO, JOSTLE, PARTY
• Graph Manipulation
– Boost.Graph library
• Vector/Signal/Image Processing
– VSIPL, PESSL (IBMs Parallel Engineering Scientific Subroutine Library
• General Parallelization
– MPI, pthreads
• Other Domain Specific Libraries
– NAMD, NWChem, Fluent, Gaussian, LS-DYNA
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
51
Application Domain Overview
• Linear Algebra Libraries
– Provide optimized methods for constructing sets of linear equations,
performing operations on them (matrix-matrix products, matrix-vector
products) and solving them (factoring, forward & backward
substitution.
– Commonly used libraries include BLAS, ATLAS, LAPACK,
ScaLAPACK, PaLAPACK
• PDE Solvers:
– General-purpose, parallel numerical PDE libraries
– Usual toolsets include manipulation of sparse data structures,
iterative linear system solvers, preconditioners, nonlinear solvers and
time-stepping methods.
– Commonly used libraries for solving PDEs include SAMRAI, PETSc,
PARASOL, Overture, among others.
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
52
Application Domain Overview
• Mesh manipulation and Load Balancing
– These libraries help in partitioning meshes in roughly equal sizes
across processors, thereby balancing the workload while
minimizing size of separators and communication costs.
– Commonly used libraries for this purpose include METIS, ParMetis,
Chaco, JOSTLE among others.
• Other packages:
– FFTW: features highly optimized Fourier transform package
including both real and complex multidimensional transforms in
sequential, multithreaded, and parallel versions.
– NAMD: molecular dynamics library available for Unix/Linux,
Windows, OS X
– Fluent: computational fluid dynamics package, used for such
applications as environment control systems, propulsion, reactor
modeling etc.
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
53
Outline
•
•
•
•
•
•
•
Why libraries
What is a library
How to use a library
Standard library support
Application domains
Linear Algebra Library: BLAS
Summary - Materials for Test
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
54
BLAS
• (Updated set of) Basic Linear Algebra Subprograms
• The BLAS functionality is divided into three levels:
– Level 1: contains vector operations of the form:
as well as scalar dot products and vector norms
– Level 2: contains matrix-vector operations of the form
as well as Tx = y solving for x with T being triangular
– Level 3: contains matrix-matrix operations of the form
as well as solving
for triangular matrices T. This level
contains the widely used General Matrix Multiply operation.
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
55
BLAS
• Several implementations for different languages exist
– Reference implementation (F77 and C-wrapper)
http://www.netlib.org/blas/
– ATLAS, highly optimized for particular processor architectures
– A generic C++ template class library providing BLAS
functionality: uBLAS
http://www.boost.org
– Several vendors provide libraries optimized for their architecture
(AMD, HP, IBM, Intel, NEC, NViDIA, Sun)
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
56
BLAS: F77 naming conventions
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
57
BLAS: C naming conventions
• F77 routine name is changed to lowercase and prefixed with
cblas_
• All routines accepting two dimensional arrays have a new
additional first parameter specifying the matrix memory layout
(row major or column major)
• Character parameters are replaced by corresponding enum
values
• Input arguments are declared const
• Non-complex scalar input parameters are passed by value
• Complex scalar input arguments are passed using a void*
• Arrays are passed by address
• Output scalar arguments are passed by address
• Complex functions become subroutines which return the result
via an additional last parameter (void*), appending _sub to
the name
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
58
BLAS Level 1 routines
• Vector operations
(xROT, xSWAP, xCOPY etc.)
• Scalar dot products
(xDOT etc.)
• Vector norms
(IxAMX etc.)
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2010
59
BLAS Level 2 routines
• Matrix-vector operations
(xGEMV, xGBMV, xHEMV, xHBMV etc.)
• Solving Tx = y for x, where T is triangular
(xGER, xHER etc.)
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2010
60
BLAS Level 3 routines
• Matrix-matrix operations
(xGEMM etc.)
• Solving
for triangular matrices
(xTRMM)
• Widely used matrix-matrix multiply (xSYMM, xGEMM)
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2010
61
Outline
•
•
•
•
•
•
•
Why libraries
What is a library
How to use a library
Standard library support
Introduction to High Performance Libraries
Linear Algebra Libraries (BLAS, LAPACK)
Summary - Materials for Test
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
62
Summary – Material for the Test
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Why Libraries: (Slide 5)
Calling Subroutines: (Slide 12)
What is a Library: (Slides 16-18)
Library (by locality) : (Slide 19)
Library (by domain): (Slide 20)
Application domains: (Slide 21)
Creating a library: (Slides 27-28)
Multi language programming: (Slide 31)
Standard runtime libraries: (Slide 41)
Iterator Categories & Behavior: (Slides 46-47)
Containers & Iterators: (Slide 48)
What is all that about: (Slide 49)
High performance libraries (Slides 51-53)
Linear algebra libraries: BLAS (Slides 55-61)
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2010
References
1.
2.
3.
Todd Vedhuizen, What is a Library?, Talk given at the Dagstuhl workshop Software Libraries:
Design and Evaluation, Schloss Dagstuhl, Germany, March 9-11 2005
Rene Heinzl, Modern Application Design using Modern Programming Paradigms and
a Library-Centric Software Approach, OOPSLA 2006, Workshop on Library Centric Software
Design, Portland, Oregon, October 2006.
C puzzles, http://www.gowrikumar.com/c/index.html
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
64
CSC 7600 Lecture 19 : HPC Libraries 1
Spring 2011
Descargar

Slide 1