PARALLEL DISTRIBUTED SEISMIC IMAGING

ALGORITHMS ON PARAM 10000

S. PHADKE, R. RASTOGI, S. YERNENI and S. CHAKRABORTY

Centre for Development of Advanced Computing, Pune University Campus, Pune 411007, India

Summary (interfaces in the velocity field). This technique is known as

seismic migration.
Obtaining high-resolution images of the underground
geological structures using seismic reflection data in prestack Forward modelling, where the synthetic data is generated for a
or poststack domain is crucial for exploration of oil and gas given earth model, is a key step in the process of seismic
deposits. In the last decade the development of parallel inversion, where one tries to estimate the physical properties
distributed computing platforms, related system software and of the earth. 80 to 90 percent of the computer time in an
programming environments have made it possible to use inversion algorithm is spent on generating synthetic data.
parallel codes for high resolution imaging. Centre for Parallel and efficient algorithms are therefore necessary for
Development of Advanced Computing (C-DAC) located at this purpose.
Pune, developed the OpenFrame architecture for scalable In this paper we shall describe several migration and
parallel computing applications. Several seismic migration and modelling algorithms that are developed and parallelized for a
modelling algorithms were developed and implemented for distributed memory machine. Performance and efficiency is
imaging purposes. In this presentation we shall discuss several achieved by proper restructuring of the codes. All the imaging
wave equation based 2D and 3D seismic migration and algorithms have been tested for both synthetic and real data
modelling algorithms and their parallel implementation using sets.
MPI message passing environment. Large-scale problems can
be solved by implementation of highly efficient and scalable Parallel Computing and Seismic Data Processing
codes. These codes can be easily ported across cluster of
There is a strong consensus amongst the computer
workstations.
professionals, that the greatest gains in price/performance can
Introduction only be achieved through multiple processor parallel systems.
Parallel computers are characterized by two or more
High-performance computers are now essentials tools in processing elements and memory, tied together by some
scientific and technological research and development. With interconnection network. Abundance of relatively slow
their high-speed processing capability, large-scale storage processors, working together to solve one problem, provides
capacity and efficient I/O, computers are now important tools the necessary performance.
for simulation experiments and for processing large volumes
of data. Future developments in high-speed and large-scale The trend in parallel computing is to move away from
supercomputers will play a significant role in the research and specialized traditional supercomputing platforms, such as Cray
development of advanced technology for the 21st century. / SGI T3E, to cheaper and general purpose systems consisting
Parallel processing is the key technology to make large-scale of loosely coupled components built up from single or
processing capability possible. multiprocessor PCs or workstations. This approach has a
number of advantages, including being able to build a
Seismic imaging is a form of echo-reconstructive technique platform for a given budget, which is suitable for a large class
based on experiments, in which a certain earth volume is of applications and workloads.
illuminated by an explosive or vibratory source, and the
backscattered energy by the inhomogeneties of the medium is The hardware technology and economic forces are right for an
recorded on the surface in digital form. The inhomogeneties explosion of parallel processing into the market at all levels.
act as reflecting surfaces, which cause signal echoing; the Parallel processing, or concurrent computing as it is
echoes are then recorded at the surface and processed through sometimes termed, is not conceptually new. The jobs that can
a “computational lens” defined by a propagation model to be broken into multiple tasks that in turn be handed out to
yield an image of the inhomogeneties. individual workers for simultaneous execution, are most
suitable for parallel machines.
By far the widest commercial application of non-intrusive
imaging and that, for which the algorithms are most Recently, cluster of workstations or network of workstations
sophisticated, is seismic exploration for oil and gas. The has gained popularity as they provide a very cost-effective
seismic experiments are generally based on wave propagation, parallel-computing environment. Most of these clusters use
where ray paths are strongly curved by variations of Network File System (NFS) and MPI (Message Passing
compressional and shear wave velocities with depth. The Interface) as message passing library. MPI calls allow us to
wavefield can be trapped and multiply reflected between communicate and synchronize between the processors. One
lithological layers. By solving the one-way scalar wave limitation of NFS is that the I/O nodes are driven by standard
equation or full scalar wave equation with recorded data as the UNIX read and write calls, which are blocking requests. This
initial condition, we solve for the complexities of the wave is not a problem for applications with small volume of I/O, but
propagation, leading to an image of the reflecting surfaces as the volume increases (as in 3D seismic acquisition), it is
necessary to be able to overlap computations with the I/O to

SPG 4th Conference & Exposition on Petroleum Geophysics — Mumbai, India, 7 - 9 January 2002 Page 1
maintain efficient operation (Olfield et al., 1998, Poole, 1994). as post-stack migration. Migration can also be carried out in
In the present study we have used both MPI and MPI I/O to the prestack domain and the results obtained are more accurate
improve the performance and efficiency of the codes than that of poststack domain. However the computational
(Bhardwaj et. al. 2000). requirements of prestack migration algorithms is orders of
magnitude more than that of poststack migration algorithms.
Conceptually, MPI consists of distributed support software Processor speed, memory and I/O play a crucial role in the
that executes on participating UNIX / LINUX hosts on a implementation of these algorithms.
network, allowing them to interconnect and cooperate in a
parallel distributed computing environment. MPI offers an Most of the migration methods comprise of two steps,
inexpensive platform for developing and running application. extrapolation and imaging. In the extrapolation step the
Heterogeneous machines can be used in a networked wavefield is downward continued using some form of the
environment. The MPI model is a set of message passing acoustic wave equation. At each depth the image is formed at t
routines, which allows data to be exchanged between tasks by = 0. The extrapolation of the wavefield can be carried out in t-
sending and receiving messages. x-y, ω-x-y or ω-kx-ky domain. Here we shall describe the
implementation of migration in ω-x-y and ω-kx-ky domains.
Seismic Data Processing occupies a significant role in the
Another technique, Reverse Time Migration (RTM), which
exploration of oil and natural gases. Over the last two decades
makes use of the full wave equation is also developed and
the computational requirements of the SDP activities have
implemented on PARAM.
grown up many folds due to the increase in the data volume as
well as the development in the mathematical algorithms. Three 3D Depth Migration in ω-x-y domain
dimensional data acquisition has become routine as it has
become necessary to look at the minor details of the For 3D depth migration, the extrapolation equation in ω-x-y
underground geology. domain is a parabolic partial differential equation (Claerbout
1985) consisting of a diffraction term and a thin lens term. The
Wave equation based methods (Phadke et.al. 1998) are thin lens term, which accounts for lateral velocity variations, is
gaining more and more popularity in recent years as they usually ignored in time migration. The diffraction term is
provide finer detailed geological features than other numerically solved by the method of splitting, which is the
conventional methods as well as they preserve amplitude basis for the onepass approach. A Crank-Nikolson finite
information. Advanced techniques are distinguished primarily difference scheme with absorbing boundary conditions on the
by their use of wave equation. The most common advanced sides of the model is used for the solution. The thin lens term
techniques include seismic migration and forward modelling. is solved analytically. Imaging is the summation of all the
Finite difference methods are most suitable for migration and frequencies at t=0 for each depth.
modelling as they offer most direct solution to the problem in
terms of the basic equation and initial and boundary
conditions.
By nature most seismic problems carry an inherent parallelism
in subdivision by source, receivers, frequency or wave
number. Indeed the problem decomposition in several
domains is possible. With the change in demand it has become
very difficult for a processing facility build around a serial
architecture machine to cope up with increase in data volume.
The I/O problems are also better solved in parallel processing.
The wave equation based methods are computationally more
expensive but suitable for parallelization. The seismic
processing industries all over the world have found parallel
processing as the only solution to the challenges in probing the
earth’s interior for natural resources.
The digital data that needs to be processed before obtaining an
interpretable image of the subsurface geological structures in
enormous, amounting to 100s of GB (Giga Bytes) or a few TB
(Tera Bytes) for 3D acquisition. All this numerical input will
be passed perhaps 10 to 20 times through a major computer
facility, and only after the complex numerical operations, the
final processed sections are examined by geophysicists and
geologists to formulate an initial or penultimate interpretation.
Parallel processing is the only answer to cope with increase in Figure 1: (a) Zero-offset section of a line from 3D volume of
data volume and changes in processing methodology. we are SEG/EAGE overthrust model. (b) 3D depth migrated
fortunate that Seismic Data Processing (SDP) is an ideal section. The velocity model is also superimposed on the
application for parallel architecture machines. migrated section.
Migration Algorithms The depth migration algorithm in ω-x-y domain is inherently
The stacking of seismic data is a form of data compression, parallel in terms of frequencies. The parabolic approximation
which improves signal-to-noise ratio and produces idealized of the wave equation in frequency-space domain has
seismic traces simulating a coincident source-receiver decomposed the wave field into monochromatic plane waves
experiment. Migration of the resultant data set, called the zero- that are propagating downwards. Therefore, each frequency
offset seismic section or the post-stack time section, is known harmonic can be extrapolated in depth independently on each
processor and there is no need of inter-task communication.
SPG 4th Conference & Exposition on Petroleum Geophysics — Mumbai, India, 7 - 9 January 2002 Page 2
One can introduce parallel task allocation into each frequency 3D Depth migration with PSPI Algorithm
harmonic component with the ultimate goal being to have as
many processors as frequencies. At each depth step all The phase-shift migration method (Gazdag, 1978) downward
frequency components after extrapolation are summed up continues the wavefield in wavenumber-frequency domain,
(Imaging Condition) to give the migrated image. The under the horizontally layered velocity assumption. If the
summation is carried out by automatic merging using migration velocity has no horizontal variations, the phase-shift
MPI_Reduce. MPI I/O is used for reading and writing input method extrapolates the wavefield exactly by rotating the
data, velocity data and output data. phases of each Fourier component. In the presence of lateral
velocity variations, the exact extrapolation equation is no
We first tested the migration algorithm for the data set of longer valid. PSPI (Phase Shift Plus Interpolation) method
SEG/EAGE (1997) Overthrust model. The original data had circumvents the problem of lateral changes in migration
101X25 CDP traces with inline spacing of 100m and crossline velocity by downward extrapolating the wavefield with
spacing of 100m. We interpolated this data volume to 401X97 several reference velocities and then interpolating the
CDP traces to make both inline and crossline spacing 25m for wavefield for the correct velocity (Gazdag and Sguazzero,
avoiding spatial aliasing. The input Fourier Transformed data 1984).
size was of the order of 46MB. This data set was migrated
with a depth step of 25m for 161 depth steps. Figure 1 shows The parallel implementation of the PSPI method is also
the zero-offset section for one of the lines and the 3D migrated straightforward. The method is inherently parallel in terms of
data for the same line. The velocity model is also frequency. Here also the data if first Fourier transformed and
superimposed on the migrated data to show the accuracy of then different processors read and migrate their share of
the migration algorithm. Figure 2 illustrates the execution time frequencies. At each depth step phase-shift are applied for the
as a function of number of processors. Since the problem size reference velocities and then wavefield is interpolated for the
is small the speedup is not linear. actual velocity. One of the processors, which act as the master,
collects and images the data. The method was developed and
The second data set used for testing comprised of 950X665 implemented on PARAM 10000 and was tested by applying it
CDPs. The inline spacing was 25m, the crossline spacing was to both synthetic and real data sets.
37.5m, and the depth step size was 12.5m. The data was
migrated for 480 depth steps. Table 1 shows all the other Reverse Time Migration (RTM)
parameters and the time required to migrate this data set with Reverse time migration technique solves the full wave
64 processors. It is not possible to carry out a speedup analysis equation by extrapolation in time, allowing both the upgoing
on this data volume, since there is not enough memory and downgoing wave to propagate. The full wave equation is
available on a smaller number of processors and the execution solved using finite-differences and the wavefield recorded at
time required will also be very large. the surface is used as boundary condition. McMechan (1983)
has given the description of the method in detail and
demonstrated its ability to image all dips with great accuracy.
Time marching of the wavefield is similar to any modelling
algorithm. The parallelization is carried out using domain
decomposition scheme. A good description of wave
propagation using finite differences is given in the next section
on modelling algorithms.
RTM has the same problems with the stability and numerical
dispersion that finite-difference (FD) modelling has, and it is
straightforward (but computationally expensive) to control
these problems. We have implemented a central difference FD
scheme for RTM on PARAM 1000 using domain
decomposition. The application of the method to both the
synthetic and real data sets will be shown during presentation.
Modelling Algorithms
A basic problem in theoretical seismology is to determine the
wave response of a given earth model to the excitation of an
Figure 2: Number of processors versus execution time chart impulsive source by solving the wave equation. In scalar
for SEG/EAGE Overthrust model. approximation, the acoustic wave equation may be solved to
evaluate the waveform but only compressional waves are
Size of FFT data 1.3 GB considered. A more complete approach is to study the vector
displacement field using the full elastic wave equation for
Size of Velocity model 1.2 GB modelling both, compressional waves and shear waves.
However, important wave properties such as attenuation and
Frequency band 5 - 40 Hz dispersion require a more sophisticated set of equations. These
Number of Processors 64 properties will be incorporated in the future versions of codes.
2D Acoustic / Elastic Wave Modelling
Total Execution time with MPI-IO 7 hrs 44 mins
The mathematical model for elastic wave propagation in 2D
Table 1: Problem size for the second data set and the heterogeneous media consists of coupled second order partial
execution time on 64 processors. differential equations governing motions in x- and z-
directions

SPG 4th Conference & Exposition on Petroleum Geophysics — Mumbai, India, 7 - 9 January 2002 Page 3
 ∂u& ∂σ xx ∂σ xz edges of the model (Sochaki et al. 1987). Free-surface
ρ = + (1) boundary condition is used for top edge.
∂t ∂x ∂z
The parallel implementation of an algorithm involves the
∂w
& ∂σ xz ∂σ zz
ρ = + (2) division of total workload into a number of smaller tasks,
∂t ∂x ∂z which can be assigned to different processors and executed
and the stress-strain relations are given by concurrently. This allows us to solve a large problem more
quickly. The most important part in parallelization is to map
∂u ∂w out a problem on a multiprocessor environment. The choice of
σ xx = (λ + 2µ) +λ (3) an approach to the problem decomposition depends upon the
∂x ∂z
computational scheme. Here we have implemented a domain
∂u ∂w decomposition scheme.
σ xz = λ ( + ) (4)
∂z ∂x The idea of this scheme is simple. First, the problem domain is
divided into a number of subdomains that are assigned to
∂u ∂w
σ zz = λ + (λ + 2µ) (5) separate processors. The upper part of Figure 3 shows an
∂x ∂z example of the division of problem domain into nine
Where u and w are horizontal and vertical displacements, subdomains. Depending upon the number of available
processors and the problem, one can divide the problem
u& and w
& are the horizontal and vertical particle velocities,
domain into any number of subdomains. Since MacCormack
σ xx , σ zz and σ xz are the stress components, λ and µ DUH scheme uses a nine-point difference star, the calculation of the
the Lamé parameters and ρ is the density. wavefield at an advanced time level for any grid point,
requires the knowledge of the wavefield at 9 grid points of the
Instead of solving these second order coupled partial current time level. For grid points along the boundaries of the
differential equations we formulate them as a first order subdomain, the information about the neighbouring grid points
hyperbolic system (Virieux 1986, Vafidis 1988, Dai et al. comes from the adjacent subdomains. Therefore after each
1996): time step the subdomains have to exchange some wavefield
data. Lower part of Figure 3 shows the required memory space
∂Q ∂Q ∂Q
=A +B (6) for each 2D array of the subdomain and the communication
∂t ∂x ∂z between two adjacent subdomains. The data in the darker
Where, region is sent to the lighter region of the neighbouring
subdomian using MPI message passing calls.
 u&   0 0 ρ −1 0 0  The two most important issues in this implementation are (1)
 &   
 w   0 0 0 0 ρ −1  to balance workload (2) to minimize the communication time.
Q = σ xx  , A = λ + 2µ 0 0 0 0  In a homogeneous multiprocessor environment, as in our case,
    the load balancing is assured if all the subdomains are of the
 σ zz   λ 0 0 0 0  same size. Minimizing the perimeters of the subdomain
σ   
 xz   0 µ 0 0 0  boundaries minimizes communication.
0 0 0 0 ρ −1 
  Subdomain Subdomain Subdomain
 0 0 0 ρ −1 0  1 2 3
and B = 0 λ 0 0 0 
 
 0 λ + 2µ 0 0 0 
 
µ 0 0 0 0 
Subdomain Subdomain Subdomain
4 5 6
When we move from elastic to acoustic media, the value of µ
becomes zero. By substituting µ   LQ WKH DERYH HTXDWLRQ ZH
get a first order system of hyperbolic partial differential
equations which governs the acoustic wave propagation.
Subdomain Subdomain Subdomain
p  0 K 0  0 0 K 7 8 9
&    
Q =  u  , A = ρ −1 0 0 and B =  0 0 0  (7)
−
 w
& 
 0 0 0 ρ 0 0 
1

Where p is the negative pressure wavefield, and K=λ is the
incompressibility.
For solving the first order hyperbolic system (6), we use the
method of splitting in time (Vafidis 1988). An explicit finite
difference method based on the MacCormack scheme is used
for the numerical solution (Mitchell and Griffiths, 1981). This Figure 3: The upper picture shows the division of problem
scheme is fourth order accurate in space and second order domain into a number of subdomains. The lower picture
accurate in time. The model discretization is based upon shows the communication between two adjacent tasks.
regular grid. Sponge boundary conditions are used for In the MPI implementation of the modelling codes there is a
attenuating the reflected energy from the left, right and bottom
master task and there are a number of worker tasks. The main
job of master task is to divide the model domain into
SPG 4th Conference & Exposition on Petroleum Geophysics — Mumbai, India, 7 - 9 January 2002 Page 4
subdomains and distribute them to worker tasks. The worker shortest wavelength. The finite difference approximation (2) is
tasks perform time marching and communicate after each time stable if
step. As per the requirement of the user the snapshot and
min(∆x , ∆y, ∆z)
synthetic seismogram data are collected by the master and ∆t ≤ (10)
written out on the disk. 2 Vmax
The wave propagation described by equation 6 is valid for where ( V = K ρ) and Vmax is the maximum wave velocity in
both acoustic and elastic media. This is because when the
Poisson’s ratio becomes 0.5 the medium becomes acoustic the medium.
(Phadke et. al. 2000). The upper part of Figure 4 shows the P- 0.5 1.0 1.5 2.0
wave velocity model used for calculating the synthetic data in 4000
a marine environment. There is a water layer at the top. The
3500
water bottom is quite undulating. Poisson’s ratio and density
in other layers are 0.25 and 2.2gm/cc respectively. The 3000
snapshots of the wave propagation through this model are also
shown in Figure 4. The synthetic seismogram data for this 2500

model are shown in Figure 5. A gain function is applied for 2000

display purposes. Since the free-surface boundary condition is
used for the top edge, all kinds of multiples are also modeled. 1500
The example clearly demonstrates the capability of this Marine Velocity Model
approach for generating synthetic seismograms in realistic
marine models. Another advantage of this approach is that
acoustic and elastic wave propagation is modeled by the same
code (µ = 0 for acoustic wave propagation).
3D Acoustic wave Modelling
The acoustic wave equation in a 3D heterogeneous medium is
given by

1 ∂ 2p ∂  1 ∂p  ∂  1 ∂p  ∂  1 ∂p  Snapshot at 0.08 sec

=  +  +   (8)
K ∂t 2 ∂x  ρ ∂x  ∂y  ρ ∂y  ∂z  ρ ∂z 

Where, p is the negative pressure wavefield, , ρ is the density

and K is the incompressibility.
We divide the 3D geological model into a grid of I X J X K
points. In order to obtain finite difference approximation to
equations (1), let us introduce a set of indices i, j, k and n such
that
Snapshot at 0.2 sec
x = i ∆x i = 0 ,1, 2 ,...... I , y = j ∆y j = 0 ,1, 2 ,...... J

z = k ∆z k = 0 ,1, 2 ,...... K , t = n ∆t n = 0 ,1, 2 ,...... N

where ∆x, ∆y and ∆z are the grid spacing and I ,J and K are the
number of grid points in x- y- and z- directions respectively,
∆t is the time step and N is the total number of time steps.
Physical parameters, density ρ(i,j,k) and incompressibility
K(i,j,k) are specified at each grid point.
Substituting central difference approximations of the
derivatives in equation (1), an expression is obtained for Snapshot at 0.4 sec
calculating the wavefield p in,+j,1k from the knowledge of the
Figure 4: Snapshots of the wave propagation through the
wavefield at previous time levels i.e. p in, j,k and p in, −j,1k as marine velocity model. Free surface boundary condition is
applied on the top edge of the model and absorbing
p in. +j,1k = 2p in, j,k − p in, −j,1k + A i, j,k p in−1, j,k + Bi, j,k p in+1, j,k boundary conditions are applied to left, right and bottom
edges of the model.
+ D i, j,k p in, j−1,k + E i, j,k p in, j+1,k + Fi, j,k p in, j,k −1
(9) Since a digital computer has finite memory capabilities, we
+ H i, j,k p in, j,k +1 − ( A i, j,k + Bi, j,k + D i, j,k
have to restrict the model size to a fixed number of grid points.
+ E i, j,k + Fi, j,k + H i, j,k ) p in, j,k This introduces artificial boundaries at the edges of the model.
In reality the earth is infinite and therefore all the energy
where A, B, D, E, F, and G are the functions of physical impinging on these boundaries must be absorbed. For the
parameters K and ρ (Phadke et. al. 2000). finite difference scheme presented here a sponge boundary
condition as described by Sochacki et.al. 1987, is used for
Equation (9) is programmed to calculate the wave propagation
attenuating the energy impinging on the left, right, bottom,
in heterogeneous media. This approximation is second order
front and back edges of the model. To implement sponge
accurate in both space and time. Grid dispersion is minimized
boundary condition extra grid points are added to gradually
by keeping the grid spacing smaller than one tenth of the

SPG 4th Conference & Exposition on Petroleum Geophysics — Mumbai, India, 7 - 9 January 2002 Page 5
attenuate the energy. The free-surface condition is applied to workers, compiles it and writes it on the disk in a proper
the top boundary. manner.
Distance (km) Finite-difference computation of the snapshots can help in our
0.4 0.6 0.8 1.0 1.2
0 understanding of wave propagation in the medium. We have
used a constant velocity model as a numerical example for
0.1
generating snapshots of 3D acoustic wave propagation. Source
is placed at the center of the cubic model. For simplicity sake
there is no density variation within the model. However, the
0.2 algorithm can handle density variations. The source wavelet
used for calculation of snapshots is the second derivative of a
0.3 Gaussian function with a dominant frequency of 30Hz. Figure
Two-way time (sec)

7 shows the snapshots of the 3D acoustic wave propagation

through the constant velocity model.
0.4

0.5

0.6
t = 0.07 sec

0.7

Figure 5: Synthetic seismogram for the marine model. A

uniform gain function is applied for plotting purposes.
In the second order central difference scheme implemented t = 0.075 sec
here, one can observe that the calculation of the wavefield at a
grid point at an advanced time level involves the knowledge of
the wavefield at five grid points of the current time level and
one grid point of the previous level. Therefore, it is a seven
point differencing star. Therefore, if we use a domain
decomposition scheme for solving this problem only first
order neighbors will be involved in communication for central
difference scheme.
t = 0.08 sec
The parallel implementation of the algorithm is based on
domain decomposition. Domain decomposition involves
assigning subdomains of the computational domain to
different processors and solving the equations for each
subdomain concurrently. The problem domain is a cuboid as Figure 7: Snapshots of the 3D acoustic wave propagation
shown in the figure 6, through the constant velocity model.
y We performed the benchmark tests of the parallel algorithm
for problem size of 400 X 400 X 400 and a smaller problem
x size of 200 X 200 X 200. The grid spacing in all three
directions was 2m. A time step of 0.0001sec was used and the
wave propagation was carried out for 0.1sec. Since the model
z size 400 X 400 X 400 is too large to fit into the processor
memory of PARAM 10000, the test was performed using
minimum of 8 processors for the bigger model.
Figure 6: Problem domain (grid point size: I X J X K).
We have used three types of partitioning for the domain
This domain can be partitioned in three ways viz., stripe, decomposition and have experimented with all the three types.
hybrid stripe and checkerboard (Phadke et. al., 2000). The For implementation point of view all three types of
checkerboard partitioning involves the least communications partitioning play an important role on the basis of memory
and therefore is the most efficient. access pattern. Theoretically, checkerboard partitioning has
Again a message passing paradigm, MPI, is used for the best memory access pattern as the partitioned data can
implementation of the 3D modelling algorithm. The present reside in the first level of the cache available. In the case of
implementation is analogous to a Master-Worker system, stripe and hybrid stripe partitioning, the access of data from
where master works as the manager and assigns tasks to his memory may require swapping between first and second
workers. The main job of master is to provide the required levels of cache, which is an expensive operation. Hybrid stripe
data to all the workers and distribute workload properly, so partitioning has better access patter as compared to stripe
that the idle time of the workers is minimized. Also, at the partitioning. Bar charts for execution time verses number of
end, the master collects the completed work from all the processors for 3D acoustic wave modeling, shown in Figure
8a for two different problem sizes, support this statement.

SPG 4th Conference & Exposition on Petroleum Geophysics — Mumbai, India, 7 - 9 January 2002 Page 6
A speedup analysis for the two model sizes (Figure 8b) shows Bhardwaj, D., S. Phadke and Sudhakar Yerneni, 2000, On improving
a sub-linear speedup as we increase the number of processors. performance of migration algorithms using MPI and MPI-IO,
For a fixed model size the compute to communication ratio Expanded Abstracts, Society of Exploration Geophysicists.
decreases with the increase in the number of processors. Claerbout, J. F., 1985, Imaging the Earth’s interior, Blackwell
Therefore if we increase the size of the problem, better Scientific Publications
speedup can be achieved for large number of processes.
Dai, N., Vafidis, A., and Kanasewich, E. R., 1996, Seismic migration
Conclusions and absorbing boundaries with a one way wave system for
heterogeneous media, Geophysical Prospecting, 44, 719-739.
In this paper we have presented several migration and
Gazdag, J, 1978, Wave equation migration with the phase-shift
modelling algorithms for seismic imaging on a parallel method, Geophysics, 43, P. 1342-1351.
distributed computer. PSPI algorithm and ω-x-y algorithm are
both parallelized in frequency domain. RTM algorithm is Gazdag, J, and Sguazzero, P, 1984, Migration of seismic data by
parallelized by domain decomposition. Highly efficient and phase shift plus interpolation, Geophysics, 49, P. 124-131.
scalable codes were developed for these algorithms and McMechan, G. A., 1983, Migration by extrapolation of time-
implemented on PARAM 1000. The algorithms were tested dependent boundary values, Geophysical Prospecting, 31, P. 413-
for both synthetic and real data sets. Modelling algorithms for 420.
wave propagation in heterogeneous media were developed and Mitchell, A. R. and Griffiths, D. F., 1981, The finite difference
parallelized using a domain decomposition scheme. Efficient method in partial differential equations: John Wiley & Sons Inc.
codes for both acoustic and elastic wave propagation were
developed. These codes form an integral part of the seismic Oldfield, R. A., D. E. Womble and C. C. Ober, Efficient Parallel I/O
inversion algorithms for estimating the physical properties of in Seismic Imaging, The Int. J. of High Performance Computing
Applications, Vol. 12, No. 3, Fall 1998, pp 333-344.
the subsurface.
Phadke, S., Bhardwaj D. & Yerneni, S., 1998, Wave equation based
(a) migration and modelling algorithms on parallel computers, Proc. of
5000 SPG (Society of Petroleum Geophysicists) second conference, pp.
55 - 59.
4000
Phadke, S., D. Bhardwaj and S. Yerneni, 2000, 3D seismic modeling
3000 in a message passing environment, In proceedings of 3rd
Execution
Time (sec) 2000 Conference and Exposition on Petroleum Geophysics (SPG’2000),
P. 168-172.
1000 Phadke, S., and D. Bharadwaj and Sudhakar Yerneni, 2000, Marine
synthetic seismograms using elastic wave equation, Expanded
0 Abstracts, Society of Exploration Geophysicists.
8 16 32 64
(b) Poole, J., Preliminary survey of I/O intensive applications, Technical
Report CCSF-38, Scalable I/O initiative, Caltech Concurrent
600 supercomputing facilities, California Institute of Technology,
500 Pasadena, 1995.

400 SEG/EAGE 3-d modeling series No. 1, 1997, 3-D salt and overthrust
models, SEG publications.
Execution 300
Time (sec) 200 Sochacki, J., Kubichek, R., George, J., Fletcher, W. R. and Smithson,
S., 1987, Absorbing boundary conditions and surface waves:
100 Geophysics, 52, 60-71.
0 Vafidis, A., 1988, Supercomputer finite difference methods for
8 16 32 64 seismic wave propagation: Ph.D. Thesis, University of Alberta,
No. of Processors Edmonton, Canada.
Figure 8: Comparison of execution time for Stripe, Hybrid- Virieux, J.,1986, P-SV wave propagation in heterogeneous media:
Stripe and Checkerboard partitioning for 3-D acoustic wave velocity stress finite difference method: Geophysics, 51, 889-90.
modeling for two model sizes viz., (a) 400 X 400 X 400,
(b) 200 X 200 X 200.
Acknowledgements
Authors wish to thank the Executive Director, C-DAC for
providing computational facility on PARAM 10000 and
permission to publish this work. We are also thankful to
Department of Science and Technology, Government of India,
for funding a part of this study under DCS (Deep Continental
Studies) program. Discussions with the scientists of GEOPIC,
ONGC, Dehradun, were also helpful in improving the quality
of the codes.
References
Bhardwaj, D., S. Yerneni, and S. Phadke, 2000, Efficient parallel I/O
for seismic imaging in a distributed computing environment, In
proceedings of 3rd Conference and Exposition on Petroleum
Geophysics (SPG’2000), P. 105-108.

SPG 4th Conference & Exposition on Petroleum Geophysics — Mumbai, India, 7 - 9 January 2002 Page 7