IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 49, NO.

7, JULY 2003

1801

Low-Density Parity-Check Codes Over Gaussian Channels With Erasures

Jeongseok Ha and Steven W. McLaughlin, Senior Member, IEEE
(a) AbstractWe consider low-density parity-check code (LDPCC) design for additive white Gaussian noise (AWGN) channels with erasures. This model, for example, represents a common situation in magnetic and optical recording where defects or thermal asperities in the system are detected and presented to the decoder as erasures. In this correspondence, we give thresholds of regular and irregular LDPCCs and discuss practical code design over the mixed Gaussian/erasures channel. The analysis is an extension of the Gaussian approximation work of Chung et al. In the two limiting cases of no erasures and large signal-to-noise ratio (SNR), the analysis tends to the results of Chung et al. and Luby et al., respectively, giving a general tool for a class of mixed channels. We derive a steady-state equation which gives a graphical interpretation of decoder convergence. This allows one to estimate the maximum erasure capability on the mixture channel, or conversely, to estimate the additional signal power required to compensate for the loss due to erasures. We see that a good (capacity-approaching) LDPCC over an AWGN channel is also good over the mixed channel up to a moderate erasure probability. We also investigate practical issues such as the maximum number of iterations of message-passing decoders, the coded block length, and types of erasure patterns (random/block erasures). Finally, we design an optimized LDPCC for the mixed channel, which shows better performance if the erasure probability is larger than a certain value (0 1 in our simulation) at the expense of performance degradation at unerased (AWGN channel) and lower erasure probability regions (less than 0 1 in our simulation). Index TermsGaussian approximation, Gaussian noise, low-density parity-check codes (LDPCCs), random erasures.

(b) Fig. 1. AWGN with random erasures. (a) x is a message bit in 1; 0 , c is a coded symbol in 1; +1 , n is white Gaussian noise (n (0;  )), e is in 1; 0 , P (e = 0) = e , p(e = 1) = (1 e ), and x is a decoded bit ^ in 1; 0 . (b) r is a received signal and m is the mean of the received signal.

f f

g g

f0

N

I. INTRODUCTION Low-density parity-check codes (LDPCCs) are due to Gallager [1], [2], and renewed interest began as a result of a great success of turbo codes [3] and subsequent rediscovery by MacKay [4], MacKay and Neal [5], and Sipser and Spielman [6]. Since the rediscoveries, much effort has been paid to designing good LDPCCs over various channels such as binary erasure channels (BECs), binary symmetric channels (BSCs), and additive white Gaussian noise (AWGN) channels. Luby et al. [7] designed good LDPCCs over BECs and introduced an irregular edge degree distribution. Richardson and Urbanke [8] investigated variations of message densities in message-passing decoders and devised an algorithm called density evolution for iteratively computing message densities and the signal-to-noise ratio (SNR) threshold of an LDPCC. Although Chung [9] introduced a simplified density evolution called discretized density evolution (DDE), computing thresholds and designing good LDPCCs over most channels, except BECs, require intensive computations. To address the computational difficulty, Chung [10] used the Gaussian approximation (GA) that models the message densities in message-passing decoders as Gaussian distributions for regular LDPCCs and a mixture of Gaussian distributions for irregular
Manuscript received April 29, 2002; revised February 26, 2003. This work was supported in part by the National Science Foundation under grant CCR-0121565 and the State of Georgia Yamacraw Initiative. The material in this correspondence was presented in part at the IEEE International Symposium on Information Theory, Lausanne, Switzerland, June/July 2002. The authors are with the School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332-0250 USA (e-mail: [email protected]; [email protected]). Communicated by K. Abdel-Ghaffar, Associate Editor for Coding Theory. Digital Object Identifier 10.1109/TIT.2003.813480

LDPCCs. A prerequisite of the approximation is the symmetry condition [11]. At the expense of accuracy, the approximation gives a faster threshold computation, more insight into the convergence of message-passing decoders, and an analytic expression of the variations of the message densities during iterations. From the design point of view, the approximation makes the design problem a linear optimization that can be easily solved with linear programming. In this correspondence, we extend Chungs work to include a channel model that is a mixture of Gaussian noise with random erasures, as depicted in Fig. 1(a) and (b). In this model, a binary symbol is corrupted with either added white Gaussian noise or is erased with a probability of e(0) . This model, for example, represents a common channel in magnetic recordings where thermal asperities in the system are detected and represented at the decoder as erasures. A typical code must correct a burst of up to 48 bytes over a 512-byte sector. In an optical recording, defects or scratches cause long numbers of bytes (several hundred or thousand) to be erased. The probability density function of the log-likelihood ratio (LLR) of the channel output is

f (v) = p

= g (v) + e 1 (v)
(0) (0) 0

4mu

0e

(0)

+ e 1 (v)
(0) 0

(1)

where v =p rjc 0 is the LLR of r , r is the e p r jc channel output shown in Fig. 1(b), mu E v jv 6 , v jv 6  v 0 x is mu , e(0) is the random erasure probability, and x v a shifted delta function. The probability density function satisfies the symmetry condition [11], f v f 0v ev . We analyze the message density evolution over the channel model shown in Fig. 1(a) and (b) with some modifications to the Gaussian approximation. The remainder of the correspondence is organized as follows. In Section II, for the readers convenience, we briefly summarize Chungs work in [10]. In Section III, we consider regular LDPCCs and define the basic terminology used in the subsequent sections. In Section IV, we extend the results of Section III to irregular LDPCCs. In Section V, we simplify the results of Section IV using the fact that the erasure

= log [ ( =1) ( = 1)] = [ = 0] Var( = 0)= 1 ( )= ( ) ( )= ( )

0018-9448/03$17.00 2003 IEEE

1802

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 49, NO. 7, JULY 2003

probability during iterations does not depend on an SNR. The simplified equation, called a steady-state equation, gives us a graphical interpretation of decoder convergence that was originally introduced in [10]. In Section VI, we verify our analysis and the channel model by comparing theoretical Eb =N0 thresholds of regular and irregular LDPCCs with simulation results of actual LDPCCs. We also investigate some practical issues such as maximum number of iterations of message-passing decoders, coded block length, and types of erasure patterns (random/block erasures). In Section VII, we design an LDPCC for the mixed channel and compare the designed LDPCC with an LDPCC optimized for an AWGN channel in [10]. We conclude our work in Section VIII. II. GAUSSIAN APPROXIMATION (GA) Our work is based on the results of Gaussian approximation (GA) in [10]. In this section, we briefly summarize GA to help readers understand the following sections. This summary is the minimum needed to understand our work and describes GA for irregular LDPCCs only, which includes regular LDPCCs as a special case. The LLR message (message for short) of a check node at the k th iteration (u(k) and k 2 ) during message-passing decoding is approximated as a Gaussian probability density function that is completely (k) specified by its mean (mu ) and variance (Var(u(k) )). The symmetry x condition, f (x) = e f (0x), which is preserved during iterations, im(k) poses a relation between the mean and variance, Var(u(k) ) = 2mu . Thus, by tracing the changes of the mean (called updated mean) during iterations, we can watch the evolution of the density of the check node (k) (k) message, f (k) (u) = N (mu ; 2mu ), where N (; 2 ) is the normal probability density function with a mean and variance of  and  2 , respectively. The probability density function (density for short) of a variable-node message at the k th iteration (v (k) ) is approximated as a Gaussian mixture for irregular LDPCCs and a Gaussian for regular LDPCCs
d

where u is a Gaussian, mu = E [u], and Var(u) = 2mu . Thus, both sides of (3) become integral forms that are simplified by the following definition. Definition 1 [10, Definition 1]:

if x = 0. 1; (x) is continuous and monotonically decreasing on [0; 1), with (0) = 1 and (1) = 0. By applying (4) and (x) to both sides of (3), we can calculate the updated mean of a degree j check node as
(k mu; )j

(x) =

0 p1

4x

tanh

u
2

du;

if x > 0

01

0 10

(j (k i  mv; ) i

01)

i=2

The check node updated mean becomes a recursive equation

( muk) =
d

(k j mu; )j

j =2 d

=
j =2

j 01

0 10

i  mu +(i 0 1)mu 01)

(k

(j

01)
:
(5)

1, which implies
d i=2

For error-free decoding, the recursive equation converges to 1 as k

i=2

i  mu

+ (i

0 1)m 0
(k u

1)

! 0;

as k

!1

because 01 (x) ! 1 as x ! 0. For 0  s < 1 and 0 < r  1, hi (s; r) and h(s; r) are defined in [10] as

(k)

(v ) =
i=2

i N mv; i ; 2mv; i
(k) (k)

hi (s; r) =  s + (i 0 1)
d

j 01

j =2

0 (1 0 r) 0
j

where i is the fraction of edges belonging to degree i variable nodes, (k ( mv; ) = mu + (i 0 1)muk01) , mu is the mean of an LLR message i (0) out of channel, and mu = 0. Between a check node of degree dc and the variable nodes, there is the well-known tanh rule [12]:

h(s; r) =
i=2

i hi (s; r) rk = h (s; rk01 ) and r0 = (mu ).

and (5) is equivalently expressed as (6) where s = mu In the recursion, the degree distribution pair ((x); (x)) and s determine the convergence to a fixed point (not necessarily zero). As (k) shown in (5) and (6), mu ! 1 if and only if rk ! 0. Thus, the convergence condition rk (s) ! 0 is satisfied if and only if

tanh

(k)

01
tanh

=
j =1

vj

(k)

(2)

where vj , j = 1; . . . ; dc 0 1 are the incoming messages from dc 0 1 neighbors and u(k) is the message sent to the remaining node. The expectation of each side of (2) can be simplified as
(k)

h(s; r) < r;

8 r 2 (0; (s)):

(7)

tanh

u(k)
2

tanh

v(k)
2

01

Definition 2 [10, Definition 2]: The threshold s3 is the infimum of (k) all s in + such that rk (s)(mu (s)) converges to 0(1) as k ! 1. III. ANALYSIS OF REGULAR LDPCCS The message flows between a check and a variable node are depicted in Fig. 2(a) and (b), where the square and circle symbols represent check nodes and variable nodes, respectively. The function h(k) (u) (k) (g (v )) is the continuous part of the probability density of a check (variable) node message. The term "(k) (e(k) ) is the probability that a check (variable) node messages is equal to zero. dc (dv ) is the number

(3)

Because the variable node messages are independent and identically distributed (i.i.d.), we can omit the index j in (2) and simplify the product. The expectation E tanh u can be expressed as 2

tanh

u
2

1 p4m

tanh
u

u
2

du

(4)

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 49, NO. 7, JULY 2003

1803

sum of dv 0 incident check node messages as shown in Fig. 2. Thus, the probability that the message of a variable node is zero can be computed as

P v(k) = 0

=e

(0)

"(k01)

01

(11)

(a)

which is the probability that all incident check messages are zeros and the variable node is erased. The probability density of a variable message can be factored into three terms as shown in (12). The first term gives the probability in (11), the second term is made up of the incident check node messages given the variable node is erased, and the last term is the combination of the check node messages with the unerased received message. The last two terms comprise the continuous part g(k) v used in (10).

() f k (v) = e k 1 (v) + g k (v) = e k 1 (v)

( ) ( ) 0 ( ) ( ) 0

Unrecovered erased symbols

+e
(b) Fig. 2. Message flows between a check and a variable node. (a) LLR message of a check node at the k th iteration. (b) LLR message of a variable node at the (k + 1)th iteration.

(0)

01

i=1
(0)

(d

01) i

(k)

( ( imuk01) ; 2imuk01)

+ 10e
2
d

Erased symbols

01

of neighbors of a check (variable) node. uj vj is the message of a check (variable) node emitting through the j th edge (hereafter, without loss of generality, we will drop the index j ). In the sum-product decoding algorithm, if at least one of the variable nodes connected to a check node is erased, the check node has zero LLR as its message. Thus, the probability of "(k) is expressed as
( )

(k )

( k)

i=0

(d

01) i

(k)

( ( imuk01) + mu ; 2imuk01) + 2mu Unerased symbols

(12) where

N (m; 2m) = p 1
(k) n m

sage is determined by one variable, the updated mean value mu at the k th iteration. The updated mean is determined from the following relation:
( )

= 1 0 P vi 6= 0 = 1 0 1 0 e : (8) i If none of d 0 1 variable messages is erased, the probabilistic characteristics of the check message can be approximated by a Gaussian density denoted by h k (u) in Fig. 2. The Gaussian density of a mesk
"
(k) (k) (k) =1 c ( )

01

01

and n m is the binomial coefficient. The continuous part of a variable-node message density derived in (12) expands (10) to
( muk) = 01

4m e = nCm " k0

(

;
n0m

1)

1 0 " k0
(

1)

10

1 (1 0 e k )d 0
( ) (0)

1 (k) ( 01) i  imuk01)

express the updated mean value as

( muk) = 01

= E tanh u =2 u 6= 0 d 0 = E tanh v k =2 v k 6= 0 d 0 = 1 01e k tanh v k =2 ; g k (v) (9) where hf (v ); g (v )i = f (v)g(v) dv , and g k (v) is the probability density of a variable node message given v k 6= 0. From (9), we can
mu
(k) (k) (k) ( ) ( ) 1 ( ) ( ) 1 ( ) ( ) ( )

10

2 10e

01

i=0
(0)

(d

0 10e 2

01
(d

i=0

( imuk01) + mu

01) i d 01
(13)

(k)

where h v= ; N x; x i 0  x . Now, the updated mean becomes a recursive equation for a given random erasure rate e(0) . IV. ANALYSIS OF IRREGULAR LDPCCS

tanh( 2) ( 2 ) = 1

()

10
(k)

(1 0 e )

To describe irregular LDPCCs, let

d

01

tanh

v(k) =2 ; g(k) (v)

(k)

01

: (10)

(x) =

d i=2

i xi01 and (x) =

d i=2

i xi01

To make (10) a recursive equation of mu , we need an expression for the probability density of a variable node message. In the log-probability domain, the variable node message is determined by the linear

be a pair of generating functions (used in [12]) of the degree distributions for the variable and check edges, respectively. To include the degree distribution in calculating the probability of "(k) and the updated

1804

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 49, NO. 7, JULY 2003

mean mu that describe the check node message density, we need to modify (8) and (9) to form (14) and (15), respectively
( "dk) s = P u(k) = 0 j dc = s =

(k )

SNR of the transmitted signal to infinity, which becomes a pure erasure channel, the probability density of a variable node message becomes

=1 0
"(k) =
d s=2 d s=2

s01 i=1

(k) P vi=1 6= 0

=10 10e k

( )

s01

( P (dc = s) "dk) s =

culated with proper conditioning on j , and the probability density of a variable node message can be factorized into three parts as we did in (12)
(k)

1 (v): The continuous part of the probability density, g k (v ), can be cal( 1) 0 ( )

f (k) (v) = e(0) 

1 0  1 0 e k0

s
(k)

10 10e
= ( )

(k)

s01

=10 10e
c

(14)

f (k) (v) = e(k) 10 (v) + g(k) (v) = e(k) 10 (v)

s check node. Thus, by making use of (10), we compute the updated

mean as

1 0  mu j d s = E tanh u k =2 u k 6= 0; d = s s0 = E tanh v k =2 v k 6= 0 (15) where P (d = s) is the probability that an edge belongs to a degree
( ) ( ) ( ) 1 c

+e

(0)

j =2
(0)

j

j 01 i=1 d

(j

01) i

(k)

N
(j

( ( imuk01) ; 2imuk01)

+ 10e
2N
d s=2

j =2

j 2

j 01 i=0

01) i

(k)

( ( imuk01) + mu ; 2imuk01) + 2mu

: (18)

mu

(k)

= =

d s=2 d s=2

P (dc = s) mu j d
(k)

Thus, the continuous part, g (k) v expands (16) to

=s

()

s 01

2 tanh

1 0 (1 0 e k )s0
( )

( muk) =

s 01
d j =2

1 0 (1 0 e1k )s0
( )

v =2 ; g
(k)

(k)

(v)

s01

(16)
s01

10

j e(0)
(0)

j 01 i=0

(j

(k) ( 01) i  imuk01)

where
( mukj)d =s

= 0 1 0 E tanh
1

v(k) =2 v(k)

6= 0

+ 10e

j 01 i=0

(k) (k01) + mu (j 01) i  imu

s01

(19) which becomes a recursive equation of the updated mean mu . Because we model the message densities as Gaussians, the bit-error probability can be computed as a weighted sum of Q functions
( Pe k) = e0 (k)

and

tanh

v(k) =2 v(k)

= (1 0 e1k )s0
( )

6= 0

s01

tanh

v(k) =2 ; g(k) (v)

s01

d j =2

The probability density of a variable node message can be calculated by modifying (11) and (12). The expression of e(k) in (11) is changed to

0j

j i=0
0

j i

(k)

Q
j i=0 j j i

( imuk)

e(k) = P v(k) = 0

+ (1 0 e )
d

d j =2

0j

(k)

( imuk) + mu

d j =2
(0)

P (dv = j ) e(0) "(k01)

d j =2 j 01 j " 01)
(k (k

j 01

=e =e

=e

j =2

0j "(k)

2
0j
j i=1 d j =2 j i
(k)

Unrecovered erasures

= e  " 01)
(0) (0)

where P dv j is the probability that an edge belongs to a degree-j variable node. The recursive erasure probability of a variable node e(k) is exactly the same as the erasure probability derived by Luby (0) et al. in [7]. There is a threshold emax ;  that is the largest real (0) value in ; satisfying emax ;   0  0 e < e for any (0) e 2 ; emax ;  . The threshold does not depend on the SNR but on the code structure specified by a degree distribution pair  x ,  x . Thus, as long as the erasure probability e(0) is less than the threshold (0) emax ;  , we can make the probability of an erased symbol as small as possible with the sum-product algorithm. If we increase the

( = )

1 0  1 0 e k0
(

1)

+e
(17)

d j =2

( imuk)

Erased symbols

(0 1] (0 ( )] ( ))

( ) ) (1 (1 ( )

+ (1 0 e )
0

0j

j i=0

j i

(k)

( imuk) + mu

(20)

))

Unerased symbols

( ( ) ( ))

where

0j

j =j i =i

i=2

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 49, NO. 7, JULY 2003

1805

and

e0

d j =2

0

j j

(k ) 0

Q(0)

= e0

j =2

(j (" 2

(k)

) )

In the error probability, the first term is half of the unrecovered erasure probability at the k th iteration, the second term is the error probability due to the erased symbols, and the last term comes from the unerased symbols. V. STEADY-STATE RECURSIVE EQUATION In this section, we provide a simplified steady-state equation which gives us a graphical interpretation of the convergence of LDPCCs. As (0) long as e(0) is less than the threshold emax (; ), e(k) gradually reduces to zero and the recursive equation (19) becomes d d ( ( muk) = s 01 1 0 1 0 j e(0)  (j 0 1)muk01) s=2 j =2 s01 (0) ( + 10e  (j 0 1)muk01) + mu (21) which is a steady-state equation. For the steady-state recursive equation to grow up to infinity, the following inequality must be satisfied d r> j e(0) hj (0; r) + 1 0 e(0) hj (mu ; r) j =2
=e
(0)

Fig. 3. h (s; r ) r for i = 2; . . . ; 20 (top to bottom), and for s = 0 and 6:320891.

0g

h(s; r )

0g
r

h(0; r) +

0e

(0)

h(mu ; r)

=H

mu ; e(0) ; r ;
d j =2

8 r 2 (0;  (mu

))

(22)

where

h(s; r) =
and

j hj (s; r) = E [hj (s; r)]

d

i=2 as defined in Section II. The inequality is the same as (7) if e(0) = 0, which corresponds to an AWGN channel. From (22), we can notice that the variable node degree distribution j must be optimized to minimize mu , while satisfying the inequality for a given erasure proba(0) bility e(0) < emax (; ), which becomes a simple linear optimization. The details of the design process will be discussed in Section VII. We explain the steady-state equation with a graphical interpretation by evaluating a 1=2 rate irregular code in [9] whose degree distributions are

hj (s; r) =  s + (j 0 1)

i 01

0 (1 0 r)i0

Fig. 4. A magnified view of e = 0:38.

H (s; e

; r)

when

= 6:320891 and

the threshold mG is the minimum of mu such that if s  mu then u H (s; 0; r) 0 r = h(s; r) 0 r < 0, 8 r 2 (0; (s)], which is the same as Definition 2 in Section II. Fig. 4 shows a magnification of H (s; e(0) ; r) 0 r when e(0) = 0:38 and s = 0:6320891. From the curve, we can see that H (s; e(0) ; r) 0 r barely avoids touching zero. If it does reach zero, the point of intersection becomes a fixed point of the recursive equation. VI. PERFORMANCE PREDICTION In this section, we verify our analysis and the channel model by comparing theoretical Eb =N0 thresholds of regular and irregular LDPCCs with simulation results of actual LDPCCs. We also investigate some practical issues concerning small maximum number of iterations of the message-passing decoders, types of erasure patterns, and short block lengths. In magnetic storage, a code block length of 4096 and coding rates of 4=5, 8=9, and 16=17 are common. In Fig. 5, we compare simulation results of corresponding regular LDPCCs for a bit-error rate (BER) of 04 with the thresholds predicted by GA. In the simulation, we limit 10 the maximum number of iterations of the message-passing decoder to 25. The computed theoretical thresholds fairly predict asymptotic performances of the LDPCCs, which confirms our analysis of regular LDPCCs over the mixed channel.

(x) = 0:23403x +0:21242x2 +0:14690x5 +0:10284x6 19 +0:30381x

and In Fig. 3, the dotted and the long-dashed gray lines represent hi (s; r) 0 = 2; . . . ; 20 from the top to the bottom, respectively. By averaging hi (s; r) 0 r and hi (0; r) 0 r with j , we can compute h(s; r) 0 r and h(0; r) 0 r denoted by the black dotted and the long-dashed lines, respectively. Finally, H (s; e(0) ; r) 0 r is calculated by averaging h(s; r) 0 r and h(0; r) 0 r by (1 0 e(0) ) and e(0) , respectively. To satisfy the inequality in (22), H (s; e(0) ; r) 0 r must be less than zero, 8 r 2 (0; (s)]. Therefore, over the mixed M channel, the threshold mu (e(0) ) is the minimum of mu such that (0) for a given erasure probability e(0) < emax (; ), if s  mu , then H (s; e(0) ; r) 0 r < 0, 8 r 2 (0; (s)]. Over an AWGN channel,

(x) = 0:71875x

+0:28125x

(23)

r and hi (0; r) 0 r for i

1806

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 49, NO. 7, JULY 2003

Fig. 5. Comparison of E =N variations of regular LDPCCs with erasures at the coding rates of 4=5, 8=9, and 16=17 which are made of three ( = 1) nonzero terms in each column and 15 ( = 1), 27 ( = 1), and 51 ( = 1) nonzero terms in each row of the parity-check matrices, respectively; dots represent simulation results for a code length of 4096 and a BER of 10 , and lines are corresponding thresholds predicted with GA.

Fig. 7. E =N variations regarding erasure probabilities; filled and unfilled circles and triangles represent the results of an irregular LDPCC having code block length of 131702 bits, a coding rate of 1=2, and maximum iterations of 25 and 200, respectively. The edge degree distributions are

(x) = 0:23403x + 0:21242x + 0:14690x and (x) = 0:71875x + 0:28125x .

+ 0:10284x + 0:30381x

Fig. 6. BERs of an irregular LDPCC having a code block length of 131702 bits, a coding rate of 1=2, maximum iterations of 200, and edge degree distributions (x) = 0:23403x + 0:21242x + 0:14690x + 0:10284x + 0:30381x and (x) = 0:71875x + 0:28125x .

Fig. 8. E =N gaps from the capacity; filled and unfilled circles and triangles represent the results of an irregular LDPCC having code block length of 131702 bits, a coding rate of 1=2, and maximum iterations of 25 and 200, respectively. Edge degree distributions are (x) = 0:23403x + 0:21242x + 0:14690x + 0:10284x + 0:30381x and (x) = 0:71875x + 0:28125x .

Although regular LDPCCs may be a more practical choice, irregular LDPCCs give us more freedom in searching for capacity-approaching LDPCCs. In Fig. 6 we evaluate BER performances of an irregular LDPCC. The code is of length 131072 and is implemented with the degree distribution pair in (23). The analysis with GA fairly predicts the thresholds of the LDPCC over the mixed channel. In Fig. 7 we depict thresholds predicted by GA and DDE [9], which are compared with the required Eb =N0 s for a BER of 1004 shown in Fig. 6. To see the effect of the maximum number of iterations and erasure patterns, we did the same simulation with two different values for the maximum number of iterations (25 and 200) and types of erasure patterns (random and block erasures). The thresholds predicted by GA and DDE have a 0.2-dB discrepancy, which was observed for the AWGN channel in [10]. In Fig. 8, we depict a magnified view of the gaps between the thresholds and the capacity of the mixed channel CBPSK (1 0 e(0) ). The threshold from DDE is less than 0.2 dB from the capacity as long as the erasure probability is less than 0.2. Thus,

good (capacity-approaching) LDPCCs over AWGN channels are still good over the mixed channels up to a moderate erasure probability. However, at a high erasure probability there is a more distinct gap from the capacity. We can design an LDPCC optimized for a high erasure probability or for a small average gap from the capacity over several erasure probabilities. Although the optimized LDPCC has better thresholds at high erasure probabilities than those of an LDPCC for an AWGN channel, the LDPCC will suffer some loss in performance at low erasure probabilities. Figs. 7 and 8 also show the effects of the maximum iterations and the types of erasure patterns (random and block erasures). In both erasure cases, we erase a fixed number of symbols. In the random erasure case, the positions of erased symbols are chosen randomly. In the block erasure case, we randomly choose the beginning of an erasure block from which we erase consecutive symbols. The block and random erasure cases (unfilled circles and triangles) show

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 49, NO. 7, JULY 2003

1807

the performance variation decreases as the number of maximum iterations increases. Block and random erasure patterns have the same required Eb =N0 as the maximum number of iterations becomes large enough (200 in our simulations). Figs. 9 and 10 have a similar trend to Figs. 7 and 8. Thus, either a large number of iterations or a proper interleaver can remove the performance variations due to the erasure patterns. Yang and Ryan [14] designed LDPCCs based on our results and showed that better designed LDPCCs can also efficiently avoid the performance variation between random and block erasure patterns. VII. DESIGN OF AN LDPCC FOR THE MIXED CHANNEL In this section, we discuss code optimization with respect to left and right degree distributions to minimize the following metric: L min M = wj fE ((x); (x); ej ) 0 K (ej ; r )g argf(x); (x)g j =0
d
Fig. 9. E =N variations of LDPCCs for a BER of 10 with the erasure probabilities between 0 and 0:4 by a 0:1 step, a maximum number of iterations of 200, and the random and block erasure patterns.

given r = 1 0

j =2 d

j =j

(24)
j =j

Fig. 10. E =N variations of LDPCCs for a BER of 10 with different maximum iterations (25, 50, 100, 150, and 200), erasure probability of 0:1, and random and block erasure patterns.

j =2 where E (1) is the Eb =N0 threshold computed with GA given the degree distribution pair ((x); (x)), K (1) is Eb =N0 of the capacity for a coding rate of r , ej s are the erasure probabilities at which E (1) and K (1) are evaluated, and wj s are positive real weighting factors. For an LDPCC on an AWGN channel, the weight factors become w0 = 1 and wj = 0 for all j 6= 0, and e0 = 0. The optimization on the left degree distribution can be done through a linear optimization as described in Section V for a given right edge degree distribution that is jointly optimized through the differential evolution technique in [13]. It is also possible to design an LDPCC with the differential evolution technique on the left and right degree distributions with the recursive equation (20). We state the latter design procedure as follows. 1) [Initialization] Set 0 < min Eb =N0 < max Eb =N0 , 1  Nmax , 0 < min Pe  1, 0 < min Pe  1, and n = 0. 2) n = n + 1. 3) [Differential Evolution] Get a trial vector composed of a pair of distributions (n (x); n (x)) from the differential evolution whose coding rate is fixed. 4) [Compute Thresholds] Compute the threshold
E (n (x); n (x); ej )

almost the same performance with a large number of iterations (200 iterations). We see performance degradation (filled circles and triangles) in both cases due to a smaller number of iterations (25 iterations). However, the degradation is more severe in the block erasure case. In Fig. 8, the additional Eb =N0 s (the additional gaps from the capacity) in the random erasure case do not depend on the erasure probabilities. That is, at each erasure probability, the additional Eb =N0 is almost the same. In the block erasure case, more additional Eb =N0 is required for a higher erasure probability. Because block erasures can be efficiently converted to random erasures with a proper interleaver, the results in Sections IIIV are still useful in the block erasure case. We evaluate Eb =N0 performances of the LDPCC used in Figs. 7 and 8 with several different block lengths (131072, 16384, and 4096), erasure patterns (random and block erasures), and maximum iterations. In Fig. 9, we depict the required Eb =N0 s for a BER of 1004 with a fixed maximum number of iterations, 200. At each erasure probability, the block and random erasure patterns have the same performances. Fig. 10 shows the variations of required Eb =N0 s with different maximum iterations at an erasure probability of 0:1. The results show that

at each ej a) set 1

Kmax , 0 < E Emin = min Eb =N0 ; b) k = 0, and Pe

(0)

 1, E
0

max

= max Eb =N0 ,

and

= e0 =2 + (1

0 e )Q

mu =2

c) d) e) f)

E (n (x); n (x); ej ) = (Emax + Emin )=2; if (Emax Emin ) < E go to 5) k = k + 1; compute "(k) in (14) and store "(k) for computing "(k+1)

at the next iteration; (k) (k) g) compute mu in (19) and store mu for computing (k+1) mu at the next iteration; (k) h) compute Pe defined in (20) and (k) (k01) Pe = (Pe 0 Pe(k) )=Pe(k01) (k) with "(k) and mu ; (k) i) If Pe < min Pe then
Emax = E (n (x); n (x); ej )

and go to b);

1808

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 49, NO. 7, JULY 2003

j) If Kmax < k or Pe

(k )

< min Pe then

Emin = E (n (x); n (x); ej )

and go to b); k) Go to e). 5) [Compute Metric] Compute the metric in (24) with E (n (x); n (x); ej )s for 0  j  L and return the metric to the differential evolution routine that updates the best metric and the degree distribution pair. 6) If n < Nmax then go to 2). 7) [Termination] Get the best degree distribution, (3 (x); 3 (x)) from the differential evolution routine and stop. The design procedure is a combination of the differential evolution and the binary search algorithm. In the design procedure, we search for the best degree distribution pair (3 (x); 3 (x)) that minimizes the metric defined in (24). The equality in each step is an assignment operator instead of a Boolean operator. That is, the left variable will be the same as the right variable. In step 1) we define the initial values, where max Eb =N0 and min Eb =N0 are the upper and lower limits of the Eb =N0 threshold at each erasure probability (ej ), Nmax is the maximum number of iterations of this design procedure, and min Pe and min Pe are the minimum bit-error probability and the minimum bit-error probability decrease, respectively, which are explained in step 4). In step 3), the differential evolution routine returns a randomly generated edge degree distribution pair (n (x); n (x)). In step 4), the threshold at each erasure probability (ej ) is evaluated with the binary search algorithm. In the binary search, the threshold of an LDPCC is presumed to be between Emax and Emin . In a), Emax and Emin are set to max Eb =N0 and min Eb =N0 , respectively. E (n (x); n (x); ej ) will be always the middle of Emax and Emin . In d), if the difference between Emax and Emin is less than E  1, which means the threshold is within E=2 from the true threshold, the binary search terminates. (k) From e) to h), the bit-error probability at the k th iteration, Pe , and (k) the decrease of the bit-error probability Pe are evaluated. In i), if the bit-error probability is less than min Pe , that is regarded as zero probability, the threshold is less than or equal to the current threshold. Thus, Emax = E (n (x); n (x); ej ) and a search for the thresholds between Emax and Emin is performed again. In j), if Kmax < k (too many iterations) or Pe < min Pe , which means the iterative bit-error probability falls into a fixed point, the threshold must be larger than the current threshold E (n (x); n (x); ej ). In step 5), with the thresholds found through the binary search, the metric defined in (24) is passed to the differential evolution routine that makes a new random vector in step 3). The steps from 3) to 5) will be iterated Nmax times, and we get the best results (3 (x); 3 (x)). One can design a code that has been optimized for a specific erasure probability. We are more interested in a single code that is optimized across a range of erasure probabilities. Again, this is common in recording when the code is designed to handle a maximum number of erasures. We consider an LDPCC which has a coding rate of 1=2 and is optimized with uniform weighting factors. The metric in (24) is evaluated at every erasure probability between 0 and 0:35 with a 0:05 step. In Fig. 11, we compare between a well-designed LDPCC with GA for an AWGN channel in [10] and an LDPCC designed for the mixed channel, where the thresholds are computed with GA and DDE. The edge degree distribution pairs of the LDPCC in [10] and the designed one are listed in Table I. The designed LDPCC shows better thresholds if the erasure probability is greater than 0:1 in both the GA and DDE cases. The thresholds from DDE with erasure probability less than 0:1 are poorer than the LDPCC designed for the AWGN channel, which can be either mitigated or avoided with proper weighting factors.

Fig. 11. E =N gap from the capacity; LDPCC designed for AWGN channel having a degree distribution of (x) = 0:30780x + 0:27287x + 0:41933x and (x) = 0:4x + 0:6x and LDPCC designed over the mixed channel having degree distribution (x) = 0:33175x + 0:24120x + 0:42705x and (x) = 0:45290x + 0:54710x . TABLE I EDGE DEGREE DISTRIBUTION PAIRS OF AN LDPCC DESIGNED FOR AN AWGN CHANNEL ((x); (x)) AND ONE DESIGNED FOR THE MIXED CHANNEL ( (x);  (x))

VIII. CONCLUSION The Gaussian approximation is extended to include the mixture of white Gaussian noise with erasures, and the analytic results not only provide a way to predict asymptotic performance of LDPCCs but also help us design good LDPCCs over the mixed channel. The analysis tells us that the variation of the erasure probability during iterations does not depend on the received signal power but only on the structure of LDPCCs, which can be defined with edge degree distributions. Thus, as long as the erasure probability is less than a threshold of an LDPCC over BECs, we can assume that the erasure probability monotonically decreases to zero, which gives us a simple recursive equation describing message densities, called a steady-state equation. This simplified equation leads us to a design rule of good LDPCCs over the mixed channel and a graphical interpretation of the convergence of LDPCCs over the mixed channel. To show the validity of the analytic results and the channel model, we simulated an LDPCC having a code block length of 131072 and a coding rate of 1=2. We also computed the Eb =N0 thresholds of the LDPCC using the analytic results. The comparison between the simulation results and the thresholds showed that the prediction from the theoretical results was accurate enough. We have found that well-designed LDPCCs for AWGN channels are still good over the mixed channel. In the simulation, we had less than 0.1-dB Eb =N0 loss up to an erasure probability of 0:2. We also investigated the performance variations due to types of erasure patterns with different block lengths and different values of maximum iterations. The simulation results allowed us to conclude that to mitigate the performance variations, we need a large number of iterations or a proper interleaver before the decoder.

IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 49, NO. 7, JULY 2003

1809

Although good LDPCCs are also good over the mixed channel, we can, in the average sense, improve LDPCCs over several erasure probabilities. We optimized an LDPCC for the mixed channel with a design rule from the analytic results to achieve a better average metric. The designed code shows better thresholds at erasure probabilities larger than 0:1 by 0.1 dB in GA and DDE. From a practical point of view, we believe our work would be helpful to predict thresholds of LDPCCs over the mixed channel, especially for magnetic and optical storage applications. From a theoretical point of view, it also paves the way for a new study of LDPCCs over the mixed channel. ACKNOWLEDGMENT The authors would like to acknowledge M. Yang and W. Ryan of the University of Arizona for providing a preprint of [14]. REFERENCES
[1] R. G. Gallager, Low-density parity check does, IRE Trans. Inform. Theory, vol. IT-8, pp. 2122, Jan. 1962. [2] , Low-Density Parity Check Codes. Cambridge, MA: MIT Press, 1963. [3] C. Berrou, A. Glavieux, and P. Thitimajshima, Near Shannon limit error-correcting coding and decoding, in Proc. IEEE Int. Conf. Communications, Geneva, Switzerland, May 1993, pp. 10641070. [4] D. J. C. MacKay and R. M. Neal, Near Shannon limit performance of low-density parity-check codes, Electron. Lett., vol. 32, pp. 16451646, Aug. 1996. [5] D. J. C. MacKay, Good error-correcting codes based on very sparse matrices, IEEE Trans. Inform. Theory, vol. 45, pp. 399431, Mar. 1999. [6] M. Sipser and D. Spielman, Expander codes, IEEE Trans. Inform. Theory, vol. 42, pp. 17101722, Nov. 1996. [7] M. Luby, M. Mitzenmacher, A. Shokrollahi, D. Spielman, and V. Stemann, Practical loss-resilient codes, in Proc. 29th Annu. ACM Symp. Theory of Computing, 1997, pp. 150159. [8] T. Richardson and R. Urbanke, The capacity of low-density paritycheck codes under message-passing decoding, IEEE Trans. Inform. Theory, vol. 47, pp. 599618, Feb. 2001. [9] S.-Y. Chung, On the construction of some capacity-approaching coding schemes, Ph.D. dissertation, MIT, Cambridge, MA, 2000. [10] S.-Y. Chung, T. Richardson, and R. Urbanke, Analysis of sumproduct decoding of low-density parity-check codes using a Gaussian approximation, IEEE Trans. Inform. Theory, vol. 47, pp. 657670, Feb. 2001. [11] T. Richardson, A. Shokrollahi, and R. Urbanke, Design of capacity-approaching irregular low-density parity-check codes, IEEE Trans. Inform. Theory, vol. 47, pp. 619637, Feb. 2001. [12] M. Luby, M. Mitzenmacher, A. Shokrollahi, and D. Spielman, Analysis of low density codes and improved designs using irregular graphs, in Proc. 30th Annu. ACM Symp. Theory of Computing, 1998, pp. 249258. [13] K. Price and R. Storn, Differential evolution-A simple and efficient heuristic for global optimization over continuous spaces, J. Global Optimiz., vol. 11, pp. 341359, 1997. [14] M. Yang and W. Ryan, Design of LDPC codes for the burst-erasure channel with AWGN,, paper, to be published.

New Covering Codes From an ADS-Like Construction

Markku K. Kaikkonen and Petri Rosendahl
AbstractA covering code construction is presented. Using this construction it is shown that [52 39] = 3, [36 21] = 4, [58 32] = 7, (32 2) 62 2 , and (62 5) 31 2 , where [ ] is the minimum covering radius among all binary [ ] codes and ( ) is the minimum cardinality of a binary code of length and covering radius . Four new linear codes found by computer search are also given. These include a [23 9]5 code, a [32 8]10 code, a [51 41]2 code, and a [45 20]8 code. Index TermsAmalgamated direct sum (ADS), covering code, normality.

I. INTRODUCTION Let = f0; 1g be the binary field. An (n; K ) code is a K -element subset of the space n , and an [n; k] linear code is a k -dimensional subspace of n . The covering radius R(C ) of a code C is by definition the maximum (Hamming) distance from vectors in n to the code C , i.e.,
R(C )

= max min d(x; c):

x2

c2C

An (n; K )R (or an [n; k]R) code is a code with covering radius at most R. The covering radius of a linear code can be calculated by means of its parity-check matrix H . Namely, the covering radius of an [n; k] code is the smallest integer R such that every binary column vector of length n 0 k is a sum of at most R columns of H (for a proof, see [2, Ch. 2, p. 21]). Let A be a k 2 n matrix over . We shall always assume that A has full rank. A set S  k is said to R-cover k using A if for every k there is a vector y 2 n of weight at most R and a vector x 2 s 2 S such that x = Ay + s. The following theorem is from [1] and [5] (see also [2, Ch. 3, p. 70]); it is known as the matrix construction. Theorem 1: If the set SR-covers k using A, then the code
C

has jS j2n0k codewords and covering radius at most R. The code in Theorem 1 is a union of some cosets of a linear code. In particular, if S = f0g, or more generally, if S is linear, then the code is linear. (i) Let C be a binary (n; K ) code with covering radius R. Let C0 (i) (resp., C1 ) denote the set of codewords in which the ith coordinate is 0 (resp., 1). The code C is said to be normal if there is a coordinate i such that for all x 2 n
d x; C0

f 2
c

n : Ac 2 S g

(i)

x; C1

(i)

2 +1
R

In this case, the coordinate i is said to be acceptable. The columns of a k 2 n matrix H are denoted by h1 ; h2 ; . . . ; hn . The following lemma is an immediate generalization of a theorem in [4].
Manuscript received May 2, 2001; revised February 10, 2003. The work of P. Rosendahl was supported by the Academy of Finland under Grant 44002. M. K. Kaikkonen is at Halkosuontie 65 C 5, 00660 Helsinki, Finland. P. Rosendahl is with the Department of Mathematics and the Turku Centre for Computer Science TUCS, University of Turku, 20014 Turku, Finland. Communicated by S. Litsyn, Associate Editor for Coding Theory. Digital Object Identifier 10.1109/TIT.2003.813508 0018-9448/03$17.00 2003 IEEE