Expander code: Difference between revisions
dab bipartite, degree; link regular graph |
|||
(20 intermediate revisions by 17 users not shown) | |||
Line 2: | Line 2: | ||
{{infobox code |
{{infobox code |
||
| name = Expander codes |
| name = Expander codes |
||
| image = [[File: |
| image = [[File:Tanner graph example.PNG|300px]] |
||
| image_caption = bipartite expander graph |
| image_caption = bipartite expander graph |
||
| namesake = |
| namesake = |
||
Line 15: | Line 15: | ||
In [[coding theory]], '''expander codes''' form a class of [[Error detection and correction|error-correcting codes]] that are constructed from [[Bipartite graph|bipartite]] [[expander graph]]s. |
In [[coding theory]], '''expander codes''' form a class of [[Error detection and correction|error-correcting codes]] that are constructed from [[Bipartite graph|bipartite]] [[expander graph]]s. |
||
Along with [[Justesen code]]s, expander codes are of particular interest since they have a constant positive [[ |
Along with [[Justesen code]]s, expander codes are of particular interest since they have a constant positive [[Block code#The rate R|rate]], a constant positive relative [[Block code#The distance d|distance]], and a constant [[Block code#The alphabet .CE.A3|alphabet size]]. |
||
In fact, the alphabet contains only two elements, so expander codes belong to the class of [[ |
In fact, the alphabet contains only two elements, so expander codes belong to the class of [[binary code]]s. |
||
Furthermore, expander codes can be both encoded and decoded in time proportional to the block length of the code. |
Furthermore, expander codes can be both encoded and decoded in time proportional to the block length of the code. |
||
Expander codes are the only known asymptotically good codes which can be both encoded and decoded from a constant fraction of errors in polynomial time. |
|||
==Expander codes== |
==Expander codes== |
||
In [[coding theory]], an expander code is a <math>[n,n-m]_2\,</math> [[linear block code]] whose parity check matrix is the adjacency matrix of a bipartite [[expander graph]]. These codes have good relative [[ |
In [[coding theory]], an expander code is a <math>[n,n-m]_2\,</math> [[linear block code]] whose parity check matrix is the adjacency matrix of a bipartite [[expander graph]]. These codes have good relative [[Block code#The distance d|distance]] <math>2(1-\varepsilon)\gamma\,</math>, where <math>\varepsilon\,</math> and <math>\gamma\,</math> are properties of the expander graph as defined later, [[Block code#The rate R|rate]] <math>\left(1-\tfrac{m}{n}\right)\,</math>, and decodability (algorithms of running time <math>O(n)\,</math> exist). |
||
==Definition== |
==Definition== |
||
Let <math>B</math> be a <math>(c,d)</math>-[[biregular graph]] between a set of <math>n</math> nodes <math>\{v_1,\cdots,v_n\}</math>, called ''variables'', and a set of <math>cn/d</math> nodes <math>\{C_1,\cdots,C_{cn/d}\}</math>, called ''constraints''. |
|||
Consider a [[bipartite graph]] <math>G(L,R,E)\,</math>, where <math>L\,</math> and <math>R\,</math> are the vertex sets and <math>E\,</math> is the set of edges connecting vertices in <math>L\,</math> to vertices of <math>R\,</math>. Suppose every vertex in <math>L\,</math> has [[degree (graph theory)|degree]] <math>d\,</math> (the graph is <math>d\,</math>-[[Regular graph|regular]]), <math>|L|=n\,</math>, and <math>|R|=m\,</math>, <math>m < n\,</math>. Then <math>G\,</math> is a <math>(N, M, d, \gamma, \alpha)\,</math> expander graph if every small enough subset <math>S \subset L\,</math>, <math>|S| \leq \gamma n\,</math> has the property that <math>S\,</math> has at least <math>d\alpha|S|\,</math> distinct neighbors in <math>R\,</math>. Note that this holds trivially for <math>\gamma \leq \tfrac{1}{n}\,</math>. When <math>\tfrac{1}{n} < \gamma \leq 1\,</math> and <math>\alpha = 1 - \varepsilon\,</math> for a constant <math>\varepsilon\,</math>, we say that <math>G\,</math> is a lossless expander. |
|||
Let <math>b(i,j)</math> be a function designed so that, for each constraint <math>C_i</math>, the variables neighboring <math>C_i</math> are <math>v_{b(i,1)},\cdots,v_{b(i,d)}</math>. |
|||
Since <math>G\,</math> is a bipartite graph, we may consider its <math>n \times m\,</math> adjacency matrix. Then the linear code <math>C\,</math> generated by viewing the transpose of this matrix as a parity check matrix is an expander code. |
|||
Let <math>\mathcal{S}</math> be an error-correcting code of block length <math>d</math>. The '''''expander code''''' <math>\mathcal{C}(B,\mathcal{S})</math> is the code of block length <math>n</math> whose codewords are the words <math>(x_1,\cdots,x_n)</math> such that, for <math>1\leq i\leq cn/d</math>, <math>(x_{b(i,1)},\cdots,x_{b(i,d)})</math> is a codeword of <math>\mathcal{S}</math>.<ref name="definition">{{cite journal|doi=10.1109/18.556667|title=Expander codes |year=1996 |last1=Sipser |first1=M. |last2=Spielman |first2=D.A. |journal=IEEE Transactions on Information Theory |volume=42 |issue=6 |pages=1710–1722 }}</ref> |
|||
It has been shown that nontrivial lossless expander graphs exist. Moreover, we can explicitly construct them.<ref name="lossless">[http://portal.acm.org/citation.cfm?id=510003 Randomness conductors and constant-degree lossless expanders]</ref> |
|||
It has been shown that nontrivial lossless expander graphs exist. Moreover, we can explicitly construct them.<ref name="lossless">{{cite book |first1=M. |last1=Capalbo |first2=O. |last2=Reingold |first3=S. |last3=Vadhan |first4=A. |last4=Wigderson |chapter=Randomness conductors and constant-degree lossless expanders |chapter-url=http://dl.acm.org/citation.cfm?id=510003 |title=STOC '02 Proceedings of the thirty-fourth annual ACM symposium on Theory of computing |publisher=ACM |year=2002 |isbn=978-1-58113-495-7 |pages=659–668 |doi=10.1145/509907.510003|s2cid=1918841 }}</ref> |
|||
==Rate== |
==Rate== |
||
The rate of <math>C\,</math> is its dimension divided by its block length. In this case, the parity check matrix has size <math>m \times n\,</math>, and hence <math>C\,</math> has |
The rate of <math>C\,</math> is its dimension divided by its block length. In this case, the parity check matrix has size <math>m \times n\,</math>, and hence <math>C\,</math> has rate at least <math>(n-m)/n = 1 - m/n\,</math>. |
||
==Distance== |
==Distance== |
||
Suppose <math>\varepsilon < \tfrac{1}{2}\,</math>. Then the distance of a <math>(n, m, d, \gamma, 1-\varepsilon)\,</math> expander code <math>C\,</math> is at least <math>2(1-\varepsilon)\gamma n\,</math>. |
Suppose <math>\varepsilon < \tfrac{1}{2}\,</math>. Then the distance of a <math>(n, m, d, \gamma, 1-\varepsilon)\,</math> expander code <math>C\,</math> is at least <math>2(1-\varepsilon)\gamma n\,</math>. |
||
===Proof=== |
===Proof=== |
||
Note that we can consider every codeword <math>c\,</math> in <math>C\,</math> as a subset of vertices <math>S \subset L\,</math>, by saying that vertex <math>v_i \in S\,</math> if and only if the <math>i\,</math>th index of the codeword is a 1. Then <math>c\,</math> is a codeword iff every vertex <math>v \in R\,</math> is adjacent to an even number of vertices in <math>S\,</math>. (In order to be a codeword, <math>cP = 0\,</math>, where <math>P\,</math> is the parity check matrix. Then, each vertex in <math>R\,</math> corresponds to each column of <math>P\,</math>. Matrix multiplication over <math>\text{GF}(2) = \{0,1\}\,</math> then gives the desired result.) So, if a vertex <math>v \in R\,</math> is adjacent to a single vertex in <math>S\,</math>, we know immediately that <math>c\,</math> is not a codeword. Let <math>N(S)\,</math> denote the neighbors in <math>R\,</math> of <math>S\,</math>, and <math>U(S)\,</math> denote those neighbors of <math>S\,</math> which are unique, i.e., adjacent to a single vertex of <math>S\,</math>. |
Note that we can consider every codeword <math>c\,</math> in <math>C\,</math> as a subset of vertices <math>S \subset L\,</math>, by saying that vertex <math>v_i \in S\,</math> if and only if the <math>i\,</math>th index of the codeword is a 1. Then <math>c\,</math> is a codeword iff every vertex <math>v \in R\,</math> is adjacent to an even number of vertices in <math>S\,</math>. (In order to be a codeword, <math>cP = 0\,</math>, where <math>P\,</math> is the parity check matrix. Then, each vertex in <math>R\,</math> corresponds to each column of <math>P\,</math>. Matrix multiplication over <math>\text{GF}(2) = \{0,1\}\,</math> then gives the desired result.) So, if a vertex <math>v \in R\,</math> is adjacent to a single vertex in <math>S\,</math>, we know immediately that <math>c\,</math> is not a codeword. Let <math>N(S)\,</math> denote the neighbors in <math>R\,</math> of <math>S\,</math>, and <math>U(S)\,</math> denote those neighbors of <math>S\,</math> which are unique, i.e., adjacent to a single vertex of <math>S\,</math>. |
||
Line 57: | Line 59: | ||
==Encoding== |
==Encoding== |
||
The encoding time for an expander code is upper bounded by that of a general linear code - <math>O(n^2)\,</math> by matrix multiplication. A result due to Spielman shows that encoding is possible in <math>O(n)\,</math> time.<ref name="spielman">D. Spielman |
The encoding time for an expander code is upper bounded by that of a general linear code - <math>O(n^2)\,</math> by matrix multiplication. A result due to Spielman shows that encoding is possible in <math>O(n)\,</math> time.<ref name="spielman">{{cite journal |first=D. |last=Spielman |title=Linear-time encodable and decodable error-correcting codes |journal=IEEE Transactions on Information Theory |volume=42 |issue=6 |pages=1723–31 |year=1996 |doi=10.1109/18.556668 |citeseerx=10.1.1.47.2736 }}</ref> |
||
==Decoding== |
==Decoding== |
||
Decoding of expander codes is possible in <math>O(n)\,</math> time when <math>\varepsilon < \tfrac{1}{4}\,</math> using the following algorithm. |
Decoding of expander codes is possible in <math>O(n)\,</math> time when <math>\varepsilon < \tfrac{1}{4}\,</math> using the following algorithm. |
||
Let <math>v_i\,</math> be the vertex of <math>L\,</math> that corresponds to the <math>i\,</math>th index in the codewords of <math>C\,</math>. Let <math>y \in \{0,1\}^n\,</math> be a received word, and <math>V(y) = \{v_i |
Let <math>v_i\,</math> be the vertex of <math>L\,</math> that corresponds to the <math>i\,</math>th index in the codewords of <math>C\,</math>. Let <math>y \in \{0,1\}^n\,</math> be a received word, and <math>V(y) = \{v_i \mid \text{the } i^{\text{th}} \text{ position of } y \text{ is a } 1\}\,</math>. Let <math>e(i)\,</math> be <math>|\{v \in R \mid v_i \in N(v) \text{ and } N(v) \cap V(y) \text{ is even}\}|\,</math>, and <math>o(i)\,</math> be <math>|\{v \in R \mid v_i \in N(v) \text{ and } N(v) \cap V(y) \text{ is odd}\}|\,</math>. Then consider the greedy algorithm: |
||
---- |
---- |
||
'''Input:''' received |
'''Input:''' received word <math>y\,</math>. |
||
⚫ | |||
<code> |
|||
⚫ | |||
while there is a v in R adjacent to an odd number of vertices in V(y') |
while there is a v in R adjacent to an odd number of vertices in V(y') |
||
if there is an i such that o(i) > e(i) |
if there is an i such that o(i) > e(i) |
||
flip entry i in y' |
flip entry i in y' |
||
else |
else |
||
fail |
fail |
||
'''Output:''' fail, or modified codeword <math>y'\,</math>. |
'''Output:''' fail, or modified codeword <math>y'\,</math>. |
||
---- |
---- |
||
Line 114: | Line 115: | ||
==Notes== |
==Notes== |
||
This article is based on Dr. Venkatesan Guruswami's course notes.<ref |
This article is based on Dr. Venkatesan Guruswami's course notes.<ref>{{cite web |first=V. |last=Guruswami |title=Lecture 13: Expander Codes |date=15 November 2006 |work=CSE 533: Error-Correcting |publisher=University of Washington |url=http://www.cs.washington.edu/education/courses/cse533/06au/lecnotes/lecture13.pdf }}<br/> |
||
{{cite web |first=V. |last=Guruswami |title=Notes 8: Expander Codes and their decoding |date=March 2010 |work=Introduction to Coding Theory |publisher=Carnegie Mellon University |url=https://www.cs.cmu.edu/~venkatg/teaching/codingtheory/notes/notes8.pdf }}<br/> |
|||
{{cite journal |first=V. |last=Guruswami |title=Guest column: error-correcting codes and expander graphs |journal=ACM SIGACT News |volume=35 |issue=3 |pages=25–41 |date=September 2004 |doi=10.1145/1027914.1027924 |s2cid=17550280 |url=http://dl.acm.org/citation.cfm?id=1027924|url-access=subscription }}</ref> |
|||
==References== |
==References== |
Latest revision as of 06:40, 22 July 2024
This article may require cleanup to meet Wikipedia's quality standards. The specific problem is: Missing definitions, and grammar requires heavy copy-editing. Blindly pointing to the references should not be the point of the article. Missing scholarly exposition. (July 2012) |
Expander codes | |
---|---|
Classification | |
Type | Linear block code |
Block length | |
Message length | |
Rate | |
Distance | |
Alphabet size | |
Notation | -code |
In coding theory, expander codes form a class of error-correcting codes that are constructed from bipartite expander graphs. Along with Justesen codes, expander codes are of particular interest since they have a constant positive rate, a constant positive relative distance, and a constant alphabet size. In fact, the alphabet contains only two elements, so expander codes belong to the class of binary codes. Furthermore, expander codes can be both encoded and decoded in time proportional to the block length of the code.
Expander codes
[edit]In coding theory, an expander code is a linear block code whose parity check matrix is the adjacency matrix of a bipartite expander graph. These codes have good relative distance , where and are properties of the expander graph as defined later, rate , and decodability (algorithms of running time exist).
Definition
[edit]Let be a -biregular graph between a set of nodes , called variables, and a set of nodes , called constraints.
Let be a function designed so that, for each constraint , the variables neighboring are .
Let be an error-correcting code of block length . The expander code is the code of block length whose codewords are the words such that, for , is a codeword of .[1]
It has been shown that nontrivial lossless expander graphs exist. Moreover, we can explicitly construct them.[2]
Rate
[edit]The rate of is its dimension divided by its block length. In this case, the parity check matrix has size , and hence has rate at least .
Distance
[edit]Suppose . Then the distance of a expander code is at least .
Proof
[edit]Note that we can consider every codeword in as a subset of vertices , by saying that vertex if and only if the th index of the codeword is a 1. Then is a codeword iff every vertex is adjacent to an even number of vertices in . (In order to be a codeword, , where is the parity check matrix. Then, each vertex in corresponds to each column of . Matrix multiplication over then gives the desired result.) So, if a vertex is adjacent to a single vertex in , we know immediately that is not a codeword. Let denote the neighbors in of , and denote those neighbors of which are unique, i.e., adjacent to a single vertex of .
Lemma 1
[edit]For every of size , .
Proof
[edit]Trivially, , since implies . follows since the degree of every vertex in is . By the expansion property of the graph, there must be a set of edges which go to distinct vertices. The remaining edges make at most neighbors not unique, so .
Corollary
[edit]Every sufficiently small has a unique neighbor. This follows since .
Lemma 2
[edit]Every subset with has a unique neighbor.
Proof
[edit]Lemma 1 proves the case , so suppose . Let such that . By Lemma 1, we know that . Then a vertex is in iff , and we know that , so by the first part of Lemma 1, we know . Since , , and hence is not empty.
Corollary
[edit]Note that if a has at least 1 unique neighbor, i.e. , then the corresponding word corresponding to cannot be a codeword, as it will not multiply to the all zeros vector by the parity check matrix. By the previous argument, . Since is linear, we conclude that has distance at least .
Encoding
[edit]The encoding time for an expander code is upper bounded by that of a general linear code - by matrix multiplication. A result due to Spielman shows that encoding is possible in time.[3]
Decoding
[edit]Decoding of expander codes is possible in time when using the following algorithm.
Let be the vertex of that corresponds to the th index in the codewords of . Let be a received word, and . Let be , and be . Then consider the greedy algorithm:
Input: received word .
initialize y' to y while there is a v in R adjacent to an odd number of vertices in V(y') if there is an i such that o(i) > e(i) flip entry i in y' else fail
Output: fail, or modified codeword .
Proof
[edit]We show first the correctness of the algorithm, and then examine its running time.
Correctness
[edit]We must show that the algorithm terminates with the correct codeword when the received codeword is within half the code's distance of the original codeword. Let the set of corrupt variables be , , and the set of unsatisfied (adjacent to an odd number of vertices) vertices in be . The following lemma will prove useful.
Lemma 3
[edit]If , then there is a with .
Proof
[edit]By Lemma 1, we know that . So an average vertex has at least unique neighbors (recall unique neighbors are unsatisfied and hence contribute to ), since , and thus there is a vertex with .
So, if we have not yet reached a codeword, then there will always be some vertex to flip. Next, we show that the number of errors can never increase beyond .
Lemma 4
[edit]If we start with , then we never reach at any point in the algorithm.
Proof
[edit]When we flip a vertex , and are interchanged, and since we had , this means the number of unsatisfied vertices on the right decreases by at least one after each flip. Since , the initial number of unsatisfied vertices is at most , by the graph's -regularity. If we reached a string with errors, then by Lemma 1, there would be at least unique neighbors, which means there would be at least unsatisfied vertices, a contradiction.
Lemmas 3 and 4 show us that if we start with (half the distance of ), then we will always find a vertex to flip. Each flip reduces the number of unsatisfied vertices in by at least 1, and hence the algorithm terminates in at most steps, and it terminates at some codeword, by Lemma 3. (Were it not at a codeword, there would be some vertex to flip). Lemma 4 shows us that we can never be farther than away from the correct codeword. Since the code has distance (since ), the codeword it terminates on must be the correct codeword, since the number of bit flips is less than half the distance (so we couldn't have traveled far enough to reach any other codeword).
Complexity
[edit]We now show that the algorithm can achieve linear time decoding. Let be constant, and be the maximum degree of any vertex in . Note that is also constant for known constructions.
- Pre-processing: It takes time to compute whether each vertex in has an odd or even number of neighbors.
- Pre-processing 2: We take time to compute a list of vertices in which have .
- Each Iteration: We simply remove the first list element. To update the list of odd / even vertices in , we need only update entries, inserting / removing as necessary. We then update entries in the list of vertices in with more odd than even neighbors, inserting / removing as necessary. Thus each iteration takes time.
- As argued above, the total number of iterations is at most .
This gives a total runtime of time, where and are constants.
See also
[edit]- Expander graph
- Low-density parity-check code
- Linear time encoding and decoding of error-correcting codes
- ABNNR and AEL codes
Notes
[edit]This article is based on Dr. Venkatesan Guruswami's course notes.[4]
References
[edit]- ^ Sipser, M.; Spielman, D.A. (1996). "Expander codes". IEEE Transactions on Information Theory. 42 (6): 1710–1722. doi:10.1109/18.556667.
- ^ Capalbo, M.; Reingold, O.; Vadhan, S.; Wigderson, A. (2002). "Randomness conductors and constant-degree lossless expanders". STOC '02 Proceedings of the thirty-fourth annual ACM symposium on Theory of computing. ACM. pp. 659–668. doi:10.1145/509907.510003. ISBN 978-1-58113-495-7. S2CID 1918841.
- ^ Spielman, D. (1996). "Linear-time encodable and decodable error-correcting codes". IEEE Transactions on Information Theory. 42 (6): 1723–31. CiteSeerX 10.1.1.47.2736. doi:10.1109/18.556668.
- ^ Guruswami, V. (15 November 2006). "Lecture 13: Expander Codes" (PDF). CSE 533: Error-Correcting. University of Washington.
Guruswami, V. (March 2010). "Notes 8: Expander Codes and their decoding" (PDF). Introduction to Coding Theory. Carnegie Mellon University.
Guruswami, V. (September 2004). "Guest column: error-correcting codes and expander graphs". ACM SIGACT News. 35 (3): 25–41. doi:10.1145/1027914.1027924. S2CID 17550280.