Method and System for Decoding GraphBased Codes Using MessagePassing with DifferenceMap Dynamics
A code to be decoded by messagepassing is represented by a factor graph. The factor graph includes variable nodes indexed by i and constraint nodes indexed by a connected by edges for transferring messages mi→a outgoing from the variable nodes to the constraint nodes and messages ma→i incoming from the constraint nodes to the variable nodes. The messages mi→a are initialized based on beliefs bi of a received codeword. The messages ma→i are generated by overshooting the messages mi→a at the constraint nodes. The beliefs bi are updated at the variable nodes using the messages ma→i. The codeword is outputted if found, otherwise, the messages mi→a are updated using a correction for the overshooting.
This is a Divisional Application of U.S. patent application Ser. No. 12/541,810, “Method and System for Decoding GraphBased Codes Using MessagePassing with DifferenceMap Dynamics,” filed by Yedidia et al. on Aug. 14, 2009, incorporated herein by reference.
FIELD OF THE INVENTIONThis invention relates generally to decoders, and more particularly to decoding graphbased codes, including lowdensity paritycheck (LDPC) codes, using messagepassing decoders.
BACKGROUND OF THE INVENTION Problems with Standard BP Decoders of LDPC CodesProperly designed lowdensity paritycheck (LDPC) codes, when decoded using belief propagation (BP) decoders, can achieve near Shannon limit performance in the socalled “waterfall” regime where the signaltonoise ratio (SNR) is near the code threshold. Numerous BP decoders are known in the prior art. The U.S. Patent Database reveals over 500 hundred related patents and applications.
Prior art BP decoders are instances of messagepassing methods that operate on a graphical representation known as a “factor graph,” see F. R. Kschischang, B. J. Frey, and H.A. Loeliger, “Factor Graphs and the SumProduct Algorithm,” IEEE Transactions on Information Theory, vol. 47, pp. 498519, 2001. BP decoders pass “messages” between “variable nodes” and “constraint nodes” in the factor graph, and update probabilities (“beliefs”) as to what a decoded codeword should be. Herein, “belief” is a term of art that is universally used to describe BP decoders. The major characteristics that distinguish different BP decoders are how messages and beliefs are iteratively updated.
Two versions of BP decoders are considered standard: the “minsum” (or equivalent “maxproduct”) BP decoder and the “sumproduct” BP decoder. The minsum decoder is optimal in the sense that it always returns the most likely transmitted codeword, when the factor graph representing the code is a tree. Unfortunately, the factor graphs for good codes invariably contain cycles, and in the presence of cycles, the minsum BP decoder becomes suboptimal.
Suboptimal sumproduct BP decoders nevertheless achieve near Shannon limit performance for good graphbased codes in the waterfall regime, but unfortunately, the same is not true in the highSNR regime. In the highSNR regime graphbased codes paired with sumproduct or minsum BP decoders often suffer from an “error floor” phenomena, whereby the decoder error rate improves only slowly as the SNR increases. The error floor is a significant problem for applications that require extremely high reliability for high bit densities and data rates. It limits the applicability of current generation graphbased codes in such applications, which include magnetooptic data storage, and fiberoptic communications.
Error floors in graphbased codes using BP decoders are widely believed to result from structures in the code that go under the names “nearcodewords,” “pseudocodewords,” “instantons,” “absorbing sets,” and “trapping sets,” see T. Richardson, “Error Floors of LDPC Codes,” Proceedings of the 41^{st }Allerton Conference on Communications, Control, and Computing, 2003. Error patterns resulting from channel transmission that strongly overlap these patterns are uncorrectable by the decoder.
Approaches to improve the error floor, while maintaining good waterfall behavior, can be categorized in two groups. One group constructs codes that have fewer trapping sets, and therefore improved error floors, when decoded using standard BP decoders. The number of the trapping sets, and therefore the error floor is improved by, e.g., constructing codes with fewer short cycles in the factor graph. One can also consider special classes of LDPC codes, such as EGLDPC codes, or generalized LDPC codes that have fewer trapping sets. However, for many applications, specified codes, or codes of specified blocklengths and rates, may be required, and this approach may not be possible to implement because there may be no way to construct codes meeting the required specifications that also have good error floor behavior when decoded using standard BP decoders.
The other group of approaches to improving the error floor behavior, related to the present invention, improves the decoder. This approach is logical because random regular LDPC codes have (with high probability) excellent distance properties, and therefore, would not have error floors if the decoder used optimal maximumlikelihood (ML) decoding. (Note however that optimal decoders are normally too complex to be practical.)
For instance, if the trapping sets of a particular code are enumerated, a “bimode syndromeerasure decoder” can improve the error floor behavior, see Y. Han and W. E. Ryan, “Lowfloor decoders for LDPC Codes,” Proceedings of the 45^{th }Annual Allerton Conference on Communications, Control, and Computing 2007, pp. 473479. However, for many codes the enumeration of the trapping sets is itself an extremely challenging task.
Another improved decoder is the mixedinteger linear programming (MILP) decoder, which requires no information about trapping sets and approaches ML performance, but has a large decoding complexity. To deal with the complexity of the MILP decoder, a multistage decoder can be used, see Y. Wang, J. S. Yedidia, and S. C. Draper, “Multistage Decoding Via MixedInteger Adaptive Linear Programming,” Proceedings of the IEEE International Symposium on Information Theory, 2009, pp. 21512155. The multistage decoder combines fast but poorperforming decoders with more powerful, but slower MILP decoders so that one can obtain a decoder that performs as well as the MILP decoder and with a high average throughput. Nevertheless, the multistage decoder has considerable practical difficulties, in that it requires multiple decoders, and the worstcase throughput is as slow as the MILP decoder.
Therefore, it is desired to provide practical BP decoders for LDPC codes that have an improved error floor when compared with standard BP decoders, but with comparable complexity, and no significant disadvantages.
The Divide and Concur Constraint Optimization Method
Our invention, described below, is motivated in part by a method described by Gravel and Elser, “Divide and Concur: a General Approach to Constraint Satisfaction,” Phys. Rev. E 78, 036706, 2008, hereinafter D&C.
The motivation of Gravel and Elser was to solve complex computational problems, such as the problem of reconstructing the shapes of macromolecules in xray crystallography. Other applications investigated using the D&C method relate to protein folding, finding ground states of spin glasses, satisfiability testing, protein structure prediction, Ramsey numbers, solving diophantine equations, and even solving Sudoku puzzles. Many of those applications are NPcomplete problems. Another of their motivations was to find improved solutions to some old and wellstudied optimization problems, such as sphere packing.
The D&C method is a solution technique for constraint satisfaction problems that have N variables and M constraints. Each constraint involves a subset of the N variables. The objective is to determine a configuration of the N variables such that all M constraints are satisfied. Instead of working directly with variable values, the D&C method instead is concerned with the dynamics of “replicas” of the variables. For each variable involved in a particular constraint, a replica of that variable is generated. The number of replicas of a particular variable equals the number of constraints that variable is involved in.
D&C uses “divide projections” and “concur projections.” During a divide projection, replica values are modified, by as small a change as possible, such that each constraint is locally satisfied. During a concur projection, (possibly different) replicas of the same variable are forced to equal each other by averaging their values.
A key idea in the D&C algorithm is to avoid “short cycles” (or “traps”) in the dynamics through the use of a “difference map” (DM) combination of “divide” and “concur” projections at each iteration.
Let r be the vector containing all the values of replicas of all variables. This vector can be partitioned into mutually exclusive sets in two distinct ways. First, it can be partitioned into sets of replicas corresponding to each constraint. We use r_{(a) }to denote the vector of replica values of all the replicas associated with a constraint a, where a can take on integer values between 1 and M Second, it can be partitioned into a set of replicas corresponding to each variable. We use r_{[i]} to denote the vector of replica values of all replicas associated with a variable i, where i can take on integer values between 1 and N.
The “divide projection” P_{D }forces all constraints to be satisfied, by changing the replica values r_{(a) }as little as possible, and is applied independently to each set r_{(a) }for a from 1 to M. After application of the divide projection, replicas of the same variable do not necessarily agree with one another.
The “concur projection” P_{C }equalizes the replicas of the same variable and is applied independently to each set r_{[i]} for i from 1 to N. After application of the concur projection, replicas of the same variable agree with one another but the constraints are not necessarily all satisfied.
A strategy is needed to combine the two projections to find a set of replica values that satisfy all constraints, and at the same time ensure that all replicas associated with each variable are equal. Perhaps the most obvious way to use the formalism of replicas and the divide and concur projections is a straightforward alternation between the two projections. This “alternating projections” approach works well for convex constraints, but otherwise sometimes gets “stuck” in short cycles that do not correspond to correct solutions.
The difference map (DM) dynamics improves upon the alternating projections strategy by modifying the combination of the divide and concur projections in a way that turns short cycles into “repellers.” The dynamics of the DM for a single iteration is a three step process.
In the first step, the replicas move twice the desired amount indicated by the divide projection. Their new values are now called “overshoot” values of the replicas.
In the second step, the concur projection is applied to the overshoot values of the replicas to obtain “concurred” values of the replicas.
In the third step, the concurred values of the replicas are corrected to obtain the replica for the next iteration by subtracting the amount that the replicas values were “overshot” in the first step.
The dynamics of the DM are illustrated in
Of course, DM might get caught in more complicated cycles or “strange attractors” and never find an existing solution; but least it does not get caught in simple local cycles.
SUMMARY OF THE INVENTIONThe embodiments of the invention provide belief propagation (BP) decoders for lowdensity paritycheck (LDPC) that significantly improve the error floor compared to standard belief propagation (BP) decoders.
Our decoders are motivated by the idea that the “trapping sets” that cause poor error floor performance in standard BP decoders are related to the “traps” of iterated projection algorithms, and therefore our decoders use the “difference map” idea from the DC method to create decoders that avoid these traps and have improved error floor performance. Our decoders come in two embodiments.
One embodiment more closely follows the ideas of the DC iterative constraint satisfaction process, but with alterations necessary to create an errorcorrecting decoder. This embodiment has good error floor behavior, but requires a large number of iterations to determine a codeword, and can return codewords that are not the most likely ones. These are called “undetected errors.” We refer to this BP decoder as a divide and concur (DC) belief propagation (DCBP) decoder.
Another embodiment of the BP decoder uses updating rules that are similar to the standard “minsum” or “sumproduct” BP decoder but with important alterations based on the “difference map” (DM) idea from the Divide and Concur method. This “difference map belief propagation” (DMBP) decoder can dramatically improve the error floor with complexity comparable to that of standard BP decoders, on both additive white Gaussian noise, and binary symmetric channels.
The embodiments of our invention provide methods for decoding graphbased codes, including lowdensity parity check (LDPC) codes, using novel belief propagation (BP) decoders. The codes can be used to store and communicate data. More specifically, our invention is well suited for highdensity storage, and high data rates, such as magnetooptic storage media, and fiberoptic communication systems.
Because we have done extensive work on LDPC codes and BP decoders, see U.S. Pat. Nos. and Publications 7,376,173, 7,373,585, 7,191,376, 7,103,825, 7,103,818, 6,857,097, 6,771,197, 20080316069, 20080235515, 20080052594, 20070217432, 20070174633, 20060123241, 20060048038, and numerous scientific papers, we were intrigued whether any of the principles of the D&C method could be applied to LDPC codes and BP decoders. We note that there is nothing in the conventional D&C method that anticipates the elements of the BP decoders of our invention.
We were particularly curious whether the way that traps are avoided by D&C could solve the problem plaguing standard BP decoders, namely the trapping sets that lead to unacceptable error floors.
We wondered whether D&C could be adapted to the decoding problem by introducing parity check constraints. However, we found in trying to construct a decoder based on D&C that even more important than the parity check constraints was an “energy” constraint that takes account of evidence provided by channel observations, a type of information not present in pure constraint satisfaction problems.
In our DCBP decoder, the energy constraint is needed to enforce that a correlation between channel observations, i.e., the received signal, and a desired codeword should be at least some minimal amount. The energy constraint ensures that during the decoding process the “candidate solution” does not “wander” too far from the actual channel observation.
We also wondered whether the DM concept in D&C could be adapted for the standard minsum or sumproduct BP decoder setting, to construct an improved minsum or sumproduct BP decoder that avoided traps. We have therefore invented a way to convert and adapt the difference map idea to a BP decoder to provide a difference map belief propagation (DMBP) decoder. The DMBP decoder is a minsum or sumproduct BP decoder, but with modified dynamics. Our DMBP decoder improves the error floor substantially when compared with standard minsum or sumproduct belief propagation (BP) decoders.
D&C as a MessagePassing Method
To construct a BP decoder using the ideas of the D&C method, we needed first to reconstruct D&C using the formalism of a messagepassing method. Recall that a BP decoder passes “messages” on a factor graph, which at first sight is very different from the updating of replica values that is performed in the D&C method. The “messages” and “beliefs” we define to accomplish D&C are similar to those in a conventional BP decoder, however, the messageupdate and beliefupdate rules are different.
As shown in
Each variable node is connected to associated constraints by an edge. On each edge there are two messages, one from the constraint node adjoining the edge to the variable node adjoining the edge, and one in the other direction. In
In most BP decoders, the “belief” usually indicates a probability, or a loglikelihood ratio, for a variable node having a particular value. In our DCBP decoder, described in more detail below, we make a mapping of codeword bits of value 0 or 1 to symbols of value +1 or −1 respectively. We then allow the “belief” of a bit to have any real value, with more positive values corresponding to a greater likelihood of the bit has value 0, and more negative values corresponding to a greater likelihood that the bit has value 1.
Perhaps surprisingly, we identify the D&C replica values with messages passed along the edges of our graph. We denote as r_{[i]a}(t) the value of the replica on the edge joining variable i to constraint a at the beginning of iteration t. We similarly denote the “overshoot” value of the same replica as r_{[i]a}^{over}(t) and the “concurred” value as r_{[i]a}^{conc}(t). In the corresponding language of messages, r_{[i]a}(t) is denoted m_{i→a}(t), the outgoing message from variable node i to constraint node a at iteration t.
In the first step of the D&C method as described above, replica values are updated to overshoot values by moving twice the amount indicated by the divide projection. We defined overshoot conventionally to mean to shoot or pass over, or beyond so as to miss. We deliberately overshoot the target value, because that overshooting, in combination with the later correction, functions to avoid getting stuck in local traps.
Because overshoot messages are computed locally at a constraint node as a function of incoming messages, we can think of the overshoot values r_{[i]a}^{over}(t) as the outgoing messages from node a, i.e., m_{a→i}(t). The set of all outgoing messages from node a at time t is denoted M_{a→}, The set of all incoming messages M_{→a}(t) is analogously defined.
The second step of the D&C method applies the concur projection to the overshoot replicates to obtain “concurred” replica values r_{[i]a}^{conc}(t). We can think of the concurred values as “beliefs” b_{i}, determined from all incoming messages.
In standard BP, beliefs are also functions of all incoming messages. However, while in standard minsum BP the belief is a sum of all incoming messages, in D&C, the concurred replica is an average:
where M(i) is a set of constraints to which variable i is subject.
The third step of D&C produces the replica values for the next iteration by correcting the overshoot. Specifically, we subtract half the difference between the outgoing and incoming messages from the belief b_{i}(t), formally
We experimented with replacing the predetermined parameter ½ in Equation (1) with a more general parameter λ, but found that the choice λ=½ gave the best results.
We note that the next outgoing message also depends on the value of the same message at the previous iteration, which is not the case for standard minsum BP where the ordinary update rule is
m_{i→a}(t+1)=b_{i}(t)−m_{a→i}(t). (2)
The overall structure of standard minsum BP and the D&C method as presented in the language of BP is similar. In both, beliefs are iteratively updated by incoming messages from the constraint nodes, which in turn depend on the incoming messages from the variable nodes. The differences, as in all variants of different BP decoders, are in the specific forms of the messageupdate and beliefupdate rules, and the fact that a message update rule for a message out of a variable node in the D&C version of BP also depends on the value of the same message in the previous iteration. The latter difference is particularly unobvious and clearly novel.
DCBP Decoder for LDPC Codes
We call our particular version of the divide and concur method, in the context of decoding graphbased codes, DCBP decoding. We describe our approach to decoding binary LDPC codes, although generalizations to other factor graphbased codes and qary codes are straightforward. Searching for a codeword c is equivalent to seeking a binary sequence, which simultaneously satisfies all the singleparity check (SPC) constraints 310.
In contrast to standard constraint satisfaction problems, where any valid configuration will do, in decoding we want to find the configuration (the codeword) that is most likely. To do this in the context of constrained satisfaction, we add one important additional constraint; an energy 330 constraint. This constraint enforces that the likelihood of the binary sequence found must equal or exceed some minimal amount. With this additional constraint, the decoding can be divided into many simple subproblems, which can be solved independently.
Let M and N respectively be the number of SPC constraints 310 and bits of the binary LDPC code 320. Let H be a parity check matrix, which defines the code. For the purposes of illustration, we assume that the binaryphase shift keying (BPSK) signaling modulation scheme is used, mapping a binary codeword c to a sequence x of +1 and −1 symbols. The sequence x is transmitted through a channel, and received as a sequence y. It is the goal during the BP decoding to recover x from y.
The set of loglikelihood ratio (LLRs) nodes 340 in
The most likely value of x, given y, is a sequence of ±1's that satisfies all the SPC constraints 310 while minimizing the energy E, defined as E=−ΣL_{i}x_{i}. Instead of directly minimizing the energy, we define an energy constraint, which requires that the energy be less than some maximum amount E_{max}. Our original idea was to make E_{max }sufficiently negative so that only the most likely sequence could satisfy all the constraints.
To summarize, as shown in
where different choices of the parameter E_{max }result in different decoders. For a lowdensity parity check code, each SPC constraint involves only a small number of codeword bit variables, but the energy constraint involves all codeword bit variables.
It was not obvious to us how best to select the parameter E_{max}. We performed experiments to search for an E_{max }that optimized the performance of the decoder. Surprisingly, we discovered that the best choice of E_{max }is one for which the energy constraint can actually never be satisfied:
where 0<ε<<1. Since the x values are ±1 the magnitude of the energy of any configuration is lowerbounded by
We realized that the fact that the energy constraint can never be satisfied is not a problem for DCBP because of the termination criterion we choose. The decoder terminates when it finds a codeword that satisfies all the SPC constraints 310, regardless of whether the energy constraint 330 is satisfied. Thus the role of the energy constraint is to keep beliefs substantially the same as the symbols y of the received codeword.
To explain all the details of the overall DCBP decoder, we now explain how we construct divide and concur projections for LDPC decoding.
Divide and Concur Projections for LDPC Decoding
In the following, we partition the overall divide projection P_{D }into a set of M+1 projections P^{a}_{D}. Each projection operates independently
 on the vector of the messages incoming to constraint node a,
m_{→a}(t)={m_{i→a}(t): iεN(a)},
and outputs a vector of projected messages P^{a}_{D}(m_{→a}(t)), with the same dimensionality.
The output vector is as near as possible to the original messages, m_{→a}(t), while satisfying the a^{th }constraint. Singleparity check (SPC) constraints 310 require that the variables involved in a constraint are all ±1 and that an even number are −1's. For each SPC constraint, we perform the following steps.

 Assign a “hard” decision h_{ia}=1 if m_{i→a}≧0, and h_{ia}=−1 otherwise.
 Check if h_{a }has an even number of −1's. If true, return P^{a}_{D}(m_{→a})=h_{a}.
 Else, if h_{a }has an odd number of −1's, let v=argmin m_{i→a}. For a binary symmetric channel, several messages can have the same minimum. In this case, we randomly select one, and use its index as v.
Invert h_{va}, i.e., if h_{va}=−1, set it to 1, and if h_{va}=1, set it to −1. Then return
P^{a}_{D}(m_{→a})=h_{a}.
For the energy constraint we perform the following steps. First, recall that the energy constraint is
This implies a divide projection on the vector of messages m_{→0}(t), performed as follows.

 If the energy constraint is satisfied by the current messages m_{→0}(t), return the current messages, i.e., P^{0}_{D}(m_{→0}(t))=m_{→0}(t).
 (Recall however that for the E_{max }specified above the energy constraint 330 is never satisfied.)
 If the energy constraint is not satisfied by the current messages, find the h_{0}, which is the closest vector to m_{→0}(t) and temporarily satisfies the energy constraint. The i^{th }component h_{i0}, is
and return P_{D}^{0}(m_{→0})=h_{0}.
The concur projection P_{C }can be partitioned into a set of N projections P^{i}_{C}, one for each variable i for i in the range 1 to N. Each projection operates independently on the vector of messages m_{→i}(t)={m_{a→i}(t): aεM(i)}, and returns the belief b_{i}, which is the average over the components of the vector m_{→i}.
B. DCBP Decoder
Initialization 400:
Set the maximum number of iterations to T_{max }and the current iteration t to 1, and set messages m_{i→a}(t=1)=2p_{i}−1, for all codeword bit nodes i and constraint nodes a neighboring variable node i, i.e., aεM(i). Here p_{i }is an initialization parameter indicating the belief that the i^{th }transmitted symbol x_{i }is 1. One good choice for the initialization parameter is given by the formula.
p_{i}=exp(L_{i})/(1+exp(L_{i})).
Update Messages from Constraints to Variables with Overshoot 410:
The messages m_{→a }a incoming to constraint a are used to determine the outgoing messages m_{a→} using the overshoot formula
m_{a→}(t)=m_{→a}(t)+2[P_{D}^{a}(m_{→a}(t))−m_{→a}(t)],
where P_{D}^{a }is the divide projection operation for constraint a.
Update Beliefs 420:
We update the beliefs at the variable nodes i using the concur projections
Check if Codeword has been Found 430:
Set c=1, if b (t)<0, otherwise set c=0 if b_{i}(t)>0, and randomly set c to 0 or 1, if b_{i}(t)=0. Compute the matrix product H c. If H c=0, output c as the codeword 431 and stop.
Update Messages from Variables to Constraints with Overshoot Correction 440:
Increment t=t+1 450. If t>T_{max}, terminate, and return failure 441. Otherwise, update the message outgoing from each variable node to correct for the “overshoot” as in Equation (1), and go to Step 1.
IV. DMBP DECODER
As described above, the DCBP decoder performs reasonably well, resulting in biterror rate (BER) performance equal to standard BP decoders, and even superior to them in the error floor regime. However, the DCBP decoder often takes many iterations to converge, and DCBP decoder errors are often “undetected” errors—convergence to a nontransmitted codeword. In contrast BP typically fails to converge rather than converging to an incorrect codeword. Extracting the concept of the difference map (DM) that is central to the D&C method, we adapt the DM to our BP decoder. The result is a novel differencemap belief propagation (DMBP) decoder. The complexity of the DMBP decoder is comparable to that of standard BP decoders, but has a BER significantly lower than standard BP decoders in the errorfloor regime.
As shown in
Notice that the factor graph in
Our motivation for inventing the DMBP decoder was that BP decoders generally perform well, but seem to use something like an iterated projection strategy. We hoped that the trapping sets that cause the errorfloor in standard BP decoders are related to the “traps” that the difference map dynamics of D&C are supposed to ameliorate.
Hence, we apply the difference map (DM) idea of D&C to BP decoders. For simplicity, we describe a version of DMBP in terms of the minsum BP decoder using messages and beliefs that correspond to loglikelihood ratios (LLR) 540. We term the resulting decoder minsum DMBP. We note that the minsum message update rule is much simpler to implement in hardware than the standard sumproduct rule:
Usually, sumproduct or some approximation to the sumproduct BP decoders are preferred over minsum BP decoders because of better performance. However, to our surprise, our minsum DMBP decoder has better performance than the standard sumproduct BP decoder, as well as the analogous sumproduct DMBP decoder.
We use the same notation for messages and beliefs that were used in the description of the DCBP decoder. We compare, on an intuitive level, the minsum DMBP decoder with the DCBP decoder in terms of belief updates and messageupdates at both the variable and constraint check nodes.
MinSum BP Update Rule with Overshoot Beginning with the messageupdates at a check node, the standard minsum BP update rules take incoming messages m_{i→a}, and determine outgoing messages according to
where N(a)\i is the set of variable nodes neighboring node a except for variable node i, sgn(m_{j→a})=1 if m_{j→a}>0, sgn(m_{j→a})=−1 if m_{j→a}<0, and 0 otherwise.
The standard minsum update, in some sense, also “overshoots” the correct solution. The standard minsum rules for updating the message from a check node to a variable node ignore the incoming message from that variable node. The result is that messages outgoing from check nodes, at least in terms of the sign, “overshoot” the values needed to satisfy the constraint. For example, at a constraint node that has three incoming positive messages and one incoming negative message, we obtain three outgoing negative messages, and one outgoing positive message. In a very loose interpretation, and with some stretch of the imagination, this “overshoots” the “correct” solution of having an even number of negative messages. As the standard minsum update rules for check nodes overshoot, and we need overshoot, we directly use the standard minsum update rules for messages from parity check constraints to variables unmodified for minsum DMBP.
Compromise Belief Update Rule
The standard minsum BP rule for calculating the current beliefs of a particular variable node sums all messages incoming toward that variable node, including the message from the observation of the channel output. The D&C rule instead averages the incoming messages. Therefore, for DMBP we decide to use a compromise rule
where z is a parameter selected to optimize decoder performance. Instead of the conventional value of z=1, we use values of z in a range 0.25 to 0.5, e.g., z=0.4. The reason for this value is that it works best for most of the codes we have tried.
Message Update Rule at the Variable Nodes
For the message update rule at the variable nodes, we implement a parameterized version of the overshoot “correction” rule of D&C, i.e., the difference map (DM). Our intuitive idea is that, perhaps, the conventional BP is missing some correction, which is important to escape trapping sets, like the DM does for local traps in D&C.
However, because our “overshoot” rules given by equation (3) do not obviously overshoot the correct answer by a welldefined amount, we parameterize the D&C “overshoot correction” rule. Combining the D&C overshoot correction rule equation (1) with the compromise belief rule equation (4), we obtain the message update rule
where we use a parameter α.
The parameter α can be considered a “damping” parameter, which controls how much the message m_{i→a }at time t+1 depends on a value of the message at time t. If α=½, then equation (5) is exactly the same as the combination of equation (1) and (4). However, other parameter values may give better decoding performance.
Initialization 600:
Set the maximum number of iterations to T_{max }and the current iteration t to 1, and set messages m_{i→a}(t)=L_{i }for all i and all aεM(i).
Update Messages from Constraints to Variables with Overshoot 610:
Given the messages m_{→a}(t) incoming to constraint node a, the outgoing messages m_{a→}(t) are calculated using the minsum update rule described above in Equation (3).
Update Beliefs 620:
Beliefs b_{i}(t) are computed at each variable node i using the compromise update rule described above in Equation (4).
Check if Codeword has been Found 630:
Construct a lengthN vector c as follows. Set c_{i}=1, if b_{i}(t)<0, otherwise set c_{i}=0 if b_{i}(t)>0, and randomly set c_{i }to 0 or 1, if b_{i}(t)=0. If Hc=0, output c as the codeword and stop.
Update Messages from Variables to Constraints with Overshoot Correction 640:
Increment t=t+1 650. If t>T_{max }terminate and return failure 641. Otherwise, update each message out of the variable nodes using the parameterized “overshoot correction” rule given in Equation (5) and go back to Step 1.
Although the invention has been described with reference to certain preferred embodiments, it is to be understood that various other adaptations and modifications can be made within the spirit and scope of the invention. Therefore, it is the object of the append claims to cover all such variations and modifications as come within the true spirit and scope of the invention.
Claims
1. A method for decoding a code using messagepassing, wherein the code is represented by a factor graph, wherein the factor graph includes variable nodes indexed by i and constraint nodes indexed by a, wherein the variable nodes and the constraint nodes are connected by edges for transferring messages mi→a outgoing from the variable nodes to the constraint nodes and messages ma→i incoming from the constraint nodes to the variable nodes, and wherein the messages mi→a are initialized using beliefs bi of the set of variables nodes based on a received set of symbols y, and a processor for iteratively performing steps of the method, comprising the steps of:
 generating the messages ma→i by overshooting the messages mi→a at the constraint nodes;
 updating the beliefs bi at the variable nodes using the messages ma→i;
 determining if the codeword has been found;
 outputting the codeword if true; and otherwise
 updating the messages mi→a using a correction for the overshooting if false.
2. The method of claim 1, further comprising:
 terminating if a termination condition is reached.
3. The method of claim 1, wherein the decoding is performed in a divide and concur belief propagation (DCBP) decoder.
4. The method of claim 1, wherein the decoding is performed in a difference map belief propagation (DMBP) decoder.
5. The method of claim 1, wherein the decoding is applied in a high data rate application.
6. The method of claim 1, wherein the decoding is applied in a highdensity storage application.
7. The method of claim 1, wherein the code includes symbols, and the symbols have two values or more.
8. The method of claim 3, wherein the codeword includes symbols, and wherein the factor graph has one constraint node for each parity check constraint, and one variable node for each symbol xi in the codeword c, and all the variable nodes are connected to an energy constraint node, and the energy constraint node is connected to a set of loglikelihood ratio (LLR) nodes Li.
9. The method of claim 8, wherein the symbols are bits having values 0 or 1, and further comprising:
 mapping 0 to +1, and mapping 1 to −1.
10. The method of claim 1, wherein next messages mi→a from the variable nodes to the constraint nodes depend upon a current messages ma→i and the same messages mi→a during previous iterations.
11. The method of claim 10, wherein next messages mi→a from the variable nodes to the constraint nodes are updated by subtracting a difference between the messages mi→a and the messages ma→i from the belief bi according to where λ is a predetermined parameter.
 mi→a(t+1)=bi(t)−λ[ma→i(t)−mi→a(t)],
12. The method of claim 11, wherein λ=½.
13. The method of claim 1, wherein each belief bi is an average b i ( t ) = 1 M ( i ) ∑ a ∈ M ( i ) m a → i ( t ),
 where the variable node i is subject to the set of constraint nodes, and t represents an iteration.
14. The method of claim 8, wherein the energy constraint is E≦Emax, where
 Emax is a parameter and E=−ΣLixi.
15. The method of claim 14, wherein the parameter Emax is set to a value such that the energy constraint is never satisfied to keep the beliefs bi substantially the same as the received symbols y.
16. The method of claim 4, wherein the factor graph has one constraint node for each parity check constraint, and one variable for each bit in the codeword c, and each variable node is connected to an energy constraint node, and each energy constraint node is connected to a loglikelihood ratio node Li.
17. The method of claim 16, wherein the messages ma→i are updated using m a → i = ( min j ∈ N ( a ) \ i m j → a ) ∏ j ∈ N ( a ) \ i sgn ( m j → a ),
 where sgn(mj→a)=1 if mj→a>0, sgn(mj→a)=−1 if mj→a<0, and 0 otherwise.
18. The method of claim 16, wherein the messages ma →i are updated according to m a → i = 2 tanh  1 ( ∏ j ∈ N ( a ) \ i tanh ( m j → a / 2 ) ).
19. The method of claim 1, wherein each belief bi is updated according to b i = z ( L i + ∑ a ∈ M ( i ) m a → i ),
 where z is a parameter selected to optimize a performance of the decoding.
20. The method of claim 19, where the parameter z is set to a value in a range 0.25 to 0.5.
21. The method of claim 20, where the messages mi→a are updated according to m i → a ( t + 1 ) = α ( 2 zL i + ( 2 z  1 ) m a → i ( t ) + 2 z ∑ b ∈ M ( i ) \ a m b → i ( t ) ) + ( 1  α ) m i → a ( t ),
 where α is a damping” parameter that controls how much the message mi→a at time t+1 depends on a value of the message at time t.
22. The method of claim 21, where the parameter α equals ½.
23. A decoder for decoding a code using messagepassing, wherein the code is represented by a factor graph, wherein the factor graph includes variable nodes indexed by 1 and constraint nodes indexed by a, wherein the variable nodes and the constraint nodes are connected by edges for transferring messages mi→a outgoing from the variable nodes to the constraint nodes and messages ma→i incoming from the constraint nodes to the variable nodes, wherein the messages mi→a are initialized based on beliefs bi based on a received set of symbols y, comprising a processor, comprising:
 means for generating the messages ma→i by overshooting the messages mi→a at the constraint nodes;
 means for updating the beliefs bi using the messages ma→i;
 means for determining if the codeword has been found;
 means for outputting the codeword if true; and otherwise means for updating the messages mi→a using correction for the overshooting if false.
Type: Application
Filed: Jun 21, 2010
Publication Date: Feb 17, 2011
Inventors: Jonathan Yedidia (Cambridge, MA), Yige Wang (Natick, MA), Stark Draper (Madison, WI)
Application Number: 12/819,396
International Classification: H03M 13/05 (20060101); G06F 11/10 (20060101);