Machine Translation Distortion ModelRecap: DM in Word Alignment ModelsDistance ModelLexicalized Reordering ModelsBlock Distortion ModelSlide 6Slide 7Slide 8Slide 9Slide 10Slide 11Moses CodeSlide 13Distortion Model TableDistance-based ITG Reordering ModelSummaryStephan Vogel - Machine Translation 1Machine Translation Distortion ModelStephan VogelSpring Semester 2011Stephan Vogel - Machine Translation 2Recap: DM in Word Alignment ModelsHMM alignment: Jump modelCan be conditioned on word classesBalance between data andparameters in modelLarger corpora -> richer models)(just even or ),|(11 jjjjaapIaapjCICCaapjjjjjposition at wordof class is with ),,,|(11 FE3-102Stephan Vogel - Machine Translation 3Distance ModelDecoder typically generates target sequence sequentially, while jumping forth and back on source sentenceSimplest reordering modelCost of a reordering depends only on the distance of the reorderingDistribution can be estimated from alignmentOr just a Gaussian with mean 1Or log p( aj | aj-1, I) = aj – aj-1 i.e. reordering cost proportional to distanceStephan Vogel - Machine Translation 4Lexicalized Reordering ModelsInstead of conditioning on classes, condition on actual wordsDifferent possibilities:Condition on source words vs target wordsCondition on words at start of jump (out-bound) vs words at landing point (in-bound)),,,,|(111IeeffaapjjaajjjjE EFFStephan Vogel - Machine Translation 5Block Distortion ModelGiven current block, look at links at the cornersTop: how did I come from previous phrase?Bottom: how do I continue to next phrase?FECurrentBlockLeftTopLeftBottomRightBottomRightTopPreviousBlockNextBlockCurrentBlockStephan Vogel - Machine Translation 6Block Distortion ModelTop-Left: prev-to-current = monotoneFECurrentBlockLeftTopPreviousBlockStephan Vogel - Machine Translation 7Block Distortion ModelTop-Right: prev-to-current = swapFECurrentBlockRightTopPreviousBlockStephan Vogel - Machine Translation 8Block Distortion ModelNeither top-left nor top-right: prev-to-current = disjointFECurrentBlockPreviousBlockStephan Vogel - Machine Translation 9Block Distortion ModelBottom-Right: current-to-next = monotoneFECurrentBlockNextBlockStephan Vogel - Machine Translation 10Block Distortion ModelBottom-Left: current-to-next = swapFECurrentBlockNextBlockStephan Vogel - Machine Translation 11Block Distortion ModelNeither bottom-Left nor bottom-right: current-to-next = disjointFECurrentBlockNextBlockStephan Vogel - Machine Translation 12Moses Code// orientation to previous Ebool connectedLeftTop = isAligned( sentence, startF-1, startE-1 );bool connectedRightTop = isAligned( sentence, endF+1, startE-1 );if ( connectedLeftTop && !connectedRightTop) extractFileOrientation << "mono";else if (!connectedLeftTop && connectedRightTop) extractFileOrientation << "swap";else extractFileOrientation << "other";// orientation to following Ebool connectedLeftBottom = isAligned( sentence, startF-1, endE+1 );bool connectedRightBottom = isAligned( sentence, endF+1, endE+1 );if ( connectedLeftBottom && !connectedRightBottom) extractFileOrientation << " swap";else if (!connectedLeftBottom && connectedRightBottom) extractFileOrientation << " mono";else extractFileOrientation << " other";Stephan Vogel - Machine Translation 13Block Distortion ModelFor each phrase pair 6 counts: 2 groups of 3From previous: monotone swap otherTo next: monotone swap otherNormalize for each groupWe do not model p( orientation | phase_pair_1, phrase_pair_2 )Many overlapping and embedded blocksWould be too sparseWe model p( orientation | phrase_pair, entering )and p( orientation | phrase_pair, leaving )I.e. not really looking at the previous block, but only at the alignment linkFor each entry in the phrase table we have an entry in the distortion modelStephan Vogel - Machine Translation 14Distortion Model Tableacuerdo con el lugar de ||| according to the place of ||| 0.14286 0.14286 0.71429 0.71429 0.14286 0.14286acuerdo con nuestra información ||| according to our information ||| 0.14286 0.14286 0.71429 0.71429 0.14286 0.14286acuerdo de pesca con Marruecos ||| fisheries agreement with Morocco ||| 0.92982 0.01754 0.05263 0.78947 0.01754 0.19298acuerdo entre Israel y ||| agreement ||| 0.20000 0.20000 0.60000 0.20000 0.20000 0.60000acuerdo no porque sea bueno , ||| agreement not because it is good , ||| 0.60000 0.20000 0.20000 0.60000 0.20000 0.20000acuerdo sobre este punto ||| agreed on ||| 0.20000 0.20000 0.60000 0.20000 0.20000 0.60000acuerdos a largo plazo se iniciaron en ||| long-term arrangements began in ||| 0.60000 0.20000 0.20000 0.60000 0.20000 0.20000acuerdos globales , especialmente ||| global agreements - primarily |||0.20000 0.20000 0.60000 0.60000 0.20000 0.20000Many entries 0.6 0.2 … Phrase pair seen only onceSimple smoothingStephan Vogel - Machine Translation 15Distance-based ITG Reordering ModelSimple ITG model had very weak reordering modelCondition it on size of blocks (subtrees)Condition on distance (e.g. taken from HMM alignment)FEStephan Vogel - Machine Translation 16SummaryDistortion models in word alignment modelsDecoders work on phrases -> distortion models or phrasesIn Moses: Block reordering (also called lexicalized)Conditioned on phrase pairMonotone, swap, disjointAlternativesBased on words at the boundariesInbound/OutboundEasy to have lexicalized distortion model for