NICT_LOGO.JPG KYOTO-U_LOGO.JPG

WAT

The Workshop on Asian Translation
Evaluation Results

[EVALUATION RESULTS TOP] | [BLEU] | [RIBES] | [AMFM] | [HUMAN (WAT2018)] | [HUMAN (WAT2017)] | [HUMAN (WAT2016)] | [HUMAN (WAT2015)] | [HUMAN (WAT2014)] | [EVALUATION RESULTS USAGE POLICY]

BLEU


# Team Task Date/Time DataID BLEU
Method
Other
Resources
System
Description
juman kytea mecab moses-
tokenizer
stanford-
segmenter-
ctb
stanford-
segmenter-
pku
indic-
tokenizer
unuse
1srcben-ja2018/09/16 15:52:17248043.4344.9444.05-----NMTNoTransformer with relative position, ensemble of 4 models, rerank.
2NICT-5en-ja2018/09/03 16:49:57221942.8744.4243.49-----NMTNoBig Bidirectional Transformer. 1.5M sentences only.
3srcben-ja2018/09/16 15:26:37247942.4944.1143.20-----NMTNoTransformer with relative position, ensemble of 3 models.
4NICT-5en-ja2018/08/22 18:43:00204841.9143.5042.60-----NMTNoTensor2Tensor's transformer implementation. Used recurrently stacked layers model logic. Layer recurrence logic: 1-2-3-1-2-3-1-2-3-1-2-3. Codename: Megarecurrence.
5Kyoto-Uen-ja2017/09/04 22:31:18175941.5343.2642.13---- 0.00SMTNoSyscomb of AIAYN and KNMT
6ORGANIZERen-ja2017/08/02 01:04:49173740.7942.5541.50---- 0.00NMTNoGoogle's "Attention Is All You Need"
7NTTen-ja2017/08/01 15:42:13172940.3242.8040.95---- 0.00NMTNoEnsemble 8 Models: joint BPE 16k, BiLSTM Encoder 512*2*2, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
8NICT-2en-ja2017/07/26 14:03:00147940.1742.2541.17---- 0.00NMTNoNMT 6 Ensembles * Bi-directional Reranking
9NICT-5en-ja2018/08/22 18:48:51205040.0241.7440.77-----NMTNoRecurrently stacked 6 layer model. Contrastive. Same number of params as 1 layer model.
10NTTen-ja2017/08/01 07:12:25168439.8042.2740.47---- 0.00NMTNoEnsemble 8 Models: joint BPE 16k, BiLSTM Encoder 512*2*2, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
11NTTen-ja2017/08/01 02:22:56167338.8741.4239.71---- 0.00NMTNoSingle Model: joint BPE 16k, BiLSTM Encoder 512*2*2, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
12Kyoto-Uen-ja2017/08/01 15:49:51173138.7240.6539.37---- 0.00NMTNoEnsemble of 4 BPE Averaged
13UT-KAYen-ja2017/06/27 16:58:42136938.3241.1239.32---- 0.00NMTNoHashimoto and Tsuruoka (2017), https://arxiv.org/abs/1702.02265
14NAIST-NICTen-ja2017/07/27 21:50:15150738.2540.2939.05---- 0.00NMTNoSingle Model: SPM16k/16k, BiLSTM Encoder 512*2*2, UniLSTM Decoder 512*2, Adjusted Search
15Osaka-Uen-ja2018/09/15 23:02:05243938.0140.0039.10-----NMTYesrewarding model
16EHRen-ja2018/09/08 12:42:03224537.9740.0038.66-----NMTNoSMT reranked NMT
17NTTen-ja2017/07/30 16:16:25160837.9040.4838.61---- 0.00NMTNoSingle Model: joint BPE 16k, BiLSTM Encoder 512*2*2, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
18NICT-2en-ja2017/07/26 13:52:35147536.8538.9437.87---- 0.00NMTNoNMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
19NAIST-NICTen-ja2017/07/27 21:49:04150636.4738.5437.30---- 0.00NMTNoSingle Model: SPM16k/16k, BiLSTM Encoder 512*2*2, UniLSTM Decoder 512*2, One-best Search
20ORGANIZERen-ja2018/08/14 10:58:49190036.3738.4837.15-----NMTNoNMT with Attention
21UT-IISen-ja2017/08/01 12:03:22171036.2638.9337.06---- 0.00NMTNoNMT 8 Ensembles with beam search, Sentence Piece, Embedding layer initialization
22Kyoto-Uen-ja2016/08/18 03:45:51117236.1938.2036.78---- 0.00NMTNoBPE tgt/src: 52k 2-layer lstm self-ensemble of 3
23NAISTen-ja2015/08/25 12:39:2376135.8338.1736.61--- 0.00 0.00SMTNoTravatar System with NeuralMT Reranking
24TMUen-ja2017/08/04 11:06:51174735.7338.3437.00---- 0.00NMTNobeam_size: 10, ensemble of different dropout rates.
25Senseen-ja2018/08/24 11:38:23209235.2937.2136.02-----NMTNo
26TMUen-ja2018/09/16 12:44:23246935.0837.6936.14-----NMTNoEnsemble of 6 Baseline-NMT
27NAISTen-ja2014/07/31 11:38:3711835.0337.1635.81--- 0.00 0.00SMTNoTravatar-based Forest-to-String SMT System
28NAISTen-ja2014/08/01 17:37:2312634.8437.1535.67--- 0.00 0.00SMTNoTravatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
29NICT-2en-ja2016/08/05 17:47:05109734.6736.8635.37---- 0.00SMTYesPhrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
30naveren-ja2015/08/31 14:18:1883734.6036.1435.30--- 0.00 0.00SMTNoSMT t2s + Spell correction + NMT reranking
31TMUen-ja2017/08/02 10:20:14174134.5836.9535.63---- 0.00NMTNothe ensemble system of different dropout rate.
32NAISTen-ja2015/08/25 12:47:4976334.3836.5835.16--- 0.00 0.00SMTNoTravatar System Baseline
33TMUen-ja2018/09/16 17:25:36248434.3837.0635.44-----NMTNoEnsemble of 6 NMT ( 2 Baseline + 2 Reconstructor + 2 GAN )
34TMUen-ja2017/08/01 11:56:31170934.0536.6935.32---- 0.00NMTNobaseline system with beam20
35TMUen-ja2018/09/16 12:10:49246734.0236.6935.17-----NMTNoBaseline-NMT ( Single )
36TMUen-ja2018/09/16 12:11:58246834.0036.6535.09-----NMTNoGAN-NMT ( Single )
37TMUen-ja2018/09/16 16:56:49248233.8636.5834.89-----NMTNoReconstructor-NMT ( Single )
38UT-AKYen-ja2016/08/20 12:34:43122833.5736.9534.65---- 0.00NMTNotree-to-seq NMT model (word-based decoder)
39Wen-ja2015/08/26 16:00:4678633.2336.2134.05--- 0.00 0.00SMTNoNMT, LSTM Search, 5 ensembles, beam size 20, UNK replacing, System Combination with NMT score (Pick top-1k results from NMT)
40naveren-ja2015/08/31 14:12:0383633.1435.7533.93--- 0.00 0.00SMTNoNMT only
41Kyoto-Uen-ja2015/08/30 19:59:1883233.0635.5733.99--- 0.00 0.00EBMTNoKyotoEBMT system with bilingual RNNLM reranking
42naveren-ja2015/08/25 16:20:3077032.7634.5333.47--- 0.00 0.00SMTNoSMT t2s + Spell correction
43Wen-ja2014/08/26 16:17:1520232.6935.0433.40--- 0.00 0.00SMTNoWeblio Pre-reordering SMT System (with forest inputs)
44TMUen-ja2017/08/01 11:42:03170432.6535.0533.72---- 0.00NMTNoour baseline system in 2017
45naveren-ja2015/08/04 16:48:3258132.6334.4633.34--- 0.00 0.00SMTNoSMT t2s
46Wen-ja2014/08/16 00:57:1613232.5334.8733.26--- 0.00 0.00SMTNoWeblio Pre-reordering SMT System Baseline
47TOSHIBAen-ja2015/07/28 16:24:3752432.0634.1732.76--- 0.00 0.00SMT and RBMTYesRBMT with SPE(Statistical Post Editing) system
48TOKYOMTen-ja2016/08/19 23:18:46121732.0334.7732.98---- 0.00NMTNoCombination of NMT and T2S
49TOSHIBAen-ja2015/07/30 11:22:1154031.8233.9432.54--- 0.00 0.00SMT and RBMTYesSystem combination SMT and RBMT(SPE) with RNNLM language model
50EHRen-ja2016/08/15 11:21:51114031.3233.5832.28---- 0.00SMTNoPBSMT with preordering (DL=6)
51bjtu_nlpen-ja2016/08/16 11:21:07114331.1833.4731.80---- 0.00NMTNoRNN Encoder-Decoder with attention mechanism, single model
52Kyoto-Uen-ja2014/09/01 21:06:3226731.0933.5531.73--- 0.00 0.00EBMTNoOur new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
53ORGANIZERen-ja2014/07/11 20:03:211231.0533.4432.10--- 0.00 0.00SMTNoTree-to-String SMT (2014)
54ORGANIZERen-ja2015/09/10 13:29:3587531.0533.4432.10--- 0.00 0.00SMTNoTree-to-String SMT (2015)
55Kyoto-Uen-ja2016/08/19 10:18:09120131.0333.4031.88---- 0.00EBMTNoKyotoEBMT 2016 w/o reranking
56EHRen-ja2015/09/09 12:21:3487330.8833.2031.74--- 0.00 0.00SMT and RBMTYesSystem combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
57Wen-ja2015/08/28 14:30:5681330.7234.1931.57--- 0.00 0.00OtherNoNMT, LSTM Search, Beam Size 20, Ensemble of 2 models, UNK replacing
58Kyoto-Uen-ja2015/08/27 22:56:1380530.6933.2531.71--- 0.00 0.00EBMTNoKyotoEBMT system without reranking
59SAS_MTen-ja2014/09/01 10:39:2726430.4733.0031.47--- 0.00 0.00SMTNoSyntactic reordering Hierarchical SMT (using part of data)
60EHRen-ja2015/09/09 12:22:5887430.4632.5631.28--- 0.00 0.00SMT and RBMTYesRBMT with user dictionary plus SPE.
61Kyoto-Uen-ja2014/08/25 14:16:1518630.2532.7830.84--- 0.00 0.00EBMTNoUsing n-best parses and RNNLM.
62TOKYOMTen-ja2016/08/12 11:21:43113130.2133.3831.24---- 0.00NMTNochar 1 , ens 2 , version 1
63ORGANIZERen-ja2014/09/16 13:36:3536730.1932.5630.94--- 0.00 0.00SMTNoHierarchical Phrase-based SMT (2014)
64UT-AKYen-ja2016/08/20 09:25:15122430.1433.2031.09---- 0.00NMTNotree-to-seq NMT model (character-based decoder)
65EHRen-ja2015/08/22 12:28:1574229.7832.3630.71--- 0.00 0.00SMTNoPhrase based SMT with preordering.
66Kyoto-Uen-ja2014/08/31 10:33:2325329.7632.4630.46--- 0.00 0.00EBMTNoOur new baseline system after several modifications.
67Kyoto-Uen-ja2014/08/19 10:16:0213428.9331.6129.59--- 0.00 0.00EBMTNoOur baseline system using 3M parallel sentences.
68Senseen-ja2014/08/25 01:07:2718427.9230.1828.66--- 0.00 0.00SMTNoBaseline SMT
69Senseen-ja2015/08/28 19:22:2482127.9130.0328.78--- 0.00 0.00SMTNoBaseline-2015 (train1 only)
70ORGANIZERen-ja2014/07/11 19:49:03527.4829.8028.27--- 0.00 0.00SMTNoPhrase-based SMT
71ORGANIZERen-ja2016/11/16 10:50:13133426.1928.2226.68---- 0.00NMTYesOnline A (2016/11/14)
72Senseen-ja2015/07/28 22:26:5553125.2227.2225.90--- 0.00 0.00SMTNoBaseline-2015
73Senseen-ja2015/08/18 22:04:0971524.4326.5825.36--- 0.00 0.00SMTYesPassive JSTx3
74Senseen-ja2015/08/18 21:52:0170024.1326.2424.96--- 0.00 0.00SMTNoBaseline-dictmt
75Osaka-Uen-ja2018/09/16 13:06:45247023.2425.5024.26-----SMTNopreordering with neural network
76JAPIOen-ja2016/08/17 12:50:50116520.5222.5621.05---- 0.00SMTYesPhrase-based SMT with Preordering + JAPIO corpus + rule-based posteditor
77ORGANIZERen-ja2014/07/18 11:02:253419.6621.6320.17--- 0.00 0.00OtherYesOnline A (2014)
78ORGANIZERen-ja2016/07/26 11:31:47104118.2819.8118.51---- 0.00OtherYesOnline A (2016)
79ORGANIZERen-ja2015/08/25 18:54:2977418.2219.7718.46--- 0.00 0.00OtherYesOnline A (2015)
80ORGANIZERen-ja2015/09/10 19:02:3888917.8019.5218.11--- 0.00 0.00OtherYesOnline B (2015)
81ORGANIZERen-ja2014/07/22 13:30:139117.0418.6717.36--- 0.00 0.00OtherYesOnline B (2014)
82ORGANIZERen-ja2014/07/21 11:38:126613.1814.8513.48--- 0.00 0.00OtherYesRBMT B (2014)
83ORGANIZERen-ja2015/09/10 14:26:2888313.1814.8513.48--- 0.00 0.00OtherYesRBMT B (2015)
84ORGANIZERen-ja2014/07/21 11:40:506812.8614.4313.16--- 0.00 0.00OtherYesRBMT A
85ORGANIZERen-ja2014/07/23 14:50:449512.1913.3212.14--- 0.00 0.00OtherYesRBMT C

Notice:

Back to top

RIBES


# Team Task Date/Time DataID RIBES
Method
Other
Resources
System
Description
juman kytea mecab moses-
tokenizer
stanford-
segmenter-
ctb
stanford-
segmenter-
pku
indic-
tokenizer
unuse
1srcben-ja2018/09/16 15:26:3724790.8503180.8522090.857017-----NMTNoTransformer with relative position, ensemble of 3 models.
2srcben-ja2018/09/16 15:52:1724800.8476780.8501800.855073-----NMTNoTransformer with relative position, ensemble of 4 models, rerank.
3NICT-5en-ja2018/09/03 16:49:5722190.8471340.8493990.853634-----NMTNoBig Bidirectional Transformer. 1.5M sentences only.
4ORGANIZERen-ja2017/08/02 01:04:4917370.8448960.8475590.851471----0.000000NMTNoGoogle's "Attention Is All You Need"
5NICT-2en-ja2017/07/26 14:03:0014790.8422060.8481700.849929----0.000000NMTNoNMT 6 Ensembles * Bi-directional Reranking
6NICT-5en-ja2018/08/22 18:48:5120500.8410110.8443600.848283-----NMTNoRecurrently stacked 6 layer model. Contrastive. Same number of params as 1 layer model.
7Kyoto-Uen-ja2017/09/04 22:31:1817590.8410000.8437250.847819----0.000000SMTNoSyscomb of AIAYN and KNMT
8NICT-5en-ja2018/08/22 18:43:0020480.8407760.8450420.849326-----NMTNoTensor2Tensor's transformer implementation. Used recurrently stacked layers model logic. Layer recurrence logic: 1-2-3-1-2-3-1-2-3-1-2-3. Codename: Megarecurrence.
9NTTen-ja2017/08/01 15:42:1317290.8385940.8417690.846486----0.000000NMTNoEnsemble 8 Models: joint BPE 16k, BiLSTM Encoder 512*2*2, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
10NTTen-ja2017/08/01 07:12:2516840.8358060.8399810.844326----0.000000NMTNoEnsemble 8 Models: joint BPE 16k, BiLSTM Encoder 512*2*2, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
11NAIST-NICTen-ja2017/07/27 21:50:1515070.8344920.8393210.842337----0.000000NMTNoSingle Model: SPM16k/16k, BiLSTM Encoder 512*2*2, UniLSTM Decoder 512*2, Adjusted Search
12NTTen-ja2017/08/01 02:22:5616730.8335410.8382450.841739----0.000000NMTNoSingle Model: joint BPE 16k, BiLSTM Encoder 512*2*2, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
13Kyoto-Uen-ja2017/08/01 15:49:5117310.8324720.8358700.839646----0.000000NMTNoEnsemble of 4 BPE Averaged
14UT-KAYen-ja2017/06/27 16:58:4213690.8296000.8339200.838260----0.000000NMTNoHashimoto and Tsuruoka (2017), https://arxiv.org/abs/1702.02265
15EHRen-ja2018/09/08 12:42:0322450.8287460.8333330.837806-----NMTNoSMT reranked NMT
16NTTen-ja2017/07/30 16:16:2516080.8281930.8320060.836381----0.000000NMTNoSingle Model: joint BPE 16k, BiLSTM Encoder 512*2*2, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
17UT-IISen-ja2017/08/01 12:03:2217100.8278910.8320540.836394----0.000000NMTNoNMT 8 Ensembles with beam search, Sentence Piece, Embedding layer initialization
18NICT-2en-ja2017/07/26 13:52:3514750.8267910.8344480.835255----0.000000NMTNoNMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
19Osaka-Uen-ja2018/09/15 23:02:0524390.8250610.8293280.833200-----NMTYesrewarding model
20ORGANIZERen-ja2018/08/14 10:58:4919000.8249850.8311830.833207-----NMTNoNMT with Attention
21TMUen-ja2017/08/04 11:06:5117470.8248010.8295800.832569----0.000000NMTNobeam_size: 10, ensemble of different dropout rates.
22TMUen-ja2018/09/16 12:44:2324690.8236530.8291560.831219-----NMTNoEnsemble of 6 Baseline-NMT
23NAIST-NICTen-ja2017/07/27 21:49:0415060.8219890.8272250.830116----0.000000NMTNoSingle Model: SPM16k/16k, BiLSTM Encoder 512*2*2, UniLSTM Decoder 512*2, One-best Search
24TMUen-ja2018/09/16 17:25:3624840.8209610.8267790.829178-----NMTNoEnsemble of 6 NMT ( 2 Baseline + 2 Reconstructor + 2 GAN )
25Kyoto-Uen-ja2016/08/18 03:45:5111720.8198360.8238780.828956----0.000000NMTNoBPE tgt/src: 52k 2-layer lstm self-ensemble of 3
26TMUen-ja2018/09/16 12:11:5824680.8191480.8253670.827650-----NMTNoGAN-NMT ( Single )
27TMUen-ja2018/09/16 12:10:4924670.8179480.8242390.827386-----NMTNoBaseline-NMT ( Single )
28TMUen-ja2017/08/02 10:20:1417410.8172800.8243320.825683----0.000000NMTNothe ensemble system of different dropout rate.
29TMUen-ja2018/09/16 16:56:4924820.8171530.8240660.826013-----NMTNoReconstructor-NMT ( Single )
30UT-AKYen-ja2016/08/20 12:34:4312280.8169840.8244560.827647----0.000000NMTNotree-to-seq NMT model (word-based decoder)
31TMUen-ja2017/08/01 11:56:3117090.8129260.8184430.821563----0.000000NMTNobaseline system with beam20
32NAISTen-ja2015/08/25 12:39:237610.8114790.8138270.820337---0.0000000.000000SMTNoTravatar System with NeuralMT Reranking
33TOKYOMTen-ja2016/08/12 11:21:4311310.8096910.8172580.819951----0.000000NMTNochar 1 , ens 2 , version 1
34TOKYOMTen-ja2016/08/19 23:18:4612170.8081890.8144520.818130----0.000000NMTNoCombination of NMT and T2S
35naveren-ja2015/08/31 14:12:038360.8072800.8114870.817343---0.0000000.000000SMTNoNMT only
36UT-AKYen-ja2016/08/20 09:25:1512240.8060250.8144900.815836----0.000000NMTNotree-to-seq NMT model (character-based decoder)
37Wen-ja2015/08/26 16:00:467860.8047220.8090650.814337---0.0000000.000000SMTNoNMT, LSTM Search, 5 ensembles, beam size 20, UNK replacing, System Combination with NMT score (Pick top-1k results from NMT)
38TMUen-ja2017/08/01 11:42:0317040.8022620.8096490.811057----0.000000NMTNoour baseline system in 2017
39NAISTen-ja2014/08/01 17:37:231260.8017420.8070100.811081---0.0000000.000000SMTNoTravatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
40naveren-ja2015/08/31 14:18:188370.7999660.8031540.808787---0.0000000.000000SMTNoSMT t2s + Spell correction + NMT reranking
41Wen-ja2015/08/28 14:30:568130.7968630.8026660.807186---0.0000000.000000OtherNoNMT, LSTM Search, Beam Size 20, Ensemble of 2 models, UNK replacing
42Senseen-ja2018/08/24 11:38:2320920.7962140.8007270.803453-----NMTNo
43NAISTen-ja2014/07/31 11:38:371180.7960790.8015200.806581---0.0000000.000000SMTNoTravatar-based Forest-to-String SMT System
44NAISTen-ja2015/08/25 12:47:497630.7924470.7964890.802228---0.0000000.000000SMTNoTravatar System Baseline
45Kyoto-Uen-ja2015/08/30 19:59:188320.7895140.7971820.799979---0.0000000.000000EBMTNoKyotoEBMT system with bilingual RNNLM reranking
46Wen-ja2014/08/26 16:17:152020.7850150.7900660.795027---0.0000000.000000SMTNoWeblio Pre-reordering SMT System (with forest inputs)
47NICT-2en-ja2016/08/05 17:47:0510970.7843350.7909930.793409----0.000000SMTYesPhrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
48naveren-ja2015/08/25 16:20:307700.7841740.7888110.794452---0.0000000.000000SMTNoSMT t2s + Spell correction
49naveren-ja2015/08/04 16:48:325810.7833100.7881650.793723---0.0000000.000000SMTNoSMT t2s
50Wen-ja2014/08/16 00:57:161320.7820660.7869020.792616---0.0000000.000000SMTNoWeblio Pre-reordering SMT System Baseline
51bjtu_nlpen-ja2016/08/16 11:21:0711430.7805100.7874970.791088----0.000000NMTNoRNN Encoder-Decoder with attention mechanism, single model
52ORGANIZERen-ja2016/11/16 10:50:1313340.7767870.7802170.782674----0.000000NMTYesOnline A (2016/11/14)
53Kyoto-Uen-ja2016/08/19 10:18:0912010.7712400.7797740.781960----0.000000EBMTNoKyotoEBMT 2016 w/o reranking
54TOSHIBAen-ja2015/07/28 16:24:375240.7709890.7785700.780467---0.0000000.000000SMT and RBMTYesRBMT with SPE(Statistical Post Editing) system
55TOSHIBAen-ja2015/07/30 11:22:115400.7698650.7768410.779574---0.0000000.000000SMT and RBMTYesSystem combination SMT and RBMT(SPE) with RNNLM language model
56EHRen-ja2015/09/09 12:22:588740.7685630.7765260.778113---0.0000000.000000SMT and RBMTYesRBMT with user dictionary plus SPE.
57Kyoto-Uen-ja2015/08/27 22:56:138050.7677780.7766720.778358---0.0000000.000000EBMTNoKyotoEBMT system without reranking
58EHRen-ja2015/09/09 12:21:348730.7657000.7748190.778269---0.0000000.000000SMT and RBMTYesSystem combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
59EHRen-ja2016/08/15 11:21:5111400.7599140.7714270.775023----0.000000SMTNoPBSMT with preordering (DL=6)
60Kyoto-Uen-ja2014/09/01 21:06:322670.7596080.7709080.771545---0.0000000.000000EBMTNoOur new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
61SAS_MTen-ja2014/09/01 10:39:272640.7594150.7709480.771605---0.0000000.000000SMTNoSyntactic reordering Hierarchical SMT (using part of data)
62Kyoto-Uen-ja2014/08/25 14:16:151860.7556290.7652510.766495---0.0000000.000000EBMTNoUsing n-best parses and RNNLM.
63EHRen-ja2015/08/22 12:28:157420.7535760.7660440.768105---0.0000000.000000SMTNoPhrase based SMT with preordering.
64Kyoto-Uen-ja2014/08/31 10:33:232530.7520580.7640490.766435---0.0000000.000000EBMTNoOur new baseline system after several modifications.
65ORGANIZERen-ja2014/07/11 20:03:21120.7488830.7580310.760516---0.0000000.000000SMTNoTree-to-String SMT (2014)
66ORGANIZERen-ja2015/09/10 13:29:358750.7488830.7580310.760516---0.0000000.000000SMTNoTree-to-String SMT (2015)
67Kyoto-Uen-ja2014/08/19 10:16:021340.7439690.7557440.756545---0.0000000.000000EBMTNoOur baseline system using 3M parallel sentences.
68ORGANIZERen-ja2014/09/16 13:36:353670.7347050.7469780.747722---0.0000000.000000SMTNoHierarchical Phrase-based SMT (2014)
69JAPIOen-ja2016/08/17 12:50:5011650.7234670.7285840.731474----0.000000SMTYesPhrase-based SMT with Preordering + JAPIO corpus + rule-based posteditor
70ORGANIZERen-ja2014/07/18 11:02:25340.7180190.7234860.725848---0.0000000.000000OtherYesOnline A (2014)
71Osaka-Uen-ja2018/09/16 13:06:4524700.7168890.7264690.729323-----SMTNopreordering with neural network
72ORGANIZERen-ja2016/07/26 11:31:4710410.7066390.7152220.718559----0.000000OtherYesOnline A (2016)
73ORGANIZERen-ja2015/08/25 18:54:297740.7058820.7139600.718150---0.0000000.000000OtherYesOnline A (2015)
74ORGANIZERen-ja2015/09/10 19:02:388890.6933590.7019660.703859---0.0000000.000000OtherYesOnline B (2015)
75Senseen-ja2014/08/25 01:07:271840.6904640.7005830.703049---0.0000000.000000SMTNoBaseline SMT
76Senseen-ja2015/08/28 19:22:248210.6893130.6978170.700994---0.0000000.000000SMTNoBaseline-2015 (train1 only)
77ORGANIZERen-ja2014/07/22 13:30:13910.6877970.6933900.698126---0.0000000.000000OtherYesOnline B (2014)
78ORGANIZERen-ja2014/07/11 19:49:0350.6837350.6919260.695390---0.0000000.000000SMTNoPhrase-based SMT
79ORGANIZERen-ja2014/07/21 11:38:12660.6719580.6807480.682683---0.0000000.000000OtherYesRBMT B (2014)
80ORGANIZERen-ja2015/09/10 14:26:288830.6719580.6807480.682683---0.0000000.000000OtherYesRBMT B (2015)
81ORGANIZERen-ja2014/07/21 11:40:50680.6701670.6764640.678934---0.0000000.000000OtherYesRBMT A
82ORGANIZERen-ja2014/07/23 14:50:44950.6683720.6726450.676018---0.0000000.000000OtherYesRBMT C
83Senseen-ja2015/07/28 22:26:555310.6462880.6538140.659505---0.0000000.000000SMTNoBaseline-2015
84Senseen-ja2015/08/18 21:52:017000.6373780.6427890.647831---0.0000000.000000SMTNoBaseline-dictmt
85Senseen-ja2015/08/18 22:04:097150.6359330.6415170.646682---0.0000000.000000SMTYesPassive JSTx3

Notice:

Back to top

AMFM


# Team Task Date/Time DataID AMFM
Method
Other
Resources
System
Description
juman kytea mecab moses-
tokenizer
stanford-
segmenter-
ctb
stanford-
segmenter-
pku
indic-
tokenizer
unuse
1srcben-ja2018/09/16 15:26:3724790.7810000.7810000.781000-----NMTNoTransformer with relative position, ensemble of 3 models.
2srcben-ja2018/09/16 15:52:1724800.7798200.7798200.779820-----NMTNoTransformer with relative position, ensemble of 4 models, rerank.
3NICT-5en-ja2018/09/03 16:49:5722190.7795600.7795600.779560-----NMTNoBig Bidirectional Transformer. 1.5M sentences only.
4NICT-5en-ja2018/08/22 18:43:0020480.7714000.7714000.771400-----NMTNoTensor2Tensor's transformer implementation. Used recurrently stacked layers model logic. Layer recurrence logic: 1-2-3-1-2-3-1-2-3-1-2-3. Codename: Megarecurrence.
5Kyoto-Uen-ja2017/09/04 22:31:1817590.7710200.7710200.771020----0.000000SMTNoSyscomb of AIAYN and KNMT
6NICT-5en-ja2018/08/22 18:48:5120500.7705600.7705600.770560-----NMTNoRecurrently stacked 6 layer model. Contrastive. Same number of params as 1 layer model.
7NAIST-NICTen-ja2017/07/27 21:50:1515070.7704800.7704800.770480----0.000000NMTNoSingle Model: SPM16k/16k, BiLSTM Encoder 512*2*2, UniLSTM Decoder 512*2, Adjusted Search
8ORGANIZERen-ja2017/08/02 01:04:4917370.7686300.7686300.768630----0.000000NMTNoGoogle's "Attention Is All You Need"
9NICT-2en-ja2017/07/26 14:03:0014790.7655800.7655800.765580----0.000000NMTNoNMT 6 Ensembles * Bi-directional Reranking
10NTTen-ja2017/08/01 02:22:5616730.7637700.7637700.763770----0.000000NMTNoSingle Model: joint BPE 16k, BiLSTM Encoder 512*2*2, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
11NAIST-NICTen-ja2017/07/27 21:49:0415060.7633100.7633100.763310----0.000000NMTNoSingle Model: SPM16k/16k, BiLSTM Encoder 512*2*2, UniLSTM Decoder 512*2, One-best Search
12Osaka-Uen-ja2018/09/15 23:02:0524390.7631400.7631400.763140-----NMTYesrewarding model
13NTTen-ja2017/08/01 15:42:1317290.7621700.7621700.762170----0.000000NMTNoEnsemble 8 Models: joint BPE 16k, BiLSTM Encoder 512*2*2, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
14ORGANIZERen-ja2018/08/14 10:58:4919000.7599100.7599100.759910-----NMTNoNMT with Attention
15NICT-2en-ja2017/07/26 13:52:3514750.7595700.7595700.759570----0.000000NMTNoNMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
16EHRen-ja2018/09/08 12:42:0322450.7587500.7587500.758750-----NMTNoSMT reranked NMT
17NAISTen-ja2014/08/01 17:37:231260.7587400.7587400.7587400.0000000.0000000.0000000.0000000.000000SMTNoTravatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
18NTTen-ja2017/08/01 07:12:2516840.7577400.7577400.757740----0.000000NMTNoEnsemble 8 Models: joint BPE 16k, BiLSTM Encoder 512*2*2, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
19NTTen-ja2017/07/30 16:16:2516080.7564800.7564800.756480----0.000000NMTNoSingle Model: joint BPE 16k, BiLSTM Encoder 512*2*2, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
20Kyoto-Uen-ja2017/08/01 15:49:5117310.7542200.7542200.754220----0.000000NMTNoEnsemble of 4 BPE Averaged
21UT-KAYen-ja2017/06/27 16:58:4213690.7530900.7530900.753090----0.000000NMTNoHashimoto and Tsuruoka (2017), https://arxiv.org/abs/1702.02265
22NICT-2en-ja2016/08/05 17:47:0510970.7530800.7530800.753080----0.000000SMTYesPhrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
23TMUen-ja2018/09/16 12:44:2324690.7530400.7530400.753040-----NMTNoEnsemble of 6 Baseline-NMT
24NAISTen-ja2014/07/31 11:38:371180.7528500.7528500.7528500.0000000.0000000.0000000.0000000.000000SMTNoTravatar-based Forest-to-String SMT System
25TMUen-ja2018/09/16 17:25:3624840.7524200.7524200.752420-----NMTNoEnsemble of 6 NMT ( 2 Baseline + 2 Reconstructor + 2 GAN )
26NAISTen-ja2015/08/25 12:47:497630.7522600.7522600.7522600.0000000.0000000.0000000.0000000.000000SMTNoTravatar System Baseline
27TMUen-ja2018/09/16 16:56:4924820.7521000.7521000.752100-----NMTNoReconstructor-NMT ( Single )
28TMUen-ja2017/08/02 10:20:1417410.7516600.7516600.751660----0.000000NMTNothe ensemble system of different dropout rate.
29TMUen-ja2018/09/16 12:11:5824680.7503500.7503500.750350-----NMTNoGAN-NMT ( Single )
30TMUen-ja2017/08/04 11:06:5117470.7494100.7494100.749410----0.000000NMTNobeam_size: 10, ensemble of different dropout rates.
31TMUen-ja2018/09/16 12:10:4924670.7491900.7491900.749190-----NMTNoBaseline-NMT ( Single )
32NAISTen-ja2015/08/25 12:39:237610.7488700.7488700.7488700.0000000.0000000.0000000.0000000.000000SMTNoTravatar System with NeuralMT Reranking
33Wen-ja2014/08/26 16:17:152020.7476500.7476500.7476500.0000000.0000000.0000000.0000000.000000SMTNoWeblio Pre-reordering SMT System (with forest inputs)
34Kyoto-Uen-ja2016/08/19 10:18:0912010.7475100.7475100.747510----0.000000EBMTNoKyotoEBMT 2016 w/o reranking
35UT-IISen-ja2017/08/01 12:03:2217100.7469100.7469100.746910----0.000000NMTNoNMT 8 Ensembles with beam search, Sentence Piece, Embedding layer initialization
36EHRen-ja2016/08/15 11:21:5111400.7467200.7467200.746720----0.000000SMTNoPBSMT with preordering (DL=6)
37Wen-ja2014/08/16 00:57:161320.7465700.7465700.7465700.0000000.0000000.0000000.0000000.000000SMTNoWeblio Pre-reordering SMT System Baseline
38TMUen-ja2017/08/01 11:56:3117090.7448900.7448900.744890----0.000000NMTNobaseline system with beam20
39TOSHIBAen-ja2015/07/30 11:22:115400.7446600.7446600.7446600.0000000.0000000.0000000.0000000.000000SMT and RBMTYesSystem combination SMT and RBMT(SPE) with RNNLM language model
40naveren-ja2015/08/31 14:18:188370.7446300.7446300.7446300.0000000.0000000.0000000.0000000.000000SMTNoSMT t2s + Spell correction + NMT reranking
41ORGANIZERen-ja2014/07/11 20:03:21120.7443700.7443700.7443700.0000000.0000000.0000000.0000000.000000SMTNoTree-to-String SMT (2014)
42ORGANIZERen-ja2015/09/10 13:29:358750.7443700.7443700.7443700.0000000.0000000.0000000.0000000.000000SMTNoTree-to-String SMT (2015)
43ORGANIZERen-ja2014/09/16 13:36:353670.7439000.7439000.7439000.0000000.0000000.0000000.0000000.000000SMTNoHierarchical Phrase-based SMT (2014)
44Kyoto-Uen-ja2015/08/30 19:59:188320.7437100.7437100.7437100.0000000.0000000.0000000.0000000.000000EBMTNoKyotoEBMT system with bilingual RNNLM reranking
45TOSHIBAen-ja2015/07/28 16:24:375240.7421000.7421000.7421000.0000000.0000000.0000000.0000000.000000SMT and RBMTYesRBMT with SPE(Statistical Post Editing) system
46Kyoto-Uen-ja2014/08/31 10:33:232530.7417100.7417100.7417100.0000000.0000000.0000000.0000000.000000EBMTNoOur new baseline system after several modifications.
47Senseen-ja2018/08/24 11:38:2320920.7410900.7410900.741090-----NMTNo
48Kyoto-Uen-ja2015/08/27 22:56:138050.7410500.7410500.7410500.0000000.0000000.0000000.0000000.000000EBMTNoKyotoEBMT system without reranking
49TMUen-ja2017/08/01 11:42:0317040.7406200.7406200.740620----0.000000NMTNoour baseline system in 2017
50SAS_MTen-ja2014/09/01 10:39:272640.7405800.7405800.7405800.0000000.0000000.0000000.0000000.000000SMTNoSyntactic reordering Hierarchical SMT (using part of data)
51naveren-ja2015/08/25 16:20:307700.7402100.7402100.7402100.0000000.0000000.0000000.0000000.000000SMTNoSMT t2s + Spell correction
52Kyoto-Uen-ja2016/08/18 03:45:5111720.7387000.7387000.738700----0.000000NMTNoBPE tgt/src: 52k 2-layer lstm self-ensemble of 3
53Kyoto-Uen-ja2014/09/01 21:06:322670.7386800.7386800.7386800.0000000.0000000.0000000.0000000.000000EBMTNoOur new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
54Kyoto-Uen-ja2014/08/19 10:16:021340.7378900.7378900.7378900.0000000.0000000.0000000.0000000.000000EBMTNoOur baseline system using 3M parallel sentences.
55naveren-ja2015/08/04 16:48:325810.7375900.7375900.7375900.0000000.0000000.0000000.0000000.000000SMTNoSMT t2s
56Senseen-ja2015/08/28 19:22:248210.7375600.7375600.7375600.0000000.0000000.0000000.0000000.000000SMTNoBaseline-2015 (train1 only)
57ORGANIZERen-ja2014/07/11 19:49:0350.7363800.7363800.7363800.0000000.0000000.0000000.0000000.000000SMTNoPhrase-based SMT
58Senseen-ja2014/08/25 01:07:271840.7333600.7333600.7333600.0000000.0000000.0000000.0000000.000000SMTNoBaseline SMT
59Kyoto-Uen-ja2014/08/25 14:16:151860.7323800.7323800.7323800.0000000.0000000.0000000.0000000.000000EBMTNoUsing n-best parses and RNNLM.
60Senseen-ja2015/07/28 22:26:555310.7316200.7316200.7316200.0000000.0000000.0000000.0000000.000000SMTNoBaseline-2015
61UT-AKYen-ja2016/08/20 12:34:4312280.7314400.7314400.731440----0.000000NMTNotree-to-seq NMT model (word-based decoder)
62Senseen-ja2015/08/18 22:04:097150.7314000.7314000.7314000.0000000.0000000.0000000.0000000.000000SMTYesPassive JSTx3
63Senseen-ja2015/08/18 21:52:017000.7292000.7292000.7292000.0000000.0000000.0000000.0000000.000000SMTNoBaseline-dictmt
64ORGANIZERen-ja2016/11/16 10:50:1313340.7270400.7270400.727040----0.000000NMTYesOnline A (2016/11/14)
65naveren-ja2015/08/31 14:12:038360.7258600.7258600.7258600.0000000.0000000.0000000.0000000.000000SMTNoNMT only
66Wen-ja2015/08/26 16:00:467860.7253700.7253700.7253700.0000000.0000000.0000000.0000000.000000SMTNoNMT, LSTM Search, 5 ensembles, beam size 20, UNK replacing, System Combination with NMT score (Pick top-1k results from NMT)
67TOKYOMTen-ja2016/08/19 23:18:4612170.7208100.7208100.720810----0.000000NMTNoCombination of NMT and T2S
68UT-AKYen-ja2016/08/20 09:25:1512240.7081400.7081400.708140----0.000000NMTNotree-to-seq NMT model (character-based decoder)
69Wen-ja2015/08/28 14:30:568130.7053400.7053400.7053400.0000000.0000000.0000000.0000000.000000OtherNoNMT, LSTM Search, Beam Size 20, Ensemble of 2 models, UNK replacing
70TOKYOMTen-ja2016/08/12 11:21:4311310.7052100.7052100.705210----0.000000NMTNochar 1 , ens 2 , version 1
71Osaka-Uen-ja2018/09/16 13:06:4524700.7050500.7050500.705050-----SMTNopreordering with neural network
72bjtu_nlpen-ja2016/08/16 11:21:0711430.7043400.7043400.704340----0.000000NMTNoRNN Encoder-Decoder with attention mechanism, single model
73ORGANIZERen-ja2014/07/18 11:02:25340.6954200.6954200.6954200.0000000.0000000.0000000.0000000.000000OtherYesOnline A (2014)
74ORGANIZERen-ja2015/08/25 18:54:297740.6772000.6772000.6772000.0000000.0000000.0000000.0000000.000000OtherYesOnline A (2015)
75ORGANIZERen-ja2016/07/26 11:31:4710410.6770200.6770200.677020----0.000000OtherYesOnline A (2016)
76JAPIOen-ja2016/08/17 12:50:5011650.6607900.6607900.660790----0.000000SMTYesPhrase-based SMT with Preordering + JAPIO corpus + rule-based posteditor
77ORGANIZERen-ja2015/09/10 19:02:388890.6461600.6461600.6461600.0000000.0000000.0000000.0000000.000000OtherYesOnline B (2015)
78ORGANIZERen-ja2014/07/22 13:30:13910.6430700.6430700.6430700.0000000.0000000.0000000.0000000.000000OtherYesOnline B (2014)
79EHRen-ja2015/09/09 12:21:348730.6304100.6304100.6304100.0000000.0000000.0000000.0000000.000000SMT and RBMTYesSystem combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
80ORGANIZERen-ja2014/07/21 11:40:50680.6269400.6269400.6269400.0000000.0000000.0000000.0000000.000000OtherYesRBMT A
81ORGANIZERen-ja2014/07/21 11:38:12660.6229300.6229300.6229300.0000000.0000000.0000000.0000000.000000OtherYesRBMT B (2014)
82ORGANIZERen-ja2015/09/10 14:26:288830.6229300.6229300.6229300.0000000.0000000.0000000.0000000.000000OtherYesRBMT B (2015)
83EHRen-ja2015/08/22 12:28:157420.6205000.6205000.6205000.0000000.0000000.0000000.0000000.000000SMTNoPhrase based SMT with preordering.
84EHRen-ja2015/09/09 12:22:588740.6040900.6040900.6040900.0000000.0000000.0000000.0000000.000000SMT and RBMTYesRBMT with user dictionary plus SPE.
85ORGANIZERen-ja2014/07/23 14:50:44950.5943800.5943800.5943800.0000000.0000000.0000000.0000000.000000OtherYesRBMT C

Notice:

Back to top

HUMAN (WAT2018)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1NICT-5en-ja2018/09/03 16:49:57221928.500NMTNoBig Bidirectional Transformer. 1.5M sentences only.
2srcben-ja2018/09/16 15:26:37247925.000NMTNoTransformer with relative position, ensemble of 3 models.
3NICT-5en-ja2018/08/22 18:43:00204820.250NMTNoTensor2Tensor's transformer implementation. Used recurrently stacked layers model logic. Layer recurrence logic: 1-2-3-1-2-3-1-2-3-1-2-3. Codename: Megarecurrence.
4Osaka-Uen-ja2018/09/15 23:02:0524394.500NMTYesrewarding model
5EHRen-ja2018/09/08 12:42:032245-0.500NMTNoSMT reranked NMT
6TMUen-ja2018/09/16 12:44:232469-12.000NMTNoEnsemble of 6 Baseline-NMT
7Osaka-Uen-ja2018/09/16 13:06:452470-82.250SMTNopreordering with neural network

Notice:
Back to top

HUMAN (WAT2017)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1NTTen-ja2017/08/01 15:42:13172975.750NMTNoEnsemble 8 Models: joint BPE 16k, BiLSTM Encoder 512*2*2, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
2NICT-2en-ja2017/07/26 14:03:00147974.750NMTNoNMT 6 Ensembles * Bi-directional Reranking
3NTTen-ja2017/08/01 07:12:25168472.250NMTNoEnsemble 8 Models: joint BPE 16k, BiLSTM Encoder 512*2*2, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
4NAIST-NICTen-ja2017/07/27 21:50:15150770.000NMTNoSingle Model: SPM16k/16k, BiLSTM Encoder 512*2*2, UniLSTM Decoder 512*2, Adjusted Search
5Kyoto-Uen-ja2017/08/01 15:49:51173169.750NMTNoEnsemble of 4 BPE Averaged
6ORGANIZERen-ja2017/08/02 01:04:49173769.750NMTNoGoogle's "Attention Is All You Need"
7UT-IISen-ja2017/08/01 12:03:22171068.000NMTNoNMT 8 Ensembles with beam search, Sentence Piece, Embedding layer initialization
8NAIST-NICTen-ja2017/07/27 21:49:04150663.500NMTNoSingle Model: SPM16k/16k, BiLSTM Encoder 512*2*2, UniLSTM Decoder 512*2, One-best Search
9NICT-2en-ja2017/07/26 13:52:35147562.000NMTNoNMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
10TMUen-ja2017/08/01 11:56:31170956.500NMTNobaseline system with beam20
11TMUen-ja2017/08/01 11:42:03170450.750NMTNoour baseline system in 2017

Notice:
Back to top

HUMAN (WAT2016)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1ORGANIZERen-ja2016/11/16 10:50:13133474.000NMTYesOnline A (2016/11/14)
2Kyoto-Uen-ja2016/08/18 03:45:51117255.250NMTNoBPE tgt/src: 52k 2-layer lstm self-ensemble of 3
3ORGANIZERen-ja2016/07/26 11:31:47104149.750OtherYesOnline A (2016)
4NICT-2en-ja2016/08/05 17:47:05109741.250SMTYesPhrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
5bjtu_nlpen-ja2016/08/16 11:21:07114339.500NMTNoRNN Encoder-Decoder with attention mechanism, single model
6EHRen-ja2016/08/15 11:21:51114039.000SMTNoPBSMT with preordering (DL=6)
7UT-AKYen-ja2016/08/20 12:34:43122836.250NMTNotree-to-seq NMT model (word-based decoder)
8TOKYOMTen-ja2016/08/19 23:18:46121730.500NMTNoCombination of NMT and T2S
9TOKYOMTen-ja2016/08/12 11:21:43113129.750NMTNochar 1 , ens 2 , version 1
10UT-AKYen-ja2016/08/20 09:25:15122421.750NMTNotree-to-seq NMT model (character-based decoder)
11JAPIOen-ja2016/08/17 12:50:5011654.250SMTYesPhrase-based SMT with Preordering + JAPIO corpus + rule-based posteditor

Notice:
Back to top

HUMAN (WAT2015)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1NAISTen-ja2015/08/25 12:39:2376162.250SMTNoTravatar System with NeuralMT Reranking
2Wen-ja2015/08/26 16:00:4678653.750SMTNoNMT, LSTM Search, 5 ensembles, beam size 20, UNK replacing, System Combination with NMT score (Pick top-1k results from NMT)
3naveren-ja2015/08/31 14:18:1883753.250SMTNoSMT t2s + Spell correction + NMT reranking
4Kyoto-Uen-ja2015/08/30 19:59:1883251.000EBMTNoKyotoEBMT system with bilingual RNNLM reranking
5NAISTen-ja2015/08/25 12:47:4976349.750SMTNoTravatar System Baseline
6naveren-ja2015/08/31 14:12:0383648.500SMTNoNMT only
7Wen-ja2015/08/28 14:30:5681343.500OtherNoNMT, LSTM Search, Beam Size 20, Ensemble of 2 models, UNK replacing
8Kyoto-Uen-ja2015/08/27 22:56:1380540.500EBMTNoKyotoEBMT system without reranking
9TOSHIBAen-ja2015/07/28 16:24:3752440.250SMT and RBMTYesRBMT with SPE(Statistical Post Editing) system
10ORGANIZERen-ja2015/08/25 18:54:2977434.250OtherYesOnline A (2015)
11EHRen-ja2015/08/22 12:28:1574232.500SMTNoPhrase based SMT with preordering.
12ORGANIZERen-ja2015/09/10 13:29:3587530.000SMTNoTree-to-String SMT (2015)
13ORGANIZERen-ja2015/09/10 14:26:288839.750OtherYesRBMT B (2015)
14Senseen-ja2015/08/18 22:04:09715-31.000SMTYesPassive JSTx3
15Senseen-ja2015/08/18 21:52:01700-36.250SMTNoBaseline-dictmt

Notice:
Back to top

HUMAN (WAT2014)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1NAISTen-ja2014/07/31 11:38:3711856.250SMTNoTravatar-based Forest-to-String SMT System
2NAISTen-ja2014/08/01 17:37:2312651.500SMTNoTravatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
3Wen-ja2014/08/16 00:57:1613243.250SMTNoWeblio Pre-reordering SMT System Baseline
4ORGANIZERen-ja2014/07/18 11:02:253442.500OtherYesOnline A (2014)
5Kyoto-Uen-ja2014/09/01 21:06:3226738.000EBMTNoOur new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
6Wen-ja2014/08/26 16:17:1520236.000SMTNoWeblio Pre-reordering SMT System (with forest inputs)
7ORGANIZERen-ja2014/07/11 20:03:211234.250SMTNoTree-to-String SMT (2014)
8Kyoto-Uen-ja2014/08/31 10:33:2325333.750EBMTNoOur new baseline system after several modifications.
9ORGANIZERen-ja2014/09/16 13:36:3536731.500SMTNoHierarchical Phrase-based SMT (2014)
10SAS_MTen-ja2014/09/01 10:39:2726427.500SMTNoSyntactic reordering Hierarchical SMT (using part of data)
11Senseen-ja2014/08/25 01:07:271843.750SMTNoBaseline SMT
12ORGANIZERen-ja2014/07/21 11:38:12660.750OtherYesRBMT B (2014)

Notice:
Back to top

EVALUATION RESULTS USAGE POLICY

When you use the WAT evaluation results for any purpose such as:
- writing technical papers,
- making presentations about your system,
- advertising your MT system to the customers,
you can use the information about translation directions, scores (including both automatic and human evaluations) and ranks of your system among others. You can also use the scores of the other systems, but you MUST anonymize the other system's names. In addition, you can show the links (URLs) to the WAT evaluation result pages.

NICT (National Institute of Information and Communications Technology)
Kyoto University
Last Modified: 2018-08-02