NICT_LOGO.JPG KYOTO-U_LOGO.JPG

WAT

The Workshop on Asian Translation
Evaluation Results

[EVALUATION RESULTS TOP] | [BLEU] | [RIBES] | [AMFM] | [HUMAN (WAT2019)] | [HUMAN (WAT2018)] | [HUMAN (WAT2017)] | [HUMAN (WAT2016)] | [HUMAN (WAT2015)] | [HUMAN (WAT2014)] | [EVALUATION RESULTS USAGE POLICY]

BLEU


# Team Task Date/Time DataID BLEU
Method
Other
Resources
System
Description
juman kytea mecab moses-
tokenizer
stanford-
segmenter-
ctb
stanford-
segmenter-
pku
indic-
tokenizer
unuse myseg kmseg
1JAPIOJPCzh-ja2016/08/17 11:48:56116158.6659.1958.63---- 0.00 0.00 0.00SMTYesPhrase-based SMT with Preordering + JAPIO corpus including some sentences in testset
2KNU_HyundaiJPCzh-ja2019/07/27 08:29:23315353.5953.9153.47-------NMTYesTransformer(base) + *Used ASPEC corpus* with relative position, bt, multi source, r2l rerank, 5-model ensemble
3sarahJPCzh-ja2019/07/26 11:32:28297650.9052.0451.09-------NMTNoTransformer, ensemble of 4 models
4JAPIOJPCzh-ja2017/07/25 12:22:07144750.5251.2550.57---- 0.00 0.00 0.00SMTYesPhrase-based SMT with Preordering + Japio corpus
5JAPIOJPCzh-ja2017/07/26 14:21:18148450.0650.5150.00---- 0.00 0.00 0.00NMTYesCombination of 3 NMT systems (OpenNMT(dbrnn) + JPC/Japio corpora + NMT/rule-based posteditor)
6ryanJPCzh-ja2019/07/25 22:12:26295449.9850.6750.12-------NMTNoBase Transformer
7JAPIOJPCzh-ja2017/07/26 14:09:22148249.5150.0049.48---- 0.00 0.00 0.00NMTYesOpenNMT(dbrnn) + JPC/Japio corpora + NMT/rule-based posteditor
8USTCJPCzh-ja2018/08/31 17:24:35220648.3749.7848.57----- 0.00 0.00NMTNotensor2tensor, 4 model average, r2l rerank
9EHRJPCzh-ja2018/08/31 18:51:15221048.1048.5147.96----- 0.00 0.00NMTNoSMT reranked NMT
10EHRJPCzh-ja2017/07/19 19:28:31140847.0847.4446.83---- 0.00 0.00 0.00NMTNoSMT reranked NMT (word based, by Moses and OpenNMT)
11EHRJPCzh-ja2017/07/19 20:41:27141446.5247.1746.35---- 0.00 0.00 0.00NMTNoSMT reranked NMT (character based, by Moses and OpenNMT)
12ORGANIZERJPCzh-ja2018/08/15 18:29:31196346.3246.7346.11----- 0.00 0.00NMTNoNMT with Attention
13EHRJPCzh-ja2017/07/19 20:45:00141546.0346.4245.95---- 0.00 0.00 0.00NMTNoSimple NMT (word based, by OpenNMT)
14EHRJPCzh-ja2017/07/19 19:35:03140945.2745.8745.24---- 0.00 0.00 0.00NMTNoSimple NMT (character based, by OpenNMT)
15JAPIOJPCzh-ja2017/07/25 18:26:52145845.0745.7945.10---- 0.00 0.00 0.00NMTNoOpenNMT(dbrnn)
16NTTJPCzh-ja2016/08/19 08:53:34119944.9945.8445.02---- 0.00 0.00 0.00NMTNoBaseline NMT with attention over bidirectional LSTMs (by Harvard NMT)
17JAPIOJPCzh-ja2016/08/19 08:26:57119244.3245.1244.09---- 0.00 0.00 0.00SMTYesPhrase-based SMT with Preordering + JAPIO corpus
18JAPIOJPCzh-ja2016/08/18 14:15:46118043.8744.4743.66---- 0.00 0.00 0.00SMTYesPhrase-based SMT with Preordering + JAPIO corpus
19NTTJPCzh-ja2016/08/19 08:55:20120043.4744.2743.53---- 0.00 0.00 0.00NMTNoNMT with pre-ordering and attention over bidirectional LSTMs (pre-ordering module is the same as the PBMT submission)
20ORGANIZERJPCzh-ja2016/11/16 11:19:58134142.6643.7642.95---- 0.00 0.00 0.00NMTYesOnline A (2016/11/14)
21NICT-2JPCzh-ja2016/08/05 18:06:47110041.8742.3942.13---- 0.00 0.00 0.00SMTYesPhrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
22TOSHIBAJPCzh-ja2015/07/23 14:43:3050441.8241.9041.60--- 0.00 0.00 0.00 0.00SMT and RBMTYesCombination of phrase-based SMT and SPE systems.
23Kyoto-UJPCzh-ja2015/09/02 09:25:0486441.3541.9241.16--- 0.00 0.00 0.00 0.00EBMTNoKyotoEBMT system with bilingual RNNLM reranking (only character-base model)
24TOSHIBAJPCzh-ja2015/07/28 16:30:4152641.1240.8740.59--- 0.00 0.00 0.00 0.00SMT and RBMTYesRBMT with SPE(Statistical Post Editing) system
25NICT-2JPCzh-ja2016/08/04 17:34:38107941.0941.2741.24---- 0.00 0.00 0.00SMTNoPhrase-based SMT with Preordering + Domain Adaptation
26EHRJPCzh-ja2015/08/17 14:05:2067141.0642.2441.15--- 0.00 0.00 0.00 0.00SMT and RBMTYesSystem combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
27EHRJPCzh-ja2016/07/18 15:33:03100941.0541.0540.52---- 0.00 0.00 0.00SMT and RBMTYesCombination of word-based PBSMT, character-based PBSMT and RBMT+PBSPE with DL=6.
28EHRJPCzh-ja2016/07/18 15:25:53100740.9541.2040.51---- 0.00 0.00 0.00SMTYesCombination of word-based PBSMT and character-based PBSMT with DL=6.
29NTTJPCzh-ja2016/08/19 08:28:00119340.7541.0540.68---- 0.00 0.00 0.00SMTNoPBMT with pre-ordering on dependency structures
30EHRJPCzh-ja2015/08/30 15:22:2583040.7041.4940.79--- 0.00 0.00 0.00 0.00SMTYesPhrase based SMT with preordering
31NTTJPCzh-ja2015/08/21 08:07:1873640.6041.1040.63--- 0.00 0.00 0.00 0.00SMTNoA pre-ordering-based PBMT with patent-tuned dependency parsing and phrase table smoothing.
32EHRJPCzh-ja2015/08/30 12:42:5282840.3540.1639.92--- 0.00 0.00 0.00 0.00SMT and RBMTYesRBMT with user dictionary plus SPE
33NTTJPCzh-ja2015/08/28 09:53:2481139.7740.0839.88--- 0.00 0.00 0.00 0.00SMTNoA pre-ordering-based PBMT with patent-tuned dependency parsing, learning-based pre-ordering, and phrase table smoothing.
34ORGANIZERJPCzh-ja2015/05/14 18:00:1643239.3939.9039.39--- 0.00 0.00 0.00 0.00SMTNoTree-to-String SMT (2015)
35bjtu_nlpJPCzh-ja2016/08/09 18:44:56112839.3439.7239.30---- 0.00 0.00 0.00NMTNoRNN Encoder-Decoder with attention mechanism, single model
36JAPIOJPCzh-ja2016/10/27 13:01:42132939.2940.6739.51---- 0.00 0.00 0.00SMTNoPhrase-based SMT with Preordering
37ORGANIZERJPCzh-ja2015/05/14 17:55:5143039.2239.5239.14--- 0.00 0.00 0.00 0.00SMTNoHierarchical Phrase-based SMT
38ORGANIZERJPCzh-ja2016/07/15 11:22:3599839.0739.4538.95---- 0.00 0.00 0.00SMTNoTree-to-String SMT (2016)
39NTTJPCzh-ja2016/08/19 08:26:18119139.0339.1738.99---- 0.00 0.00 0.00SMTNoBaseline PBMT (Moses)
40SenseJPCzh-ja2016/08/29 01:06:19128138.9038.5838.65---- 0.00 0.00 0.00SMTNoClustercat-C10-PBMT
41u-tkbJPCzh-ja2017/07/26 12:44:18146838.7940.4738.99---- 0.00 0.00 0.00NMTNoNMT with SMT phrase translation (phrase extraction with branching entropy; attention over bidirectional LSTMs; by Harvard NMT)
42SenseJPCzh-ja2016/08/29 09:55:33128438.7538.3238.38---- 0.00 0.00 0.00SMTNoBaseline-C10-PBMT
43SenseJPCzh-ja2016/08/29 23:08:29129238.7138.3538.38---- 0.00 0.00 0.00SMTNoBaseline-C50-PBMT
44SenseJPCzh-ja2016/08/30 07:37:39129438.7138.5138.54---- 0.00 0.00 0.00SMTNoClustercat-C50-PBMT
45ORGANIZERJPCzh-ja2015/05/14 17:58:1443138.3438.5138.22--- 0.00 0.00 0.00 0.00SMTNoPhrase-based SMT
46Kyoto-UJPCzh-ja2015/08/26 13:10:4478137.8738.6237.71--- 0.00 0.00 0.00 0.00EBMTNoBaseline w/o reranking
47WASUIPSJPCzh-ja2015/09/01 14:16:1685333.4834.5533.55--- 0.00 0.00 0.00 0.00SMTNoCombining sampling-based alignment and bilingual hierarchical sub-sentential alignment methods.
48WASUIPSJPCzh-ja2016/10/12 21:06:36132631.0031.6330.86---- 0.00 0.00 0.00SMTNoOur improved system: train=100,000 tune=1,000 test=2,000 BLEU (our PC): 33.61. Using bilingual term extraction and re-tokenization for Chinese–Japanese.
49WASUIPSJPCzh-ja2016/10/12 21:04:52132529.3830.8229.66---- 0.00 0.00 0.00SMTNoOur baseline system: train=100,000 tune=1,000 test=2,000 BLEU (our PC): 32.29.
50TOSHIBAJPCzh-ja2015/08/17 11:53:3466728.0627.4427.56--- 0.00 0.00 0.00 0.00RBMTYesRBMT
51ORGANIZERJPCzh-ja2016/07/26 11:18:45104026.9927.9127.02---- 0.00 0.00 0.00OtherYesOnline A (2016)
52ORGANIZERJPCzh-ja2015/08/14 16:52:0264726.8027.8126.89--- 0.00 0.00 0.00 0.00OtherYesOnline A (2015)
53EHRJPCzh-ja2018/05/04 14:17:39180315.7716.1215.53---- 0.00 0.00 0.00RBMTYesRBMT system for WAT2015's submission
54ORGANIZERJPCzh-ja2015/08/14 16:55:1964812.3312.7212.44--- 0.00 0.00 0.00 0.00OtherYesOnline B (2015)
55ORGANIZERJPCzh-ja2015/08/25 11:42:0275910.4910.7210.35--- 0.00 0.00 0.00 0.00RBMTNoRBMT A (2015)
56ORGANIZERJPCzh-ja2015/08/25 11:53:50760 7.94 8.07 7.73--- 0.00 0.00 0.00 0.00RBMTNoRBMT B

Notice:

Back to top

RIBES


# Team Task Date/Time DataID RIBES
Method
Other
Resources
System
Description
juman kytea mecab moses-
tokenizer
stanford-
segmenter-
ctb
stanford-
segmenter-
pku
indic-
tokenizer
unuse myseg kmseg
1KNU_HyundaiJPCzh-ja2019/07/27 08:29:2331530.8802510.8794520.879471-------NMTYesTransformer(base) + *Used ASPEC corpus* with relative position, bt, multi source, r2l rerank, 5-model ensemble
2JAPIOJPCzh-ja2017/07/26 14:21:1814840.8753980.8733900.874822----0.0000000.0000000.000000NMTYesCombination of 3 NMT systems (OpenNMT(dbrnn) + JPC/Japio corpora + NMT/rule-based posteditor)
3JAPIOJPCzh-ja2017/07/26 14:09:2214820.8726250.8705370.872038----0.0000000.0000000.000000NMTYesOpenNMT(dbrnn) + JPC/Japio corpora + NMT/rule-based posteditor
4ryanJPCzh-ja2019/07/25 22:12:2629540.8698900.8679190.868936-------NMTNoBase Transformer
5sarahJPCzh-ja2019/07/26 11:32:2829760.8691590.8671330.868496-------NMTNoTransformer, ensemble of 4 models
6JAPIOJPCzh-ja2016/08/17 11:48:5611610.8680270.8648930.866692----0.0000000.0000000.000000SMTYesPhrase-based SMT with Preordering + JAPIO corpus including some sentences in testset
7USTCJPCzh-ja2018/08/31 17:24:3522060.8662320.8642840.865423-----0.0000000.000000NMTNotensor2tensor, 4 model average, r2l rerank
8JAPIOJPCzh-ja2017/07/25 18:26:5214580.8598830.8570560.859411----0.0000000.0000000.000000NMTNoOpenNMT(dbrnn)
9EHRJPCzh-ja2017/07/19 20:41:2714140.8596190.8567840.858353----0.0000000.0000000.000000NMTNoSMT reranked NMT (character based, by Moses and OpenNMT)
10EHRJPCzh-ja2017/07/19 19:28:3114080.8590700.8563760.858888----0.0000000.0000000.000000NMTNoSMT reranked NMT (word based, by Moses and OpenNMT)
11EHRJPCzh-ja2017/07/19 20:45:0014150.8585910.8559170.858511----0.0000000.0000000.000000NMTNoSimple NMT (word based, by OpenNMT)
12EHRJPCzh-ja2018/08/31 18:51:1522100.8582590.8556490.858142-----0.0000000.000000NMTNoSMT reranked NMT
13ORGANIZERJPCzh-ja2018/08/15 18:29:3119630.8573180.8550850.856442-----0.0000000.000000NMTNoNMT with Attention
14EHRJPCzh-ja2017/07/19 19:35:0314090.8544470.8526150.853226----0.0000000.0000000.000000NMTNoSimple NMT (character based, by OpenNMT)
15NTTJPCzh-ja2016/08/19 08:53:3411990.8530040.8518590.852430----0.0000000.0000000.000000NMTNoBaseline NMT with attention over bidirectional LSTMs (by Harvard NMT)
16JAPIOJPCzh-ja2017/07/25 12:22:0714470.8477930.8437740.846081----0.0000000.0000000.000000SMTYesPhrase-based SMT with Preordering + Japio corpus
17ORGANIZERJPCzh-ja2016/11/16 11:19:5813410.8458580.8449180.845794----0.0000000.0000000.000000NMTYesOnline A (2016/11/14)
18NTTJPCzh-ja2016/08/19 08:55:2012000.8452710.8431050.844968----0.0000000.0000000.000000NMTNoNMT with pre-ordering and attention over bidirectional LSTMs (pre-ordering module is the same as the PBMT submission)
19bjtu_nlpJPCzh-ja2016/08/09 18:44:5611280.8353140.8305050.833216----0.0000000.0000000.000000NMTNoRNN Encoder-Decoder with attention mechanism, single model
20JAPIOJPCzh-ja2016/08/19 08:26:5711920.8349590.8301640.832955----0.0000000.0000000.000000SMTYesPhrase-based SMT with Preordering + JAPIO corpus
21JAPIOJPCzh-ja2016/08/18 14:15:4611800.8335860.8293600.831534----0.0000000.0000000.000000SMTYesPhrase-based SMT with Preordering + JAPIO corpus
22u-tkbJPCzh-ja2017/07/26 12:44:1814680.8321440.8336100.831209----0.0000000.0000000.000000NMTNoNMT with SMT phrase translation (phrase extraction with branching entropy; attention over bidirectional LSTMs; by Harvard NMT)
23NICT-2JPCzh-ja2016/08/05 18:06:4711000.8296400.8267440.828107----0.0000000.0000000.000000SMTYesPhrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
24Kyoto-UJPCzh-ja2015/09/02 09:25:048640.8285430.8241990.827230---0.0000000.0000000.0000000.000000EBMTNoKyotoEBMT system with bilingual RNNLM reranking (only character-base model)
25EHRJPCzh-ja2016/07/18 15:25:5310070.8280400.8245020.826864----0.0000000.0000000.000000SMTYesCombination of word-based PBSMT and character-based PBSMT with DL=6.
26EHRJPCzh-ja2016/07/18 15:33:0310090.8270480.8219400.824852----0.0000000.0000000.000000SMT and RBMTYesCombination of word-based PBSMT, character-based PBSMT and RBMT+PBSPE with DL=6.
27NICT-2JPCzh-ja2016/08/04 17:34:3810790.8270090.8226640.825323----0.0000000.0000000.000000SMTNoPhrase-based SMT with Preordering + Domain Adaptation
28EHRJPCzh-ja2015/08/17 14:05:206710.8269870.8219830.825056---0.0000000.0000000.0000000.000000SMT and RBMTYesSystem combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
29NTTJPCzh-ja2016/08/19 08:28:0011930.8259850.8221250.824840----0.0000000.0000000.000000SMTNoPBMT with pre-ordering on dependency structures
30EHRJPCzh-ja2015/08/30 15:22:258300.8242640.8210550.823192---0.0000000.0000000.0000000.000000SMTYesPhrase based SMT with preordering
31NTTJPCzh-ja2015/08/21 08:07:187360.8234360.8202520.822026---0.0000000.0000000.0000000.000000SMTNoA pre-ordering-based PBMT with patent-tuned dependency parsing and phrase table smoothing.
32TOSHIBAJPCzh-ja2015/07/28 16:30:415260.8222680.8142490.818981---0.0000000.0000000.0000000.000000SMT and RBMTYesRBMT with SPE(Statistical Post Editing) system
33TOSHIBAJPCzh-ja2015/07/23 14:43:305040.8205680.8135360.817614---0.0000000.0000000.0000000.000000SMT and RBMTYesCombination of phrase-based SMT and SPE systems.
34JAPIOJPCzh-ja2016/10/27 13:01:4213290.8203390.8173520.819850----0.0000000.0000000.000000SMTNoPhrase-based SMT with Preordering
35EHRJPCzh-ja2015/08/30 12:42:528280.8195160.8129820.816743---0.0000000.0000000.0000000.000000SMT and RBMTYesRBMT with user dictionary plus SPE
36NTTJPCzh-ja2015/08/28 09:53:248110.8162880.8119110.815543---0.0000000.0000000.0000000.000000SMTNoA pre-ordering-based PBMT with patent-tuned dependency parsing, learning-based pre-ordering, and phrase table smoothing.
37ORGANIZERJPCzh-ja2015/05/14 18:00:164320.8149190.8113500.813595---0.0000000.0000000.0000000.000000SMTNoTree-to-String SMT (2015)
38ORGANIZERJPCzh-ja2016/07/15 11:22:359980.8131350.8098930.811644----0.0000000.0000000.000000SMTNoTree-to-String SMT (2016)
39ORGANIZERJPCzh-ja2015/05/14 17:55:514300.8060580.8020590.804523---0.0000000.0000000.0000000.000000SMTNoHierarchical Phrase-based SMT
40NTTJPCzh-ja2016/08/19 08:26:1811910.8057020.7979910.802998----0.0000000.0000000.000000SMTNoBaseline PBMT (Moses)
41SenseJPCzh-ja2016/08/29 09:55:3312840.8046730.7954490.801496----0.0000000.0000000.000000SMTNoBaseline-C10-PBMT
42SenseJPCzh-ja2016/08/30 07:37:3912940.8043010.7963490.801596----0.0000000.0000000.000000SMTNoClustercat-C50-PBMT
43SenseJPCzh-ja2016/08/29 01:06:1912810.8031550.7946790.800689----0.0000000.0000000.000000SMTNoClustercat-C10-PBMT
44SenseJPCzh-ja2016/08/29 23:08:2912920.8026730.7941270.799531----0.0000000.0000000.000000SMTNoBaseline-C50-PBMT
45Kyoto-UJPCzh-ja2015/08/26 13:10:447810.7997300.7977000.798979---0.0000000.0000000.0000000.000000EBMTNoBaseline w/o reranking
46ORGANIZERJPCzh-ja2015/05/14 17:58:144310.7820190.7789210.781456---0.0000000.0000000.0000000.000000SMTNoPhrase-based SMT
47WASUIPSJPCzh-ja2015/09/01 14:16:168530.7739850.7710990.772202---0.0000000.0000000.0000000.000000SMTNoCombining sampling-based alignment and bilingual hierarchical sub-sentential alignment methods.
48TOSHIBAJPCzh-ja2015/08/17 11:53:346670.7720540.7587560.767076---0.0000000.0000000.0000000.000000RBMTYesRBMT
49WASUIPSJPCzh-ja2016/10/12 21:06:3613260.7546260.7518040.753376----0.0000000.0000000.000000SMTNoOur improved system: train=100,000 tune=1,000 test=2,000 BLEU (our PC): 33.61. Using bilingual term extraction and re-tokenization for Chinese–Japanese.
50WASUIPSJPCzh-ja2016/10/12 21:04:5213250.7518470.7484740.750678----0.0000000.0000000.000000SMTNoOur baseline system: train=100,000 tune=1,000 test=2,000 BLEU (our PC): 32.29.
51EHRJPCzh-ja2018/05/04 14:17:3918030.7217150.7100850.716303----0.0000000.0000000.000000RBMTYesRBMT system for WAT2015's submission
52ORGANIZERJPCzh-ja2015/08/14 16:52:026470.7122420.7072640.711273---0.0000000.0000000.0000000.000000OtherYesOnline A (2015)
53ORGANIZERJPCzh-ja2016/07/26 11:18:4510400.7077390.7027180.706707----0.0000000.0000000.000000OtherYesOnline A (2016)
54ORGANIZERJPCzh-ja2015/08/25 11:42:027590.6740600.6640980.667349---0.0000000.0000000.0000000.000000RBMTNoRBMT A (2015)
55ORGANIZERJPCzh-ja2015/08/14 16:55:196480.6489960.6412550.648742---0.0000000.0000000.0000000.000000OtherYesOnline B (2015)
56ORGANIZERJPCzh-ja2015/08/25 11:53:507600.5962000.5818370.586941---0.0000000.0000000.0000000.000000RBMTNoRBMT B

Notice:

Back to top

AMFM


# Team Task Date/Time DataID AMFM
Method
Other
Resources
System
Description
juman kytea mecab moses-
tokenizer
stanford-
segmenter-
ctb
stanford-
segmenter-
pku
indic-
tokenizer
unuse myseg kmseg
1JAPIOJPCzh-ja2016/08/17 11:48:5611610.8080900.8080900.808090----0.0000000.0000000.000000SMTYesPhrase-based SMT with Preordering + JAPIO corpus including some sentences in testset
2JAPIOJPCzh-ja2017/07/26 14:21:1814840.7794200.7794200.779420----0.0000000.0000000.000000NMTYesCombination of 3 NMT systems (OpenNMT(dbrnn) + JPC/Japio corpora + NMT/rule-based posteditor)
3JAPIOJPCzh-ja2017/07/26 14:09:2214820.7774600.7774600.777460----0.0000000.0000000.000000NMTYesOpenNMT(dbrnn) + JPC/Japio corpora + NMT/rule-based posteditor
4JAPIOJPCzh-ja2017/07/25 12:22:0714470.7746600.7746600.774660----0.0000000.0000000.000000SMTYesPhrase-based SMT with Preordering + Japio corpus
5USTCJPCzh-ja2018/08/31 17:24:3522060.7713100.7713100.771310-----0.0000000.000000NMTNotensor2tensor, 4 model average, r2l rerank
6EHRJPCzh-ja2018/08/31 18:51:1522100.7646700.7646700.764670-----0.0000000.000000NMTNoSMT reranked NMT
7ORGANIZERJPCzh-ja2018/08/15 18:29:3119630.7618200.7618200.761820-----0.0000000.000000NMTNoNMT with Attention
8EHRJPCzh-ja2017/07/19 20:41:2714140.7613700.7613700.761370----0.0000000.0000000.000000NMTNoSMT reranked NMT (character based, by Moses and OpenNMT)
9EHRJPCzh-ja2017/07/19 19:35:0314090.7571300.7571300.757130----0.0000000.0000000.000000NMTNoSimple NMT (character based, by OpenNMT)
10EHRJPCzh-ja2017/07/19 19:28:3114080.7563500.7563500.756350----0.0000000.0000000.000000NMTNoSMT reranked NMT (word based, by Moses and OpenNMT)
11EHRJPCzh-ja2017/07/19 20:45:0014150.7559000.7559000.755900----0.0000000.0000000.000000NMTNoSimple NMT (word based, by OpenNMT)
12JAPIOJPCzh-ja2017/07/25 18:26:5214580.7549700.7549700.754970----0.0000000.0000000.000000NMTNoOpenNMT(dbrnn)
13NTTJPCzh-ja2016/08/19 08:53:3411990.7522000.7522000.752200----0.0000000.0000000.000000NMTNoBaseline NMT with attention over bidirectional LSTMs (by Harvard NMT)
14JAPIOJPCzh-ja2016/08/19 08:26:5711920.7512000.7512000.751200----0.0000000.0000000.000000SMTYesPhrase-based SMT with Preordering + JAPIO corpus
15NTTJPCzh-ja2016/08/19 08:55:2012000.7492700.7492700.749270----0.0000000.0000000.000000NMTNoNMT with pre-ordering and attention over bidirectional LSTMs (pre-ordering module is the same as the PBMT submission)
16JAPIOJPCzh-ja2016/08/18 14:15:4611800.7483300.7483300.748330----0.0000000.0000000.000000SMTYesPhrase-based SMT with Preordering + JAPIO corpus
17ORGANIZERJPCzh-ja2016/11/16 11:19:5813410.7472400.7472400.747240----0.0000000.0000000.000000NMTYesOnline A (2016/11/14)
18EHRJPCzh-ja2016/07/18 15:25:5310070.7450800.7450800.745080----0.0000000.0000000.000000SMTYesCombination of word-based PBSMT and character-based PBSMT with DL=6.
19Kyoto-UJPCzh-ja2015/09/02 09:25:048640.7441900.7441900.7441900.0000000.0000000.0000000.0000000.0000000.0000000.000000EBMTNoKyotoEBMT system with bilingual RNNLM reranking (only character-base model)
20TOSHIBAJPCzh-ja2015/07/28 16:30:415260.7419900.7419900.7419900.0000000.0000000.0000000.0000000.0000000.0000000.000000SMT and RBMTYesRBMT with SPE(Statistical Post Editing) system
21TOSHIBAJPCzh-ja2015/07/23 14:43:305040.7401800.7401800.7401800.0000000.0000000.0000000.0000000.0000000.0000000.000000SMT and RBMTYesCombination of phrase-based SMT and SPE systems.
22NICT-2JPCzh-ja2016/08/05 18:06:4711000.7398900.7398900.739890----0.0000000.0000000.000000SMTYesPhrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
23EHRJPCzh-ja2016/07/18 15:33:0310090.7350100.7350100.735010----0.0000000.0000000.000000SMT and RBMTYesCombination of word-based PBSMT, character-based PBSMT and RBMT+PBSPE with DL=6.
24JAPIOJPCzh-ja2016/10/27 13:01:4213290.7333000.7333000.733300----0.0000000.0000000.000000SMTNoPhrase-based SMT with Preordering
25NICT-2JPCzh-ja2016/08/04 17:34:3810790.7330200.7330200.733020----0.0000000.0000000.000000SMTNoPhrase-based SMT with Preordering + Domain Adaptation
26NTTJPCzh-ja2015/08/21 08:07:187360.7324500.7324500.7324500.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoA pre-ordering-based PBMT with patent-tuned dependency parsing and phrase table smoothing.
27Kyoto-UJPCzh-ja2015/08/26 13:10:447810.7314200.7314200.7314200.0000000.0000000.0000000.0000000.0000000.0000000.000000EBMTNoBaseline w/o reranking
28NTTJPCzh-ja2016/08/19 08:28:0011930.7301900.7301900.730190----0.0000000.0000000.000000SMTNoPBMT with pre-ordering on dependency structures
29u-tkbJPCzh-ja2017/07/26 12:44:1814680.7295800.7295800.729580----0.0000000.0000000.000000NMTNoNMT with SMT phrase translation (phrase extraction with branching entropy; attention over bidirectional LSTMs; by Harvard NMT)
30ORGANIZERJPCzh-ja2015/05/14 17:55:514300.7293700.7293700.7293700.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoHierarchical Phrase-based SMT
31ORGANIZERJPCzh-ja2016/07/15 11:22:359980.7285200.7285200.728520----0.0000000.0000000.000000SMTNoTree-to-String SMT (2016)
32ORGANIZERJPCzh-ja2015/05/14 18:00:164320.7259200.7259200.7259200.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoTree-to-String SMT (2015)
33NTTJPCzh-ja2015/08/28 09:53:248110.7232000.7232000.7232000.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoA pre-ordering-based PBMT with patent-tuned dependency parsing, learning-based pre-ordering, and phrase table smoothing.
34ORGANIZERJPCzh-ja2015/05/14 17:58:144310.7231100.7231100.7231100.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoPhrase-based SMT
35bjtu_nlpJPCzh-ja2016/08/09 18:44:5611280.7214600.7214600.721460----0.0000000.0000000.000000NMTNoRNN Encoder-Decoder with attention mechanism, single model
36EHRJPCzh-ja2015/08/17 14:05:206710.7214000.7214000.7214000.0000000.0000000.0000000.0000000.0000000.0000000.000000SMT and RBMTYesSystem combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
37NTTJPCzh-ja2016/08/19 08:26:1811910.7202600.7202600.720260----0.0000000.0000000.000000SMTNoBaseline PBMT (Moses)
38SenseJPCzh-ja2016/08/29 23:08:2912920.7193300.7193300.719330----0.0000000.0000000.000000SMTNoBaseline-C50-PBMT
39SenseJPCzh-ja2016/08/29 01:06:1912810.7185900.7185900.718590----0.0000000.0000000.000000SMTNoClustercat-C10-PBMT
40SenseJPCzh-ja2016/08/29 09:55:3312840.7154500.7154500.715450----0.0000000.0000000.000000SMTNoBaseline-C10-PBMT
41SenseJPCzh-ja2016/08/30 07:37:3912940.7152900.7152900.715290----0.0000000.0000000.000000SMTNoClustercat-C50-PBMT
42WASUIPSJPCzh-ja2015/09/01 14:16:168530.7097000.7097000.7097000.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoCombining sampling-based alignment and bilingual hierarchical sub-sentential alignment methods.
43EHRJPCzh-ja2015/08/30 15:22:258300.7065500.7065500.7065500.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTYesPhrase based SMT with preordering
44EHRJPCzh-ja2015/08/30 12:42:528280.7018800.7018800.7018800.0000000.0000000.0000000.0000000.0000000.0000000.000000SMT and RBMTYesRBMT with user dictionary plus SPE
45ORGANIZERJPCzh-ja2015/08/14 16:52:026470.6938400.6938400.6938400.0000000.0000000.0000000.0000000.0000000.0000000.000000OtherYesOnline A (2015)
46ORGANIZERJPCzh-ja2016/07/26 11:18:4510400.6937200.6937200.693720----0.0000000.0000000.000000OtherYesOnline A (2016)
47WASUIPSJPCzh-ja2016/10/12 21:06:3613260.6860300.6860300.686030----0.0000000.0000000.000000SMTNoOur improved system: train=100,000 tune=1,000 test=2,000 BLEU (our PC): 33.61. Using bilingual term extraction and re-tokenization for Chinese–Japanese.
48WASUIPSJPCzh-ja2016/10/12 21:04:5213250.6791100.6791100.679110----0.0000000.0000000.000000SMTNoOur baseline system: train=100,000 tune=1,000 test=2,000 BLEU (our PC): 32.29.
49TOSHIBAJPCzh-ja2015/08/17 11:53:346670.6687800.6687800.6687800.0000000.0000000.0000000.0000000.0000000.0000000.000000RBMTYesRBMT
50EHRJPCzh-ja2018/05/04 14:17:3918030.6233000.6233000.623300----0.0000000.0000000.000000RBMTYesRBMT system for WAT2015's submission
51ORGANIZERJPCzh-ja2015/08/14 16:55:196480.5883800.5883800.5883800.0000000.0000000.0000000.0000000.0000000.0000000.000000OtherYesOnline B (2015)
52ORGANIZERJPCzh-ja2015/08/25 11:42:027590.5571300.5571300.5571300.0000000.0000000.0000000.0000000.0000000.0000000.000000RBMTNoRBMT A (2015)
53ORGANIZERJPCzh-ja2015/08/25 11:53:507600.5021000.5021000.5021000.0000000.0000000.0000000.0000000.0000000.0000000.000000RBMTNoRBMT B
54ryanJPCzh-ja2019/07/25 22:12:2629540.0000000.0000000.000000-------NMTNoBase Transformer
55sarahJPCzh-ja2019/07/26 11:32:2829760.0000000.0000000.000000-------NMTNoTransformer, ensemble of 4 models
56KNU_HyundaiJPCzh-ja2019/07/27 08:29:2331530.0000000.0000000.000000-------NMTYesTransformer(base) + *Used ASPEC corpus* with relative position, bt, multi source, r2l rerank, 5-model ensemble

Notice:

Back to top

HUMAN (WAT2019)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1sarahJPCzh-ja2019/07/26 11:32:282976UnderwayNMTNoTransformer, ensemble of 4 models
2KNU_HyundaiJPCzh-ja2019/07/27 08:29:233153UnderwayNMTYesTransformer(base) + *Used ASPEC corpus* with relative position, bt, multi source, r2l rerank, 5-model ensemble

Notice:
Back to top

HUMAN (WAT2018)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description

Notice:
Back to top

HUMAN (WAT2017)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1JAPIOJPCzh-ja2017/07/26 14:21:18148480.250NMTYesCombination of 3 NMT systems (OpenNMT(dbrnn) + JPC/Japio corpora + NMT/rule-based posteditor)
2EHRJPCzh-ja2017/07/19 20:41:27141469.750NMTNoSMT reranked NMT (character based, by Moses and OpenNMT)
3EHRJPCzh-ja2017/07/19 19:28:31140868.250NMTNoSMT reranked NMT (word based, by Moses and OpenNMT)
4JAPIOJPCzh-ja2017/07/25 12:22:07144760.500SMTYesPhrase-based SMT with Preordering + Japio corpus
5u-tkbJPCzh-ja2017/07/26 12:44:18146855.500NMTNoNMT with SMT phrase translation (phrase extraction with branching entropy; attention over bidirectional LSTMs; by Harvard NMT)

Notice:
Back to top

HUMAN (WAT2016)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1ORGANIZERJPCzh-ja2016/11/16 11:19:58134154.250NMTYesOnline A (2016/11/14)
2NTTJPCzh-ja2016/08/19 08:55:20120046.500NMTNoNMT with pre-ordering and attention over bidirectional LSTMs (pre-ordering module is the same as the PBMT submission)
3JAPIOJPCzh-ja2016/08/19 08:26:57119246.250SMTYesPhrase-based SMT with Preordering + JAPIO corpus
4JAPIOJPCzh-ja2016/08/18 14:15:46118043.500SMTYesPhrase-based SMT with Preordering + JAPIO corpus
5NICT-2JPCzh-ja2016/08/05 18:06:47110043.250SMTYesPhrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
6NTTJPCzh-ja2016/08/19 08:28:00119339.250SMTNoPBMT with pre-ordering on dependency structures
7EHRJPCzh-ja2016/07/18 15:25:53100739.000SMTYesCombination of word-based PBSMT and character-based PBSMT with DL=6.
8NICT-2JPCzh-ja2016/08/04 17:34:38107936.750SMTNoPhrase-based SMT with Preordering + Domain Adaptation
9EHRJPCzh-ja2016/07/18 15:33:03100935.500SMT and RBMTYesCombination of word-based PBSMT, character-based PBSMT and RBMT+PBSPE with DL=6.
10bjtu_nlpJPCzh-ja2016/08/09 18:44:56112832.250NMTNoRNN Encoder-Decoder with attention mechanism, single model
11ORGANIZERJPCzh-ja2016/07/26 11:18:451040-19.750OtherYesOnline A (2016)

Notice:
Back to top

HUMAN (WAT2015)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1Kyoto-UJPCzh-ja2015/09/02 09:25:0486427.500EBMTNoKyotoEBMT system with bilingual RNNLM reranking (only character-base model)
2TOSHIBAJPCzh-ja2015/07/28 16:30:4152624.250SMT and RBMTYesRBMT with SPE(Statistical Post Editing) system
3EHRJPCzh-ja2015/08/17 14:05:2067122.000SMT and RBMTYesSystem combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
4ORGANIZERJPCzh-ja2015/05/14 18:00:1643220.750SMTNoTree-to-String SMT (2015)
5NTTJPCzh-ja2015/08/21 08:07:1873616.250SMTNoA pre-ordering-based PBMT with patent-tuned dependency parsing and phrase table smoothing.
6TOSHIBAJPCzh-ja2015/07/23 14:43:3050414.500SMT and RBMTYesCombination of phrase-based SMT and SPE systems.
7Kyoto-UJPCzh-ja2015/08/26 13:10:4478114.500EBMTNoBaseline w/o reranking
8EHRJPCzh-ja2015/08/30 12:42:528288.250SMT and RBMTYesRBMT with user dictionary plus SPE
9NTTJPCzh-ja2015/08/28 09:53:248118.000SMTNoA pre-ordering-based PBMT with patent-tuned dependency parsing, learning-based pre-ordering, and phrase table smoothing.
10ORGANIZERJPCzh-ja2015/08/14 16:52:02647-7.000OtherYesOnline A (2015)
11WASUIPSJPCzh-ja2015/09/01 14:16:16853-12.000SMTNoCombining sampling-based alignment and bilingual hierarchical sub-sentential alignment methods.
12ORGANIZERJPCzh-ja2015/08/25 11:42:02759-39.250RBMTNoRBMT A (2015)

Notice:
Back to top

HUMAN (WAT2014)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description

Notice:
Back to top

EVALUATION RESULTS USAGE POLICY

When you use the WAT evaluation results for any purpose such as:
- writing technical papers,
- making presentations about your system,
- advertising your MT system to the customers,
you can use the information about translation directions, scores (including both automatic and human evaluations) and ranks of your system among others. You can also use the scores of the other systems, but you MUST anonymize the other system's names. In addition, you can show the links (URLs) to the WAT evaluation result pages.

NICT (National Institute of Information and Communications Technology)
Kyoto University
Last Modified: 2018-08-02