NICT_LOGO.JPG KYOTO-U_LOGO.JPG

WAT

The Workshop on Asian Translation
Evaluation Results

[EVALUATION RESULTS TOP] | [BLEU] | [RIBES] | [AMFM] | [HUMAN (WAT2019)] | [HUMAN (WAT2018)] | [HUMAN (WAT2017)] | [HUMAN (WAT2016)] | [HUMAN (WAT2015)] | [HUMAN (WAT2014)] | [EVALUATION RESULTS USAGE POLICY]

BLEU


# Team Task Date/Time DataID BLEU
Method
Other
Resources
System
Description
juman kytea mecab moses-
tokenizer
stanford-
segmenter-
ctb
stanford-
segmenter-
pku
indic-
tokenizer
unuse myseg kmseg
1SenseJPCko-ja2015/08/19 02:23:0171985.2485.2185.23--- 0.00 0.00 0.00 0.00SMTNo에이트->エイト
2SenseJPCko-ja2015/08/15 15:36:3465785.2385.2085.23--- 0.00 0.00 0.00 0.00SMTNoUnicode2String with devtest MERT run2
3SenseJPCko-ja2015/08/14 10:07:0164384.2684.2384.26--- 0.00 0.00 0.00 0.00SMTNoUnicode2Stringwith devtest. MERT run1
4KNU_HyundaiJPCko-ja2019/07/26 23:59:03309373.7374.5574.03-------NMTNoTransformer + Relative Position + ensemble of 11 models
5ryanJPCko-ja2019/07/23 11:57:16283873.5374.3173.78-------NMTNoBase Transformer
6sarahJPCko-ja2019/07/26 16:27:49301673.1673.9373.40-------NMTNoTransformer, ensemble of 4 models
7JAPIOJPCko-ja2017/07/25 17:12:46144873.0073.7173.23---- 0.00 0.00 0.00SMTYesWord-based PBSMT + JPC/Japio corpora + rule-based posteditor
8JAPIOJPCko-ja2017/07/25 17:18:17145073.0073.7373.25---- 0.00 0.00 0.00SMTYesCombination of word- and character-based PBSMTs
9JAPIOJPCko-ja2017/07/25 17:15:54144971.9772.9172.17---- 0.00 0.00 0.00SMTYesCharacter-based PBSMT + JPC/Japio corpora + rule-based posteditor
10SenseJPCko-ja2015/08/19 21:23:1772371.6371.7270.90--- 0.00 0.00 0.00 0.00SMTNoBytes2String
11EHRJPCko-ja2017/07/19 21:55:39141671.5272.3471.82---- 0.00 0.00 0.00NMTNoSMT reranked NMT (word based, by Moses and OpenNMT)
12EHRJPCko-ja2016/07/17 18:23:55100571.5172.3271.77---- 0.00 0.00 0.00SMTYesCombination of word-based PBSMT and character-based PBSMT with DL=0. Parentheses surrounding number in Korean sentences are deleted.
13EHRJPCko-ja2018/08/31 19:01:15221571.4572.3571.74----- 0.00 0.00NMTNoSMT reranked NMT
14ORGANIZERJPCko-ja2018/08/15 18:44:10196671.4272.2171.68----- 0.00 0.00NMTNoNMT with Attention
15naverJPCko-ja2015/08/28 17:07:1081671.3872.2771.68--- 0.00 0.00 0.00 0.00SMTNoSMT PB + NMT reranking
16EHRJPCko-ja2017/07/19 21:57:18141771.3672.2671.65---- 0.00 0.00 0.00NMTNoSMT reranked NMT (character based, by Moses and OpenNMT)
17TOSHIBAJPCko-ja2015/08/03 09:37:4956871.0171.4471.26--- 0.00 0.00 0.00 0.00SMTYesPhrase-based SMT with RNNLM reranking + post-processing
18naverJPCko-ja2015/08/11 17:38:5963070.9171.7671.18--- 0.00 0.00 0.00 0.00SMTNocombined two phrase-based systems, post-processing
19NICTJPCko-ja2015/07/24 14:00:0651370.8171.7071.11--- 0.00 0.00 0.00 0.00SMTNoour baseline + post-processing of bracket balancing
20EHRJPCko-ja2015/09/02 16:39:4686570.8171.5771.00--- 0.00 0.00 0.00 0.00SMTYesPhrase based SMT
21EHRJPCko-ja2015/07/22 12:24:0150070.6771.5270.93--- 0.00 0.00 0.00 0.00SMT and RBMTYesSystem combination of RBMT with user dictionary plus SPE and phrase based SMT. Candidate selection by language model score.
22NICTJPCko-ja2015/07/23 14:13:3850170.6271.5270.92--- 0.00 0.00 0.00 0.00SMTNoour baseline: character-based / PB SMT in MOSES (DL=0, max-phrase-len=9, no lex-reordering) / SRILM (9-gram)
23TOSHIBAJPCko-ja2015/07/31 19:15:0055470.5170.8470.71--- 0.00 0.00 0.00 0.00SMT and RBMTYesSystem combination SMT, SPE and RBMT with RNNLM + post-processing
24JAPIOJPCko-ja2016/08/19 17:10:56120970.3271.0770.52---- 0.00 0.00 0.00SMTYesPhrase-based SMT + JPC/JAPIO corpora + rule-based posteditor
25SenseJPCko-ja2015/08/31 05:00:3583370.2371.1170.51--- 0.00 0.00 0.00 0.00SMTNoBaseline with only train.ja/ko
26EHRJPCko-ja2017/07/19 21:58:48141870.2371.1470.54---- 0.00 0.00 0.00NMTNoSimple NMT (word based, by OpenNMT)
27EHRJPCko-ja2015/08/30 15:52:0783170.1370.8670.35--- 0.00 0.00 0.00 0.00SMT and RBMTYesRBMT with user dictionary plus SPE
28EHRJPCko-ja2017/07/19 22:00:25141969.4270.2869.70---- 0.00 0.00 0.00NMTNoSimple NMT (character based, by OpenNMT)
29transictktJPCko-ja2016/08/21 19:13:09126769.3069.4269.77---- 0.00 0.00 0.00SMTNoPhrase-based SMT.
30ORGANIZERJPCko-ja2015/06/10 18:44:0843869.2270.3669.73--- 0.00 0.00 0.00 0.00SMTNoPhrase-based SMT
31JAPIOJPCko-ja2016/10/28 13:37:08133169.1070.0369.34---- 0.00 0.00 0.00SMTNoPhrase-based SMT + rule-based posteditor
32TOSHIBAJPCko-ja2015/07/31 19:11:0455268.8369.2869.04--- 0.00 0.00 0.00 0.00SMT and RBMTYesPhrase-based SPE with RNNLM reranking + post-processing
33EHRJPCko-ja2016/07/17 15:54:31100468.7869.6569.06---- 0.00 0.00 0.00SMTYesComb. of PBSMT by word and character. DL=0. Parens in Ko side and not in Ja side are added to Ja training and dev sets. After decoding, parens surrounding number are deleted.
34JAPIOJPCko-ja2016/08/19 15:34:39120668.6269.4968.90---- 0.00 0.00 0.00SMTYesPhrase-based SMT + JAPIO corpus + rule-based posteditor
35ORGANIZERJPCko-ja2015/06/10 18:46:3443967.4168.6568.00--- 0.00 0.00 0.00 0.00SMTNoHierarchical Phrase-based SMT (2015)
36ORGANIZERJPCko-ja2016/07/13 01:35:3796567.4168.6568.00---- 0.00 0.00 0.00SMTNoHierarchical Phrase-based SMT (2015)
37EHRJPCko-ja2016/07/17 18:25:46100662.3364.1762.75---- 0.00 0.00 0.00SMTYesCombination of word-based PBSMT and character-based PBSMT with DL=0. Parentheses in Korean side and not in Japanese side are added to Japanese for training and dev sets.
38TOSHIBAJPCko-ja2015/08/17 11:55:2266858.6059.3458.49--- 0.00 0.00 0.00 0.00RBMTYesRBMT
39ORGANIZERJPCko-ja2015/08/14 17:30:2165255.0556.8455.46--- 0.00 0.00 0.00 0.00OtherYesOnline A (2015)
40ORGANIZERJPCko-ja2016/07/13 00:54:1096355.0556.8455.46---- 0.00 0.00 0.00OtherYesOnline A (2015)
41ORGANIZERJPCko-ja2016/07/26 11:16:02103954.7856.6855.14---- 0.00 0.00 0.00OtherYesOnline A (2016)
42SenseJPCko-ja2015/08/19 21:26:2872446.0448.3046.62--- 0.00 0.00 0.00 0.00SMTNoUnicode2String
43ORGANIZERJPCko-ja2016/11/25 11:27:15134444.4245.1444.72---- 0.00 0.00 0.00NMTNoOnline A (2016/11/25)
44ORGANIZERJPCko-ja2015/08/14 17:37:5765342.0043.9742.45--- 0.00 0.00 0.00 0.00OtherYesRBMT A (2015)
45ORGANIZERJPCko-ja2016/07/13 01:34:5696442.0043.9742.45---- 0.00 0.00 0.00OtherYesRBMT A (2015)
46ORGANIZERJPCko-ja2015/08/14 17:17:3165136.4138.7237.01--- 0.00 0.00 0.00 0.00OtherYesOnline B (2015)
47ORGANIZERJPCko-ja2015/08/14 17:40:1165434.7437.5135.54--- 0.00 0.00 0.00 0.00OtherYesRBMT B

Notice:

Back to top

RIBES


# Team Task Date/Time DataID RIBES
Method
Other
Resources
System
Description
juman kytea mecab moses-
tokenizer
stanford-
segmenter-
ctb
stanford-
segmenter-
pku
indic-
tokenizer
unuse myseg kmseg
1SenseJPCko-ja2015/08/15 15:36:346570.9545060.9544480.954435---0.0000000.0000000.0000000.000000SMTNoUnicode2String with devtest MERT run2
2SenseJPCko-ja2015/08/19 02:23:017190.9544980.9544630.954435---0.0000000.0000000.0000000.000000SMTNo에이트->エイト
3SenseJPCko-ja2015/08/14 10:07:016430.9503180.9501090.950235---0.0000000.0000000.0000000.000000SMTNoUnicode2Stringwith devtest. MERT run1
4KNU_HyundaiJPCko-ja2019/07/26 23:59:0330930.9500000.9489240.949410-------NMTNoTransformer + Relative Position + ensemble of 11 models
5ryanJPCko-ja2019/07/23 11:57:1628380.9499540.9489800.949567-------NMTNoBase Transformer
6sarahJPCko-ja2019/07/26 16:27:4930160.9495750.9485700.949128-------NMTNoTransformer, ensemble of 4 models
7JAPIOJPCko-ja2017/07/25 17:18:1714500.9469850.9458410.946745----0.0000000.0000000.000000SMTYesCombination of word- and character-based PBSMTs
8JAPIOJPCko-ja2017/07/25 17:12:4614480.9468800.9457540.946645----0.0000000.0000000.000000SMTYesWord-based PBSMT + JPC/Japio corpora + rule-based posteditor
9EHRJPCko-ja2017/07/19 21:57:1814170.9461260.9448120.945888----0.0000000.0000000.000000NMTNoSMT reranked NMT (character based, by Moses and OpenNMT)
10EHRJPCko-ja2018/08/31 19:01:1522150.9457710.9442840.945497-----0.0000000.000000NMTNoSMT reranked NMT
11ORGANIZERJPCko-ja2018/08/15 18:44:1019660.9455930.9446440.945126-----0.0000000.000000NMTNoNMT with Attention
12EHRJPCko-ja2016/07/17 18:23:5510050.9446510.9435140.944606----0.0000000.0000000.000000SMTYesCombination of word-based PBSMT and character-based PBSMT with DL=0. Parentheses surrounding number in Korean sentences are deleted.
13EHRJPCko-ja2017/07/19 21:55:3914160.9445160.9429400.944219----0.0000000.0000000.000000NMTNoSMT reranked NMT (word based, by Moses and OpenNMT)
14JAPIOJPCko-ja2017/07/25 17:15:5414490.9444350.9436180.943960----0.0000000.0000000.000000SMTYesCharacter-based PBSMT + JPC/Japio corpora + rule-based posteditor
15naverJPCko-ja2015/08/11 17:38:596300.9439280.9428000.943376---0.0000000.0000000.0000000.000000SMTNocombined two phrase-based systems, post-processing
16naverJPCko-ja2015/08/28 17:07:108160.9438140.9428050.943584---0.0000000.0000000.0000000.000000SMTNoSMT PB + NMT reranking
17TOSHIBAJPCko-ja2015/08/03 09:37:495680.9437940.9412870.943181---0.0000000.0000000.0000000.000000SMTYesPhrase-based SMT with RNNLM reranking + post-processing
18NICTJPCko-ja2015/07/24 14:00:065130.9434630.9425190.942904---0.0000000.0000000.0000000.000000SMTNoour baseline + post-processing of bracket balancing
19NICTJPCko-ja2015/07/23 14:13:385010.9433480.9424020.942748---0.0000000.0000000.0000000.000000SMTNoour baseline: character-based / PB SMT in MOSES (DL=0, max-phrase-len=9, no lex-reordering) / SRILM (9-gram)
20EHRJPCko-ja2017/07/19 21:58:4814180.9431790.9416950.942757----0.0000000.0000000.000000NMTNoSimple NMT (word based, by OpenNMT)
21EHRJPCko-ja2015/07/22 12:24:015000.9430400.9418930.942771---0.0000000.0000000.0000000.000000SMT and RBMTYesSystem combination of RBMT with user dictionary plus SPE and phrase based SMT. Candidate selection by language model score.
22EHRJPCko-ja2015/09/02 16:39:468650.9429990.9416000.942680---0.0000000.0000000.0000000.000000SMTYesPhrase based SMT
23SenseJPCko-ja2015/08/31 05:00:358330.9424150.9408870.941687---0.0000000.0000000.0000000.000000SMTNoBaseline with only train.ja/ko
24TOSHIBAJPCko-ja2015/07/31 19:15:005540.9421830.9395450.941471---0.0000000.0000000.0000000.000000SMT and RBMTYesSystem combination SMT, SPE and RBMT with RNNLM + post-processing
25JAPIOJPCko-ja2016/08/19 17:10:5612090.9421370.9405440.941746----0.0000000.0000000.000000SMTYesPhrase-based SMT + JPC/JAPIO corpora + rule-based posteditor
26EHRJPCko-ja2015/08/30 15:52:078310.9418870.9403410.941517---0.0000000.0000000.0000000.000000SMT and RBMTYesRBMT with user dictionary plus SPE
27ORGANIZERJPCko-ja2015/06/10 18:44:084380.9413020.9397290.940756---0.0000000.0000000.0000000.000000SMTNoPhrase-based SMT
28transictktJPCko-ja2016/08/21 19:13:0912670.9412450.9376300.940497----0.0000000.0000000.000000SMTNoPhrase-based SMT.
29EHRJPCko-ja2016/07/17 15:54:3110040.9410710.9401690.940682----0.0000000.0000000.000000SMTYesComb. of PBSMT by word and character. DL=0. Parens in Ko side and not in Ja side are added to Ja training and dev sets. After decoding, parens surrounding number are deleted.
30JAPIOJPCko-ja2016/10/28 13:37:0813310.9403670.9390570.939990----0.0000000.0000000.000000SMTNoPhrase-based SMT + rule-based posteditor
31TOSHIBAJPCko-ja2015/07/31 19:11:045520.9397530.9372100.939180---0.0000000.0000000.0000000.000000SMT and RBMTYesPhrase-based SPE with RNNLM reranking + post-processing
32JAPIOJPCko-ja2016/08/19 15:34:3912060.9384740.9370660.938230----0.0000000.0000000.000000SMTYesPhrase-based SMT + JAPIO corpus + rule-based posteditor
33ORGANIZERJPCko-ja2015/06/10 18:46:344390.9371620.9359030.936570---0.0000000.0000000.0000000.000000SMTNoHierarchical Phrase-based SMT (2015)
34ORGANIZERJPCko-ja2016/07/13 01:35:379650.9371620.9359030.936570----0.0000000.0000000.000000SMTNoHierarchical Phrase-based SMT (2015)
35EHRJPCko-ja2017/07/19 22:00:2514190.9364400.9356210.936136----0.0000000.0000000.000000NMTNoSimple NMT (character based, by OpenNMT)
36SenseJPCko-ja2015/08/19 21:23:177230.9328030.9312680.930872---0.0000000.0000000.0000000.000000SMTNoBytes2String
37EHRJPCko-ja2016/07/17 18:25:4610060.9270650.9272150.927017----0.0000000.0000000.000000SMTYesCombination of word-based PBSMT and character-based PBSMT with DL=0. Parentheses in Korean side and not in Japanese side are added to Japanese for training and dev sets.
38TOSHIBAJPCko-ja2015/08/17 11:55:226680.9167760.9139820.915122---0.0000000.0000000.0000000.000000RBMTYesRBMT
39ORGANIZERJPCko-ja2015/08/14 17:30:216520.9091520.9093850.908838---0.0000000.0000000.0000000.000000OtherYesOnline A (2015)
40ORGANIZERJPCko-ja2016/07/13 00:54:109630.9091520.9093850.908838----0.0000000.0000000.000000OtherYesOnline A (2015)
41ORGANIZERJPCko-ja2016/07/26 11:16:0210390.9073200.9076520.906743----0.0000000.0000000.000000OtherYesOnline A (2016)
42ORGANIZERJPCko-ja2015/08/14 17:37:576530.8763960.8737340.875146---0.0000000.0000000.0000000.000000OtherYesRBMT A (2015)
43ORGANIZERJPCko-ja2016/07/13 01:34:569640.8763960.8737340.875146----0.0000000.0000000.000000OtherYesRBMT A (2015)
44ORGANIZERJPCko-ja2016/11/25 11:27:1513440.8576420.8541580.857083----0.0000000.0000000.000000NMTNoOnline A (2016/11/25)
45ORGANIZERJPCko-ja2015/08/14 17:17:316510.8517450.8522630.851945---0.0000000.0000000.0000000.000000OtherYesOnline B (2015)
46SenseJPCko-ja2015/08/19 21:26:287240.8510410.8542980.851636---0.0000000.0000000.0000000.000000SMTNoUnicode2String
47ORGANIZERJPCko-ja2015/08/14 17:40:116540.8457120.8490140.846228---0.0000000.0000000.0000000.000000OtherYesRBMT B

Notice:

Back to top

AMFM


# Team Task Date/Time DataID AMFM
Method
Other
Resources
System
Description
juman kytea mecab moses-
tokenizer
stanford-
segmenter-
ctb
stanford-
segmenter-
pku
indic-
tokenizer
unuse myseg kmseg
1SenseJPCko-ja2015/08/19 02:23:017190.9193800.9193800.9193800.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNo에이트->エイト
2SenseJPCko-ja2015/08/15 15:36:346570.9192100.9192100.9192100.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoUnicode2String with devtest MERT run2
3SenseJPCko-ja2015/08/14 10:07:016430.9159600.9159600.9159600.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoUnicode2Stringwith devtest. MERT run1
4JAPIOJPCko-ja2017/07/25 17:18:1714500.8732000.8732000.873200----0.0000000.0000000.000000SMTYesCombination of word- and character-based PBSMTs
5JAPIOJPCko-ja2017/07/25 17:12:4614480.8725100.8725100.872510----0.0000000.0000000.000000SMTYesWord-based PBSMT + JPC/Japio corpora + rule-based posteditor
6EHRJPCko-ja2017/07/19 21:57:1814170.8711100.8711100.871110----0.0000000.0000000.000000NMTNoSMT reranked NMT (character based, by Moses and OpenNMT)
7EHRJPCko-ja2018/08/31 19:01:1522150.8695400.8695400.869540-----0.0000000.000000NMTNoSMT reranked NMT
8ORGANIZERJPCko-ja2018/08/15 18:44:1019660.8687600.8687600.868760-----0.0000000.000000NMTNoNMT with Attention
9JAPIOJPCko-ja2017/07/25 17:15:5414490.8681700.8681700.868170----0.0000000.0000000.000000SMTYesCharacter-based PBSMT + JPC/Japio corpora + rule-based posteditor
10EHRJPCko-ja2016/07/17 18:23:5510050.8663700.8663700.866370----0.0000000.0000000.000000SMTYesCombination of word-based PBSMT and character-based PBSMT with DL=0. Parentheses surrounding number in Korean sentences are deleted.
11TOSHIBAJPCko-ja2015/08/03 09:37:495680.8661300.8661300.8661300.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTYesPhrase-based SMT with RNNLM reranking + post-processing
12EHRJPCko-ja2017/07/19 21:55:3914160.8660600.8660600.866060----0.0000000.0000000.000000NMTNoSMT reranked NMT (word based, by Moses and OpenNMT)
13naverJPCko-ja2015/08/11 17:38:596300.8655500.8655500.8655500.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNocombined two phrase-based systems, post-processing
14naverJPCko-ja2015/08/28 17:07:108160.8644500.8644500.8644500.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoSMT PB + NMT reranking
15NICTJPCko-ja2015/07/24 14:00:065130.8643500.8643500.8643500.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoour baseline + post-processing of bracket balancing
16JAPIOJPCko-ja2016/08/19 17:10:5612090.8636600.8636600.863660----0.0000000.0000000.000000SMTYesPhrase-based SMT + JPC/JAPIO corpora + rule-based posteditor
17NICTJPCko-ja2015/07/23 14:13:385010.8635400.8635400.8635400.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoour baseline: character-based / PB SMT in MOSES (DL=0, max-phrase-len=9, no lex-reordering) / SRILM (9-gram)
18EHRJPCko-ja2017/07/19 21:58:4814180.8623400.8623400.862340----0.0000000.0000000.000000NMTNoSimple NMT (word based, by OpenNMT)
19TOSHIBAJPCko-ja2015/07/31 19:15:005540.8613000.8613000.8613000.0000000.0000000.0000000.0000000.0000000.0000000.000000SMT and RBMTYesSystem combination SMT, SPE and RBMT with RNNLM + post-processing
20SenseJPCko-ja2015/08/31 05:00:358330.8607400.8607400.8607400.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoBaseline with only train.ja/ko
21EHRJPCko-ja2017/07/19 22:00:2514190.8605200.8605200.860520----0.0000000.0000000.000000NMTNoSimple NMT (character based, by OpenNMT)
22JAPIOJPCko-ja2016/10/28 13:37:0813310.8597900.8597900.859790----0.0000000.0000000.000000SMTNoPhrase-based SMT + rule-based posteditor
23TOSHIBAJPCko-ja2015/07/31 19:11:045520.8592300.8592300.8592300.0000000.0000000.0000000.0000000.0000000.0000000.000000SMT and RBMTYesPhrase-based SPE with RNNLM reranking + post-processing
24JAPIOJPCko-ja2016/08/19 15:34:3912060.8581900.8581900.858190----0.0000000.0000000.000000SMTYesPhrase-based SMT + JAPIO corpus + rule-based posteditor
25transictktJPCko-ja2016/08/21 19:13:0912670.8568300.8568300.856830----0.0000000.0000000.000000SMTNoPhrase-based SMT.
26ORGANIZERJPCko-ja2015/06/10 18:44:084380.8562200.8562200.8562200.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoPhrase-based SMT
27EHRJPCko-ja2016/07/17 15:54:3110040.8516600.8516600.851660----0.0000000.0000000.000000SMTYesComb. of PBSMT by word and character. DL=0. Parens in Ko side and not in Ja side are added to Ja training and dev sets. After decoding, parens surrounding number are deleted.
28ORGANIZERJPCko-ja2016/07/13 01:35:379650.8506100.8506100.850610----0.0000000.0000000.000000SMTNoHierarchical Phrase-based SMT (2015)
29ORGANIZERJPCko-ja2015/06/10 18:46:344390.8505600.8505600.8505600.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoHierarchical Phrase-based SMT (2015)
30EHRJPCko-ja2015/07/22 12:24:015000.8296500.8296500.8296500.0000000.0000000.0000000.0000000.0000000.0000000.000000SMT and RBMTYesSystem combination of RBMT with user dictionary plus SPE and phrase based SMT. Candidate selection by language model score.
31EHRJPCko-ja2015/09/02 16:39:468650.8247500.8247500.8247500.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTYesPhrase based SMT
32EHRJPCko-ja2015/08/30 15:52:078310.8224200.8224200.8224200.0000000.0000000.0000000.0000000.0000000.0000000.000000SMT and RBMTYesRBMT with user dictionary plus SPE
33TOSHIBAJPCko-ja2015/08/17 11:55:226680.8197700.8197700.8197700.0000000.0000000.0000000.0000000.0000000.0000000.000000RBMTYesRBMT
34EHRJPCko-ja2016/07/17 18:25:4610060.8180300.8180300.818030----0.0000000.0000000.000000SMTYesCombination of word-based PBSMT and character-based PBSMT with DL=0. Parentheses in Korean side and not in Japanese side are added to Japanese for training and dev sets.
35SenseJPCko-ja2015/08/19 21:26:287240.8159900.8159900.8159900.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoUnicode2String
36ORGANIZERJPCko-ja2016/07/13 00:54:109630.8006100.8006100.800610----0.0000000.0000000.000000OtherYesOnline A (2015)
37ORGANIZERJPCko-ja2015/08/14 17:30:216520.8004600.8004600.8004600.0000000.0000000.0000000.0000000.0000000.0000000.000000OtherYesOnline A (2015)
38ORGANIZERJPCko-ja2016/07/26 11:16:0210390.7987500.7987500.798750----0.0000000.0000000.000000OtherYesOnline A (2016)
39ORGANIZERJPCko-ja2016/11/25 11:27:1513440.7838500.7838500.783850----0.0000000.0000000.000000NMTNoOnline A (2016/11/25)
40SenseJPCko-ja2015/08/19 21:23:177230.7788600.7788600.7788600.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoBytes2String
41ORGANIZERJPCko-ja2015/08/14 17:17:316510.7287500.7287500.7287500.0000000.0000000.0000000.0000000.0000000.0000000.000000OtherYesOnline B (2015)
42ORGANIZERJPCko-ja2016/07/13 01:34:569640.7127000.7127000.712700----0.0000000.0000000.000000OtherYesRBMT A (2015)
43ORGANIZERJPCko-ja2015/08/14 17:37:576530.7120200.7120200.7120200.0000000.0000000.0000000.0000000.0000000.0000000.000000OtherYesRBMT A (2015)
44ORGANIZERJPCko-ja2015/08/14 17:40:116540.6431500.6431500.6431500.0000000.0000000.0000000.0000000.0000000.0000000.000000OtherYesRBMT B
45ryanJPCko-ja2019/07/23 11:57:1628380.0000000.0000000.000000-------NMTNoBase Transformer
46sarahJPCko-ja2019/07/26 16:27:4930160.0000000.0000000.000000-------NMTNoTransformer, ensemble of 4 models
47KNU_HyundaiJPCko-ja2019/07/26 23:59:0330930.0000000.0000000.000000-------NMTNoTransformer + Relative Position + ensemble of 11 models

Notice:

Back to top

HUMAN (WAT2019)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1sarahJPCko-ja2019/07/26 16:27:493016UnderwayNMTNoTransformer, ensemble of 4 models
2KNU_HyundaiJPCko-ja2019/07/26 23:59:033093UnderwayNMTNoTransformer + Relative Position + ensemble of 11 models

Notice:
Back to top

HUMAN (WAT2018)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description

Notice:
Back to top

HUMAN (WAT2017)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1JAPIOJPCko-ja2017/07/25 17:12:46144848.750SMTYesWord-based PBSMT + JPC/Japio corpora + rule-based posteditor
2JAPIOJPCko-ja2017/07/25 17:18:17145048.500SMTYesCombination of word- and character-based PBSMTs
3EHRJPCko-ja2017/07/19 21:57:18141711.250NMTNoSMT reranked NMT (character based, by Moses and OpenNMT)
4EHRJPCko-ja2017/07/19 21:55:3914166.250NMTNoSMT reranked NMT (word based, by Moses and OpenNMT)

Notice:
Back to top

HUMAN (WAT2016)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1EHRJPCko-ja2016/07/17 18:25:46100621.750SMTYesCombination of word-based PBSMT and character-based PBSMT with DL=0. Parentheses in Korean side and not in Japanese side are added to Japanese for training and dev sets.
2JAPIOJPCko-ja2016/08/19 17:10:56120917.500SMTYesPhrase-based SMT + JPC/JAPIO corpora + rule-based posteditor
3ORGANIZERJPCko-ja2016/07/26 11:16:0210398.000OtherYesOnline A (2016)
4EHRJPCko-ja2016/07/17 18:23:551005-3.000SMTYesCombination of word-based PBSMT and character-based PBSMT with DL=0. Parentheses surrounding number in Korean sentences are deleted.
5JAPIOJPCko-ja2016/08/19 15:34:391206-9.000SMTYesPhrase-based SMT + JAPIO corpus + rule-based posteditor
6ORGANIZERJPCko-ja2016/11/25 11:27:151344-55.750NMTNoOnline A (2016/11/25)

Notice:
Back to top

HUMAN (WAT2015)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1ORGANIZERJPCko-ja2015/08/14 17:30:2165238.750OtherYesOnline A (2015)
2naverJPCko-ja2015/08/28 17:07:1081614.750SMTNoSMT PB + NMT reranking
3NICTJPCko-ja2015/07/24 14:00:0651310.500SMTNoour baseline + post-processing of bracket balancing
4EHRJPCko-ja2015/07/22 12:24:0150010.250SMT and RBMTYesSystem combination of RBMT with user dictionary plus SPE and phrase based SMT. Candidate selection by language model score.
5NICTJPCko-ja2015/07/23 14:13:385018.250SMTNoour baseline: character-based / PB SMT in MOSES (DL=0, max-phrase-len=9, no lex-reordering) / SRILM (9-gram)
6naverJPCko-ja2015/08/11 17:38:596306.750SMTNocombined two phrase-based systems, post-processing
7EHRJPCko-ja2015/08/30 15:52:078316.500SMT and RBMTYesRBMT with user dictionary plus SPE
8TOSHIBAJPCko-ja2015/08/03 09:37:495684.500SMTYesPhrase-based SMT with RNNLM reranking + post-processing
9SenseJPCko-ja2015/08/31 05:00:358333.500SMTNoBaseline with only train.ja/ko
10TOSHIBAJPCko-ja2015/07/31 19:15:005543.000SMT and RBMTYesSystem combination SMT, SPE and RBMT with RNNLM + post-processing
11ORGANIZERJPCko-ja2015/06/10 18:46:344392.750SMTNoHierarchical Phrase-based SMT (2015)
12ORGANIZERJPCko-ja2015/08/14 17:37:57653-7.250OtherYesRBMT A (2015)
13SenseJPCko-ja2015/08/15 15:36:34657-17.750SMTNoUnicode2String with devtest MERT run2

Notice:
Back to top

HUMAN (WAT2014)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description

Notice:
Back to top

EVALUATION RESULTS USAGE POLICY

When you use the WAT evaluation results for any purpose such as:
- writing technical papers,
- making presentations about your system,
- advertising your MT system to the customers,
you can use the information about translation directions, scores (including both automatic and human evaluations) and ranks of your system among others. You can also use the scores of the other systems, but you MUST anonymize the other system's names. In addition, you can show the links (URLs) to the WAT evaluation result pages.

NICT (National Institute of Information and Communications Technology)
Kyoto University
Last Modified: 2018-08-02