NICT_LOGO.JPG KYOTO-U_LOGO.JPG

WAT

The Workshop on Asian Translation
Evaluation Results

[EVALUATION RESULTS TOP] | [BLEU] | [RIBES] | [AMFM] | [HUMAN (WAT2019)] | [HUMAN (WAT2018)] | [HUMAN (WAT2017)] | [HUMAN (WAT2016)] | [HUMAN (WAT2015)] | [HUMAN (WAT2014)] | [EVALUATION RESULTS USAGE POLICY]

BLEU


# Team Task Date/Time DataID BLEU
Method
Other
Resources
System
Description
juman kytea mecab moses-
tokenizer
stanford-
segmenter-
ctb
stanford-
segmenter-
pku
indic-
tokenizer
unuse myseg kmseg
1NICT-4ALTmy-en2018/09/13 15:36:402303---29.14---- 0.00 0.00OtherYesMany PBSMT and NMT n-best lists combined and reranked. Use monolingual data for back-translation and language model trainings.
2NICT-4ALTmy-en2018/09/13 14:31:162290---22.53---- 0.00 0.00OtherNoMany PBSMT and NMT n-best lists combined and reranked
3NICT-4ALTmy-en2018/08/23 10:29:462069---21.97---- 0.00 0.00NMTNoNMT baseline: ensemble
4NICTALTmy-en2018/09/14 10:13:172329---20.82---- 0.00 0.00NMTNo4 models ensemble
5NICT-4ALTmy-en2018/08/23 10:28:042068---18.98---- 0.00 0.00NMTNoNMT baseline: single system
6NICTALTmy-en2018/09/12 15:33:342281---16.31---- 0.00 0.00NMTNoSingle model
7NICT-5ALTmy-en2018/08/22 18:57:562056---15.44---- 0.00 0.00NMTNoSimple Mixed Fine Tuning model using transformer.
8ORGANIZERALTmy-en2018/09/04 18:38:582228---14.44---- 0.00 0.00NMTNoNMT with Attention
9ORGANIZERALTmy-en2018/08/24 15:29:412141---14.24---- 0.00 0.00OtherYesOnline A
10XMUNLPALTmy-en2018/09/16 08:46:402456---12.71---- 0.00 0.00NMTNosingle transformer model
11XMUNLPALTmy-en2018/09/15 16:42:122399---12.11---- 0.00 0.00NMTNosingle rnnsearch model
12Osaka-UALTmy-en2018/09/15 22:59:062438---11.38---- 0.00 0.00NMTYesrewarding model
13NICT-4ALTmy-en2018/08/23 10:39:022071---11.35---- 0.00 0.00SMTYesMSLR, with language model trained on common-crawl data.
14Osaka-UALTmy-en2018/09/16 11:56:522463--- 9.99---- 0.00 0.00NMTNomixed fine tuning
15UCSYNLPALTmy-en2018/09/14 13:22:242332--- 9.56---- 0.00 0.00NMTNoNMT with Attention
16NICT-4ALTmy-en2018/09/13 14:33:292291--- 9.47---- 0.00 0.00SMTNowith MSLR models, language models were trained on the target side of the parallel data
17UCSYNLPALTmy-en2018/09/15 15:44:562393--- 8.91---- 0.00 0.00SMTNoHPBSMT
18UCSYNLPALTmy-en2018/09/15 15:25:272391--- 8.84---- 0.00 0.00SMTNoOSM
19UCSMNLPALTmy-en2018/10/29 15:28:572549--- 6.01---- 0.00 0.00SMTNoBatch MIRA tuning
20UCSMNLPALTmy-en2018/09/14 15:32:102338--- 2.22---- 0.00 0.00SMTNowith PBSMT

Notice:

Back to top

RIBES


# Team Task Date/Time DataID RIBES
Method
Other
Resources
System
Description
juman kytea mecab moses-
tokenizer
stanford-
segmenter-
ctb
stanford-
segmenter-
pku
indic-
tokenizer
unuse myseg kmseg
1NICT-4ALTmy-en2018/09/13 15:36:402303---0.793960----0.0000000.000000OtherYesMany PBSMT and NMT n-best lists combined and reranked. Use monolingual data for back-translation and language model trainings.
2NICT-4ALTmy-en2018/09/13 14:31:162290---0.753767----0.0000000.000000OtherNoMany PBSMT and NMT n-best lists combined and reranked
3NICT-4ALTmy-en2018/08/23 10:29:462069---0.753209----0.0000000.000000NMTNoNMT baseline: ensemble
4NICTALTmy-en2018/09/14 10:13:172329---0.740819----0.0000000.000000NMTNo4 models ensemble
5NICT-4ALTmy-en2018/08/23 10:28:042068---0.740401----0.0000000.000000NMTNoNMT baseline: single system
6NICT-5ALTmy-en2018/08/22 18:57:562056---0.717430----0.0000000.000000NMTNoSimple Mixed Fine Tuning model using transformer.
7NICTALTmy-en2018/09/12 15:33:342281---0.710528----0.0000000.000000NMTNoSingle model
8ORGANIZERALTmy-en2018/09/04 18:38:582228---0.696861----0.0000000.000000NMTNoNMT with Attention
9XMUNLPALTmy-en2018/09/16 08:46:402456---0.682031----0.0000000.000000NMTNosingle transformer model
10XMUNLPALTmy-en2018/09/15 16:42:122399---0.662820----0.0000000.000000NMTNosingle rnnsearch model
11Osaka-UALTmy-en2018/09/15 22:59:062438---0.655643----0.0000000.000000NMTYesrewarding model
12Osaka-UALTmy-en2018/09/16 11:56:522463---0.648923----0.0000000.000000NMTNomixed fine tuning
13UCSYNLPALTmy-en2018/09/14 13:22:242332---0.642309----0.0000000.000000NMTNoNMT with Attention
14ORGANIZERALTmy-en2018/08/24 15:29:412141---0.598345----0.0000000.000000OtherYesOnline A
15UCSYNLPALTmy-en2018/09/15 15:44:562393---0.583956----0.0000000.000000SMTNoHPBSMT
16NICT-4ALTmy-en2018/08/23 10:39:022071---0.580091----0.0000000.000000SMTYesMSLR, with language model trained on common-crawl data.
17NICT-4ALTmy-en2018/09/13 14:33:292291---0.575931----0.0000000.000000SMTNowith MSLR models, language models were trained on the target side of the parallel data
18UCSYNLPALTmy-en2018/09/15 15:25:272391---0.553786----0.0000000.000000SMTNoOSM
19UCSMNLPALTmy-en2018/10/29 15:28:572549---0.536321----0.0000000.000000SMTNoBatch MIRA tuning
20UCSMNLPALTmy-en2018/09/14 15:32:102338---0.470280----0.0000000.000000SMTNowith PBSMT

Notice:

Back to top

AMFM


# Team Task Date/Time DataID AMFM
Method
Other
Resources
System
Description
juman kytea mecab moses-
tokenizer
stanford-
segmenter-
ctb
stanford-
segmenter-
pku
indic-
tokenizer
unuse myseg kmseg
1NICT-4ALTmy-en2018/09/13 15:36:402303---0.655910----0.0000000.000000OtherYesMany PBSMT and NMT n-best lists combined and reranked. Use monolingual data for back-translation and language model trainings.
2UCSYNLPALTmy-en2018/09/15 15:25:272391---0.594800----0.0000000.000000SMTNoOSM
3NICT-4ALTmy-en2018/08/23 10:28:042068---0.589310----0.0000000.000000NMTNoNMT baseline: single system
4NICTALTmy-en2018/09/12 15:33:342281---0.589020----0.0000000.000000NMTNoSingle model
5NICT-4ALTmy-en2018/08/23 10:29:462069---0.586770----0.0000000.000000NMTNoNMT baseline: ensemble
6NICT-4ALTmy-en2018/09/13 14:33:292291---0.584040----0.0000000.000000SMTNowith MSLR models, language models were trained on the target side of the parallel data
7NICT-4ALTmy-en2018/09/13 14:31:162290---0.582230----0.0000000.000000OtherNoMany PBSMT and NMT n-best lists combined and reranked
8NICTALTmy-en2018/09/14 10:13:172329---0.580690----0.0000000.000000NMTNo4 models ensemble
9NICT-5ALTmy-en2018/08/22 18:57:562056---0.579520----0.0000000.000000NMTNoSimple Mixed Fine Tuning model using transformer.
10ORGANIZERALTmy-en2018/08/24 15:29:412141---0.576780----0.0000000.000000OtherYesOnline A
11NICT-4ALTmy-en2018/08/23 10:39:022071---0.569370----0.0000000.000000SMTYesMSLR, with language model trained on common-crawl data.
12UCSYNLPALTmy-en2018/09/15 15:44:562393---0.560800----0.0000000.000000SMTNoHPBSMT
13UCSMNLPALTmy-en2018/10/29 15:28:572549---0.552430----0.0000000.000000SMTNoBatch MIRA tuning
14Osaka-UALTmy-en2018/09/16 11:56:522463---0.552040----0.0000000.000000NMTNomixed fine tuning
15XMUNLPALTmy-en2018/09/16 08:46:402456---0.543700----0.0000000.000000NMTNosingle transformer model
16ORGANIZERALTmy-en2018/09/04 18:38:582228---0.525950----0.0000000.000000NMTNoNMT with Attention
17UCSYNLPALTmy-en2018/09/14 13:22:242332---0.518990----0.0000000.000000NMTNoNMT with Attention
18Osaka-UALTmy-en2018/09/15 22:59:062438---0.510900----0.0000000.000000NMTYesrewarding model
19XMUNLPALTmy-en2018/09/15 16:42:122399---0.500210----0.0000000.000000NMTNosingle rnnsearch model
20UCSMNLPALTmy-en2018/09/14 15:32:102338---0.354550----0.0000000.000000SMTNowith PBSMT

Notice:

Back to top

HUMAN (WAT2019)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description

Notice:
Back to top

HUMAN (WAT2018)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1NICT-4ALTmy-en2018/08/23 10:29:46206922.250NMTNoNMT baseline: ensemble
2NICTALTmy-en2018/09/14 10:13:17232920.500NMTNo4 models ensemble
3NICT-4ALTmy-en2018/09/13 14:31:16229016.000OtherNoMany PBSMT and NMT n-best lists combined and reranked
4NICTALTmy-en2018/09/12 15:33:3422817.250NMTNoSingle model
5NICT-5ALTmy-en2018/08/22 18:57:562056-6.500NMTNoSimple Mixed Fine Tuning model using transformer.
6UCSYNLPALTmy-en2018/09/14 13:22:242332-37.500NMTNoNMT with Attention
7Osaka-UALTmy-en2018/09/15 22:59:062438-57.000NMTYesrewarding model
8Osaka-UALTmy-en2018/09/16 11:56:522463-61.000NMTNomixed fine tuning
9UCSMNLPALTmy-en2018/09/14 15:32:102338-99.500SMTNowith PBSMT

Notice:
Back to top

HUMAN (WAT2017)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description

Notice:
Back to top

HUMAN (WAT2016)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description

Notice:
Back to top

HUMAN (WAT2015)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description

Notice:
Back to top

HUMAN (WAT2014)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description

Notice:
Back to top

EVALUATION RESULTS USAGE POLICY

When you use the WAT evaluation results for any purpose such as:
- writing technical papers,
- making presentations about your system,
- advertising your MT system to the customers,
you can use the information about translation directions, scores (including both automatic and human evaluations) and ranks of your system among others. You can also use the scores of the other systems, but you MUST anonymize the other system's names. In addition, you can show the links (URLs) to the WAT evaluation result pages.

NICT (National Institute of Information and Communications Technology)
Kyoto University
Last Modified: 2018-08-02