NICT_LOGO.JPG KYOTO-U_LOGO.JPG

WAT

The Workshop on Asian Translation
Evaluation Results

[EVALUATION RESULTS TOP] | [BLEU] | [RIBES] | [AMFM] | [HUMAN (WAT2022)] | [HUMAN (WAT2021)] | [HUMAN (WAT2020)] | [HUMAN (WAT2019)] | [HUMAN (WAT2018)] | [HUMAN (WAT2017)] | [HUMAN (WAT2016)] | [HUMAN (WAT2015)] | [HUMAN (WAT2014)] | [EVALUATION RESULTS USAGE POLICY]

BLEU


# Team Task Date/Time DataID BLEU
Method
Other
Resources
System
Description
juman kytea mecab moses-
tokenizer
stanford-
segmenter-
ctb
stanford-
segmenter-
pku
indic-
tokenizer
unuse myseg kmseg
1Kyoto-U+ECNUzh-ja2020/09/17 18:41:34381352.8053.6452.92-------NMTYesensemble 9 models: structures(LSTM, Transformer, ConvS2S, Lightconv), training data(BT, out-of-domain parallel), S2S settings(deeper transformer, deep encoder shallow decoder)
2Kyoto-U+ECNUzh-ja2020/09/18 17:38:55393352.6553.4852.80-------NMTNowithout out-of-domain parallel data; others same as DataID:3813
3srcbzh-ja2019/07/27 15:48:24321052.3753.5852.57-------NMTNoTransformer(Big) with relative position, sentence-wise smooth, deep transformer, back translation, ensemble of 7 models.
4Kyoto-U+ECNUzh-ja2020/09/10 23:58:13367750.3751.2750.60-------NMTNoback-translation by using ja monolingual data from ASPEC-JE; lightconv (pay less attention) single model without ensemble
5KNU_Hyundaizh-ja2019/07/27 10:30:04317950.0250.8450.23-------NMTNoTransformer(base) + *Used ASPEC ja-en corpus* with relative position, bt, multi source, r2l rerank, 6-model ensemble
6srcbzh-ja2019/07/25 11:37:44291749.8350.8950.00-------NMTNoTransformer (Big) with relative position, layer attention, sentence-wise smooth.
7NICT-5zh-ja2018/09/10 14:14:05226749.7950.6649.89----- 0.00 0.00NMTNoMLNMT
8NICT-5zh-ja2018/08/27 14:40:35216949.6750.4649.79----- 0.00 0.00NMTNoCombining En-Ja corpus with Zh-Ja as a multilingual model. *ADDITIONAL ASPEC CORPUS USED*
9NICT-5zh-ja2018/08/22 18:51:44205248.4348.7848.52----- 0.00 0.00NMTNoMixed fine tuning by first pretraining on En-Ja ASPEC data and then continue on the En-Ja+Zh-Ja data. Transformer.
10Kyoto-Uzh-ja2017/07/29 08:02:07157748.4348.8448.51---- 0.00 0.00 0.00NMTNoEnsemble of 5 Shared BPE 40k
11Kyoto-Uzh-ja2017/08/01 14:14:49172048.3448.7648.40---- 0.00 0.00 0.00NMTNoEnsemble of 7 shared BPE, averaged
12NICT-2zh-ja2017/07/26 14:08:45148146.8447.5147.27---- 0.00 0.00 0.00NMTNoNMT 6 Ensembles * Bi-directional Reranking
13ORGANIZERzh-ja2017/08/02 09:59:33174046.8747.3047.00---- 0.00 0.00 0.00NMTNoGoogle's "Attention Is All You Need"
14Kyoto-Uzh-ja2017/07/31 15:27:21164346.7447.7946.67---- 0.00 0.00 0.00NMTNoKW replacement without KW in the test set, BPE, 6 ensemble
15Kyoto-Uzh-ja2016/10/11 10:46:03132446.3647.0246.50---- 0.00 0.00 0.00NMTNovoc: 32k ensemble of 4 independent model + Chinese short unit
16Kyoto-Uzh-ja2016/08/20 22:50:33125646.0446.7046.05---- 0.00 0.00 0.00NMTNovoc: 30k ensemble of 3 independent model + reverse rescoring
17NICT-2zh-ja2017/07/26 13:58:44147744.2644.9044.50---- 0.00 0.00 0.00NMTNoNMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
18Kyoto-Uzh-ja2016/08/20 22:48:16125544.2945.0544.32---- 0.00 0.00 0.00NMTNosrc: 200k tgt: 50k 2-layers self-ensembling
19ORGANIZERzh-ja2018/08/14 11:33:03190243.3143.5343.34----- 0.00 0.00NMTNoNMT with Attention
20NAISTzh-ja2015/08/31 08:23:3083441.7542.9541.93--- 0.00 0.00 0.00 0.00SMTNoTravatar System with NeuralMT Reranking
21UT-KAYzh-ja2016/08/20 07:12:52122140.5041.8140.67---- 0.00 0.00 0.00NMTNoEnsemble of our NMT models with and without domain adaptation
22NAISTzh-ja2014/07/31 11:42:3112040.1141.2940.30--- 0.00 0.00 0.00 0.00SMTNoTravatar-based Forest-to-String SMT System
23NICT-2zh-ja2016/08/05 18:05:03109940.0240.4540.29---- 0.00 0.00 0.00SMTYesPhrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
24NAISTzh-ja2014/08/01 17:33:0112440.2140.8240.15--- 0.00 0.00 0.00 0.00SMTNoTravatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
25EHRzh-ja2015/09/02 17:00:1686739.4339.9839.58--- 0.00 0.00 0.00 0.00SMTNoPhrase based SMT with preordering.
26NAISTzh-ja2015/08/31 08:26:3183539.3640.5139.47--- 0.00 0.00 0.00 0.00SMTNoTravatar System Baseline
27EHRzh-ja2016/07/31 17:06:57106339.7539.8539.40---- 0.00 0.00 0.00SMTYesLM-based merging of outputs of preordered word-based PBSMT(DL=6) and preordered character-based PBSMT(DL=6).
28bjtu_nlpzh-ja2016/08/12 12:50:38113838.8339.2538.68---- 0.00 0.00 0.00NMTNoRNN Encoder-Decoder with attention mechanism, single model
29Kyoto-Uzh-ja2015/08/31 22:39:3684538.5339.4138.66--- 0.00 0.00 0.00 0.00EBMTNoKyotoEBMT system with bilingual RNNLM reranking
30EHRzh-ja2015/08/19 11:23:3672037.9038.6837.98--- 0.00 0.00 0.00 0.00SMT and RBMTYesSystem combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
31UT-KAYzh-ja2016/08/20 07:09:54122037.6339.0737.82---- 0.00 0.00 0.00NMTNoAn end-to-end NMT with 512 dimensional single-layer LSTMs, UNK replacement, and domain adaptation
32Kyoto-Uzh-ja2015/08/07 13:24:5559737.3038.2637.45--- 0.00 0.00 0.00 0.00EBMTNoUpdated JUMAN and added one reordering feature, w/ reranking
33TOSHIBAzh-ja2015/07/23 15:14:5350837.4737.4437.34--- 0.00 0.00 0.00 0.00SMT and RBMTYesSystem combination SMT and RBMT(SPE) with RNNLM language model
34SAS_MTzh-ja2014/09/01 10:38:1326337.4237.6537.07--- 0.00 0.00 0.00 0.00SMTNoSyntactic reordering Hierarchical SMT (using SAS token tool)
35Kyoto-Uzh-ja2015/07/17 09:04:2249136.7637.8236.94--- 0.00 0.00 0.00 0.00EBMTNoWAT2015 baseline with reranking
36Kyoto-Uzh-ja2016/08/07 18:28:23111036.6337.5436.70---- 0.00 0.00 0.00EBMTNoKyotoEBMT 2016 w/o reranking
37ORGANIZERzh-ja2014/07/11 20:04:101336.5237.0736.64--- 0.00 0.00 0.00 0.00SMTNoTree-to-String SMT (2014)
38ORGANIZERzh-ja2015/09/10 14:00:3387936.5237.0736.64--- 0.00 0.00 0.00 0.00SMTNoTree-to-String SMT (2015)
39Kyoto-Uzh-ja2015/08/31 22:38:2284436.3037.2236.44--- 0.00 0.00 0.00 0.00EBMTNoKyotoEBMT system without reranking
40SAS_MTzh-ja2014/08/29 15:33:0723236.5836.2236.10--- 0.00 0.00 0.00 0.00SMTNoSyntactic reordering phrase-based SMT (SAS token tool)
41Kyoto-Uzh-ja2015/07/17 09:01:4249035.6636.7135.81--- 0.00 0.00 0.00 0.00EBMTNoWAT2015 baseline
42TOSHIBAzh-ja2015/07/28 16:27:3252535.8536.0235.73--- 0.00 0.00 0.00 0.00SMT and RBMTYesRBMT with SPE(Statistical Post Editing) system
43ORGANIZERzh-ja2014/07/11 19:47:27435.4335.9135.64--- 0.00 0.00 0.00 0.00SMTNoHierarchical Phrase-based SMT (2014)
44EHRzh-ja2015/09/04 11:44:2686835.5935.5635.37--- 0.00 0.00 0.00 0.00SMT and RBMTYesRBMT with user dictionary plus SPE.
45Kyoto-Uzh-ja2014/09/01 21:33:2326834.7535.8934.83--- 0.00 0.00 0.00 0.00EBMTNoOur new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
46BJTUNLPzh-ja2015/08/25 14:55:2076934.7234.8734.79--- 0.00 0.00 0.00 0.00SMTNo
47BJTUNLPzh-ja2015/09/01 21:08:1086234.7234.8734.79--- 0.00 0.00 0.00 0.00SMTNoa dependency-to-string model for SMT
48ORGANIZERzh-ja2014/07/11 19:54:58834.6535.1634.77--- 0.00 0.00 0.00 0.00SMTNoPhrase-based SMT
49EIWAzh-ja2014/08/20 11:56:0013833.5333.7433.87--- 0.00 0.00 0.00 0.00SMT and RBMTYesRBMT with user dictionary plus SPE(statistical post editing)
50Kyoto-Uzh-ja2014/08/19 09:31:0813333.2635.0933.62--- 0.00 0.00 0.00 0.00EBMTNoUsing n-best parses and RNNLM.
51Sensezh-ja2014/08/26 15:17:4920033.6633.8633.46--- 0.00 0.00 0.00 0.00SMTNoCharacter based SMT
52Kyoto-Uzh-ja2014/08/31 23:42:4125833.5734.4333.45--- 0.00 0.00 0.00 0.00EBMTNoOur new baseline system after several modifications.
53WASUIPSzh-ja2014/09/17 12:00:4638832.6532.8132.59--- 0.00 0.00 0.00 0.00SMTYesOur baseline system + additional quasi-parallel corpus (segmentation tools: stanford-ctb and juman, moses: 2.1.1).
54WASUIPSzh-ja2014/09/17 01:05:3837532.1932.5532.54--- 0.00 0.00 0.00 0.00SMTYesOur baseline system + additional quasi-parallel corpus (segmentation tools: urheen and mecab, moses: 2.1.1).
55WASUIPSzh-ja2014/09/17 10:26:4338432.4333.3632.48--- 0.00 0.00 0.00 0.00SMTYesOur baseline system + additional quasi-parallel corpus (segmentation tools: kytea, moses: 2.1.1).
56WASUIPSzh-ja2014/09/17 11:03:4638732.5232.6932.47--- 0.00 0.00 0.00 0.00SMTNoOur baseline system (segmentation tools: stanford-ctb and juman, moses: 2.1.1).
57Kyoto-Uzh-ja2014/08/19 10:21:3713532.6833.3032.45--- 0.00 0.00 0.00 0.00EBMTNoOur baseline system.
58WASUIPSzh-ja2014/09/17 01:03:5737431.8732.2632.26--- 0.00 0.00 0.00 0.00SMTNoOur baseline system (segmentation tools: urheen and mecab, moses: 2.1.1).
59WASUIPSzh-ja2014/09/17 10:24:5038332.0833.0932.18--- 0.00 0.00 0.00 0.00SMTNoOur baseline system (segmentation tools: kytea, moses: 2.1.1).
60WASUIPSzh-ja2014/09/17 00:46:0737030.4430.9230.86--- 0.00 0.00 0.00 0.00SMTYesOur baseline system + additional quasi-parallel corpus (segmentation tools: urheen and mecab, moses: 1.0).
61Sensezh-ja2015/07/29 07:20:2053329.2930.5229.45--- 0.00 0.00 0.00 0.00SMTNoBaseline-2015
62WASUIPSzh-ja2014/09/17 00:43:3836927.6628.0928.20--- 0.00 0.00 0.00 0.00SMTNoOur baseline system (segmentation tools: urheen and mecab, moses: 1.0).
63WASUIPSzh-ja2014/09/17 10:10:4738027.8628.8928.00--- 0.00 0.00 0.00 0.00SMTYesOur baseline system + additional quasi-parallel corpus (segmentation tools: kytea, moses: 1.0).
64WASUIPSzh-ja2014/09/17 10:07:4437927.3728.2827.43--- 0.00 0.00 0.00 0.00SMTNoOur baseline system (segmentation tools: kytea, moses: 1.0).
65JAPIOzh-ja2016/08/19 16:44:49120826.2427.8726.37---- 0.00 0.00 0.00SMTYesPhrase-based SMT with Preordering + JAPIO corpus + rule-based posteditor
66TOSHIBAzh-ja2015/08/17 12:11:5266919.2419.4819.12--- 0.00 0.00 0.00 0.00RBMTYesRBMT
67ORGANIZERzh-ja2016/11/16 11:28:00134218.7520.6419.04---- 0.00 0.00 0.00NMTYesOnline A (2016/11/14)
68EIWAzh-ja2014/08/20 11:52:4513718.6918.3318.32--- 0.00 0.00 0.00 0.00RBMTYesRBMT plus user dictionary
69ORGANIZERzh-ja2014/07/18 11:09:123611.6313.2111.87--- 0.00 0.00 0.00 0.00OtherYesOnline A (2014)
70ORGANIZERzh-ja2016/07/26 11:54:14104311.5612.8711.69---- 0.00 0.00 0.00OtherYesOnline A (2016)
71ORGANIZERzh-ja2015/08/25 18:58:0877611.5312.8211.68--- 0.00 0.00 0.00 0.00OtherYesOnline A (2015)
72ORGANIZERzh-ja2014/08/28 12:10:1321510.4811.2610.47--- 0.00 0.00 0.00 0.00OtherYesOnline B (2014)
73ORGANIZERzh-ja2015/09/11 10:09:2389010.4111.0310.36--- 0.00 0.00 0.00 0.00OtherYesOnline B (2015)
74ORGANIZERzh-ja2014/08/29 18:45:03239 9.37 9.87 9.35--- 0.00 0.00 0.00 0.00RBMTNoRBMT A (2014)
75ORGANIZERzh-ja2015/09/10 14:30:56885 9.37 9.87 9.35--- 0.00 0.00 0.00 0.00OtherYesRBMT A (2015)
76ORGANIZERzh-ja2014/08/29 18:48:29242 8.39 8.70 8.30--- 0.00 0.00 0.00 0.00RBMTNoRBMT D
77TMUzh-ja2018/09/14 17:30:332343 6.21 7.02 6.27----- 0.00 0.00NMTYesUnsupervised NMT with sub-character information. Both ASPEC and JPC 4.0 data (zh-ja) were also used as monolingual data in the training.

Notice:

Back to top

RIBES


# Team Task Date/Time DataID RIBES
Method
Other
Resources
System
Description
juman kytea mecab moses-
tokenizer
stanford-
segmenter-
ctb
stanford-
segmenter-
pku
indic-
tokenizer
unuse myseg kmseg
1Kyoto-U+ECNUzh-ja2020/09/17 18:41:3438130.8970530.8944410.897199-------NMTYesensemble 9 models: structures(LSTM, Transformer, ConvS2S, Lightconv), training data(BT, out-of-domain parallel), S2S settings(deeper transformer, deep encoder shallow decoder)
2Kyoto-U+ECNUzh-ja2020/09/18 17:38:5539330.8965510.8940730.896743-------NMTNowithout out-of-domain parallel data; others same as DataID:3813
3srcbzh-ja2019/07/27 15:48:2432100.8952310.8918720.895663-------NMTNoTransformer(Big) with relative position, sentence-wise smooth, deep transformer, back translation, ensemble of 7 models.
4NICT-5zh-ja2018/09/10 14:14:0522670.8896740.8864900.889853-----0.0000000.000000NMTNoMLNMT
5Kyoto-U+ECNUzh-ja2020/09/10 23:58:1336770.8892690.8855000.889503-------NMTNoback-translation by using ja monolingual data from ASPEC-JE; lightconv (pay less attention) single model without ensemble
6KNU_Hyundaizh-ja2019/07/27 10:30:0431790.8885300.8862480.888706-------NMTNoTransformer(base) + *Used ASPEC ja-en corpus* with relative position, bt, multi source, r2l rerank, 6-model ensemble
7srcbzh-ja2019/07/25 11:37:4429170.8860690.8827120.886854-------NMTNoTransformer (Big) with relative position, layer attention, sentence-wise smooth.
8NICT-5zh-ja2018/08/27 14:40:3521690.8861610.8829890.886367-----0.0000000.000000NMTNoCombining En-Ja corpus with Zh-Ja as a multilingual model. *ADDITIONAL ASPEC CORPUS USED*
9NICT-5zh-ja2018/08/22 18:51:4420520.8844260.8794560.884782-----0.0000000.000000NMTNoMixed fine tuning by first pretraining on En-Ja ASPEC data and then continue on the En-Ja+Zh-Ja data. Transformer.
10Kyoto-Uzh-ja2017/08/01 14:14:4917200.8842100.8800690.884745----0.0000000.0000000.000000NMTNoEnsemble of 7 shared BPE, averaged
11Kyoto-Uzh-ja2017/07/29 08:02:0715770.8834570.8789640.884137----0.0000000.0000000.000000NMTNoEnsemble of 5 Shared BPE 40k
12NICT-2zh-ja2017/07/26 14:08:4514810.8823560.8785800.882195----0.0000000.0000000.000000NMTNoNMT 6 Ensembles * Bi-directional Reranking
13ORGANIZERzh-ja2017/08/02 09:59:3317400.8808150.8755110.880368----0.0000000.0000000.000000NMTNoGoogle's "Attention Is All You Need"
14Kyoto-Uzh-ja2017/07/31 15:27:2116430.8780080.8729440.878627----0.0000000.0000000.000000NMTNoKW replacement without KW in the test set, BPE, 6 ensemble
15Kyoto-Uzh-ja2016/08/20 22:50:3312560.8765310.8729040.876946----0.0000000.0000000.000000NMTNovoc: 30k ensemble of 3 independent model + reverse rescoring
16Kyoto-Uzh-ja2016/10/11 10:46:0313240.8752790.8701750.875564----0.0000000.0000000.000000NMTNovoc: 32k ensemble of 4 independent model + Chinese short unit
17NICT-2zh-ja2017/07/26 13:58:4414770.8714380.8683590.871736----0.0000000.0000000.000000NMTNoNMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
18ORGANIZERzh-ja2018/08/14 11:33:0319020.8707340.8662810.870886-----0.0000000.000000NMTNoNMT with Attention
19Kyoto-Uzh-ja2016/08/20 22:48:1612550.8693600.8647480.869913----0.0000000.0000000.000000NMTNosrc: 200k tgt: 50k 2-layers self-ensembling
20UT-KAYzh-ja2016/08/20 07:12:5212210.8602140.8546900.860449----0.0000000.0000000.000000NMTNoEnsemble of our NMT models with and without domain adaptation
21NAISTzh-ja2015/08/31 08:23:308340.8550890.8477460.854587---0.0000000.0000000.0000000.000000SMTNoTravatar System with NeuralMT Reranking
22bjtu_nlpzh-ja2016/08/12 12:50:3811380.8528180.8463010.852298----0.0000000.0000000.000000NMTNoRNN Encoder-Decoder with attention mechanism, single model
23UT-KAYzh-ja2016/08/20 07:09:5412200.8474070.8420550.848040----0.0000000.0000000.000000NMTNoAn end-to-end NMT with 512 dimensional single-layer LSTMs, UNK replacement, and domain adaptation
24NAISTzh-ja2014/08/01 17:33:011240.8454860.8380920.845625---0.0000000.0000000.0000000.000000SMTNoTravatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
25NICT-2zh-ja2016/08/05 18:05:0310990.8439410.8377070.842513----0.0000000.0000000.000000SMTYesPhrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
26NAISTzh-ja2014/07/31 11:42:311200.8424770.8348240.842235---0.0000000.0000000.0000000.000000SMTNoTravatar-based Forest-to-String SMT System
27EHRzh-ja2016/07/31 17:06:5710630.8437230.8361560.841952----0.0000000.0000000.000000SMTYesLM-based merging of outputs of preordered word-based PBSMT(DL=6) and preordered character-based PBSMT(DL=6).
28Kyoto-Uzh-ja2015/08/31 22:39:368450.8406810.8344510.839063---0.0000000.0000000.0000000.000000EBMTNoKyotoEBMT system with bilingual RNNLM reranking
29EHRzh-ja2015/09/02 17:00:168670.8376780.8316820.837227---0.0000000.0000000.0000000.000000SMTNoPhrase based SMT with preordering.
30NAISTzh-ja2015/08/31 08:26:318350.8343880.8271480.834130---0.0000000.0000000.0000000.000000SMTNoTravatar System Baseline
31SAS_MTzh-ja2014/09/01 10:38:132630.8341700.8255510.833048---0.0000000.0000000.0000000.000000SMTNoSyntactic reordering Hierarchical SMT (using SAS token tool)
32TOSHIBAzh-ja2015/07/23 15:14:535080.8272910.8173950.825472---0.0000000.0000000.0000000.000000SMT and RBMTYesSystem combination SMT and RBMT(SPE) with RNNLM language model
33ORGANIZERzh-ja2014/07/11 20:04:10130.8252920.8204900.825025---0.0000000.0000000.0000000.000000SMTNoTree-to-String SMT (2014)
34ORGANIZERzh-ja2015/09/10 14:00:338790.8252920.8204900.825025---0.0000000.0000000.0000000.000000SMTNoTree-to-String SMT (2015)
35EHRzh-ja2015/08/19 11:23:367200.8260030.8186200.824806---0.0000000.0000000.0000000.000000SMT and RBMTYesSystem combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
36TOSHIBAzh-ja2015/07/28 16:27:325250.8247400.8153880.822423---0.0000000.0000000.0000000.000000SMT and RBMTYesRBMT with SPE(Statistical Post Editing) system
37Kyoto-Uzh-ja2015/08/07 13:24:555970.8226720.8170370.822340---0.0000000.0000000.0000000.000000EBMTNoUpdated JUMAN and added one reordering feature, w/ reranking
38Kyoto-Uzh-ja2016/08/07 18:28:2311100.8202590.8146610.819963----0.0000000.0000000.000000EBMTNoKyotoEBMT 2016 w/o reranking
39Kyoto-Uzh-ja2015/08/31 22:38:228440.8197430.8145810.818794---0.0000000.0000000.0000000.000000EBMTNoKyotoEBMT system without reranking
40Kyoto-Uzh-ja2015/07/17 09:04:224910.8184450.8129100.817522---0.0000000.0000000.0000000.000000EBMTNoWAT2015 baseline with reranking
41SAS_MTzh-ja2014/08/29 15:33:072320.8221800.8075350.817368---0.0000000.0000000.0000000.000000SMTNoSyntactic reordering phrase-based SMT (SAS token tool)
42EHRzh-ja2015/09/04 11:44:268680.8158420.8067260.813996---0.0000000.0000000.0000000.000000SMT and RBMTYesRBMT with user dictionary plus SPE.
43Kyoto-Uzh-ja2015/07/17 09:01:424900.8093950.8037800.808692---0.0000000.0000000.0000000.000000EBMTNoWAT2015 baseline
44EIWAzh-ja2014/08/20 11:56:001380.8113500.8005060.808504---0.0000000.0000000.0000000.000000SMT and RBMTYesRBMT with user dictionary plus SPE(statistical post editing)
45ORGANIZERzh-ja2014/07/11 19:47:2740.8104060.7987260.807665---0.0000000.0000000.0000000.000000SMTNoHierarchical Phrase-based SMT (2014)
46Kyoto-Uzh-ja2014/09/01 21:33:232680.8026290.7986310.802930---0.0000000.0000000.0000000.000000EBMTNoOur new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
47BJTUNLPzh-ja2015/08/25 14:55:207690.8070120.7924880.802430---0.0000000.0000000.0000000.000000SMTNo
48BJTUNLPzh-ja2015/09/01 21:08:108620.8070120.7924880.802430---0.0000000.0000000.0000000.000000SMTNoa dependency-to-string model for SMT
49Kyoto-Uzh-ja2014/08/31 23:42:412580.8009490.7953900.800986---0.0000000.0000000.0000000.000000EBMTNoOur new baseline system after several modifications.
50Kyoto-Uzh-ja2014/08/19 09:31:081330.7916800.7871050.791269---0.0000000.0000000.0000000.000000EBMTNoUsing n-best parses and RNNLM.
51WASUIPSzh-ja2014/09/17 12:00:463880.7967770.7817330.791219---0.0000000.0000000.0000000.000000SMTYesOur baseline system + additional quasi-parallel corpus (segmentation tools: stanford-ctb and juman, moses: 2.1.1).
52WASUIPSzh-ja2014/09/17 11:03:463870.7960590.7804020.790107---0.0000000.0000000.0000000.000000SMTNoOur baseline system (segmentation tools: stanford-ctb and juman, moses: 2.1.1).
53WASUIPSzh-ja2014/09/17 10:26:433840.7962200.7780750.789657---0.0000000.0000000.0000000.000000SMTYesOur baseline system + additional quasi-parallel corpus (segmentation tools: kytea, moses: 2.1.1).
54WASUIPSzh-ja2014/09/17 10:24:503830.7932300.7751680.787665---0.0000000.0000000.0000000.000000SMTNoOur baseline system (segmentation tools: kytea, moses: 2.1.1).
55WASUIPSzh-ja2014/09/17 01:05:383750.7958380.7800270.787591---0.0000000.0000000.0000000.000000SMTYesOur baseline system + additional quasi-parallel corpus (segmentation tools: urheen and mecab, moses: 2.1.1).
56WASUIPSzh-ja2014/09/17 01:03:573740.7943030.7778760.786422---0.0000000.0000000.0000000.000000SMTNoOur baseline system (segmentation tools: urheen and mecab, moses: 2.1.1).
57Kyoto-Uzh-ja2014/08/19 10:21:371350.7862290.7830160.786352---0.0000000.0000000.0000000.000000EBMTNoOur baseline system.
58JAPIOzh-ja2016/08/19 16:44:4912080.7905530.7806370.785917----0.0000000.0000000.000000SMTYesPhrase-based SMT with Preordering + JAPIO corpus + rule-based posteditor
59Sensezh-ja2014/08/26 15:17:492000.7894950.7743380.784012---0.0000000.0000000.0000000.000000SMTNoCharacter based SMT
60WASUIPSzh-ja2014/09/17 00:46:073700.7898240.7731420.781475---0.0000000.0000000.0000000.000000SMTYesOur baseline system + additional quasi-parallel corpus (segmentation tools: urheen and mecab, moses: 1.0).
61Sensezh-ja2015/07/29 07:20:205330.7746920.7648470.772410---0.0000000.0000000.0000000.000000SMTNoBaseline-2015
62ORGANIZERzh-ja2014/07/11 19:54:5880.7724980.7663840.771005---0.0000000.0000000.0000000.000000SMTNoPhrase-based SMT
63WASUIPSzh-ja2014/09/17 00:43:383690.7791830.7629490.770846---0.0000000.0000000.0000000.000000SMTNoOur baseline system (segmentation tools: urheen and mecab, moses: 1.0).
64WASUIPSzh-ja2014/09/17 10:10:473800.7765500.7567210.769409---0.0000000.0000000.0000000.000000SMTYesOur baseline system + additional quasi-parallel corpus (segmentation tools: kytea, moses: 1.0).
65WASUIPSzh-ja2014/09/17 10:07:443790.7744230.7537490.767073---0.0000000.0000000.0000000.000000SMTNoOur baseline system (segmentation tools: kytea, moses: 1.0).
66TOSHIBAzh-ja2015/08/17 12:11:526690.7416650.7271550.738298---0.0000000.0000000.0000000.000000RBMTYesRBMT
67EIWAzh-ja2014/08/20 11:52:451370.7401830.7202810.732466---0.0000000.0000000.0000000.000000RBMTYesRBMT plus user dictionary
68ORGANIZERzh-ja2016/11/16 11:28:0013420.7190220.7171730.720095----0.0000000.0000000.000000NMTYesOnline A (2016/11/14)
69ORGANIZERzh-ja2014/08/29 18:45:032390.6662770.6524020.661730---0.0000000.0000000.0000000.000000RBMTNoRBMT A (2014)
70ORGANIZERzh-ja2015/09/10 14:30:568850.6662770.6524020.661730---0.0000000.0000000.0000000.000000OtherYesRBMT A (2015)
71ORGANIZERzh-ja2014/08/29 18:48:292420.6411890.6264000.633319---0.0000000.0000000.0000000.000000RBMTNoRBMT D
72ORGANIZERzh-ja2014/08/28 12:10:132150.6007330.5960060.600706---0.0000000.0000000.0000000.000000OtherYesOnline B (2014)
73ORGANIZERzh-ja2014/07/18 11:09:12360.5959250.5981720.598573---0.0000000.0000000.0000000.000000OtherYesOnline A (2014)
74ORGANIZERzh-ja2015/09/11 10:09:238900.5973550.5928410.597298---0.0000000.0000000.0000000.000000OtherYesOnline B (2015)
75ORGANIZERzh-ja2016/07/26 11:54:1410430.5898020.5893970.593361----0.0000000.0000000.000000OtherYesOnline A (2016)
76ORGANIZERzh-ja2015/08/25 18:58:087760.5882850.5903930.592887---0.0000000.0000000.0000000.000000OtherYesOnline A (2015)
77TMUzh-ja2018/09/14 17:30:3323430.5700400.5623310.565795-----0.0000000.000000NMTYesUnsupervised NMT with sub-character information. Both ASPEC and JPC 4.0 data (zh-ja) were also used as monolingual data in the training.

Notice:

Back to top

AMFM


# Team Task Date/Time DataID AMFM
Method
Other
Resources
System
Description
unuse unuse unuse unuse unuse unuse unuse unuse unuse unuse
1Kyoto-U+ECNUzh-ja2020/09/17 18:41:3438130.8233900.8233900.823390-------NMTYesensemble 9 models: structures(LSTM, Transformer, ConvS2S, Lightconv), training data(BT, out-of-domain parallel), S2S settings(deeper transformer, deep encoder shallow decoder)
2Kyoto-U+ECNUzh-ja2020/09/18 17:38:5539330.8216600.8216600.821660-------NMTNowithout out-of-domain parallel data; others same as DataID:3813
3srcbzh-ja2019/07/27 15:48:2432100.8190200.8190200.819020-------NMTNoTransformer(Big) with relative position, sentence-wise smooth, deep transformer, back translation, ensemble of 7 models.
4Kyoto-U+ECNUzh-ja2020/09/10 23:58:1336770.8176100.8176100.817610-------NMTNoback-translation by using ja monolingual data from ASPEC-JE; lightconv (pay less attention) single model without ensemble
5srcbzh-ja2019/07/25 11:37:4429170.8124500.8124500.812450-------NMTNoTransformer (Big) with relative position, layer attention, sentence-wise smooth.
6KNU_Hyundaizh-ja2019/07/27 10:30:0431790.8099900.8099900.809990-------NMTNoTransformer(base) + *Used ASPEC ja-en corpus* with relative position, bt, multi source, r2l rerank, 6-model ensemble
7NICT-5zh-ja2018/08/27 14:40:3521690.8057500.8057500.805750-----0.0000000.000000NMTNoCombining En-Ja corpus with Zh-Ja as a multilingual model. *ADDITIONAL ASPEC CORPUS USED*
8NICT-5zh-ja2018/09/10 14:14:0522670.8049200.8049200.804920-----0.0000000.000000NMTNoMLNMT
9NICT-5zh-ja2018/08/22 18:51:4420520.8006700.8006700.800670-----0.0000000.000000NMTNoMixed fine tuning by first pretraining on En-Ja ASPEC data and then continue on the En-Ja+Zh-Ja data. Transformer.
10Kyoto-Uzh-ja2017/08/01 14:14:4917200.7998400.7998400.799840----0.0000000.0000000.000000NMTNoEnsemble of 7 shared BPE, averaged
11NICT-2zh-ja2017/07/26 14:08:4514810.7996800.7996800.799680----0.0000000.0000000.000000NMTNoNMT 6 Ensembles * Bi-directional Reranking
12Kyoto-Uzh-ja2017/07/29 08:02:0715770.7995200.7995200.799520----0.0000000.0000000.000000NMTNoEnsemble of 5 Shared BPE 40k
13ORGANIZERzh-ja2017/08/02 09:59:3317400.7981100.7981100.798110----0.0000000.0000000.000000NMTNoGoogle's "Attention Is All You Need"
14Kyoto-Uzh-ja2017/07/31 15:27:2116430.7934100.7934100.793410----0.0000000.0000000.000000NMTNoKW replacement without KW in the test set, BPE, 6 ensemble
15NICT-2zh-ja2017/07/26 13:58:4414770.7889400.7889400.788940----0.0000000.0000000.000000NMTNoNMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
16Kyoto-Uzh-ja2016/10/11 10:46:0313240.7879300.7879300.787930----0.0000000.0000000.000000NMTNovoc: 32k ensemble of 4 independent model + Chinese short unit
17Kyoto-Uzh-ja2016/08/20 22:50:3312560.7859100.7859100.785910----0.0000000.0000000.000000NMTNovoc: 30k ensemble of 3 independent model + reverse rescoring
18Kyoto-Uzh-ja2016/08/20 22:48:1612550.7843800.7843800.784380----0.0000000.0000000.000000NMTNosrc: 200k tgt: 50k 2-layers self-ensembling
19ORGANIZERzh-ja2018/08/14 11:33:0319020.7821000.7821000.782100-----0.0000000.000000NMTNoNMT with Attention
20NAISTzh-ja2015/08/31 08:23:308340.7710100.7710100.7710100.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoTravatar System with NeuralMT Reranking
21Kyoto-Uzh-ja2015/08/31 22:39:368450.7697000.7697000.7697000.0000000.0000000.0000000.0000000.0000000.0000000.000000EBMTNoKyotoEBMT system with bilingual RNNLM reranking
22EHRzh-ja2016/07/31 17:06:5710630.7694900.7694900.769490----0.0000000.0000000.000000SMTYesLM-based merging of outputs of preordered word-based PBSMT(DL=6) and preordered character-based PBSMT(DL=6).
23NICT-2zh-ja2016/08/05 18:05:0310990.7685800.7685800.768580----0.0000000.0000000.000000SMTYesPhrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
24NAISTzh-ja2014/07/31 11:42:311200.7681900.7681900.7681900.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoTravatar-based Forest-to-String SMT System
25Kyoto-Uzh-ja2016/08/07 18:28:2311100.7671200.7671200.767120----0.0000000.0000000.000000EBMTNoKyotoEBMT 2016 w/o reranking
26NAISTzh-ja2014/08/01 17:33:011240.7662700.7662700.7662700.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoTravatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
27SAS_MTzh-ja2014/09/01 10:38:132630.7657300.7657300.7657300.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoSyntactic reordering Hierarchical SMT (using SAS token tool)
28UT-KAYzh-ja2016/08/20 07:12:5212210.7655300.7655300.765530----0.0000000.0000000.000000NMTNoEnsemble of our NMT models with and without domain adaptation
29EHRzh-ja2015/08/19 11:23:367200.7650500.7650500.7650500.0000000.0000000.0000000.0000000.0000000.0000000.000000SMT and RBMTYesSystem combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
30NAISTzh-ja2015/08/31 08:26:318350.7648300.7648300.7648300.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoTravatar System Baseline
31Kyoto-Uzh-ja2015/08/07 13:24:555970.7624300.7624300.7624300.0000000.0000000.0000000.0000000.0000000.0000000.000000EBMTNoUpdated JUMAN and added one reordering feature, w/ reranking
32Kyoto-Uzh-ja2015/07/17 09:04:224910.7621800.7621800.7621800.0000000.0000000.0000000.0000000.0000000.0000000.000000EBMTNoWAT2015 baseline with reranking
33Kyoto-Uzh-ja2015/08/31 22:38:228440.7619600.7619600.7619600.0000000.0000000.0000000.0000000.0000000.0000000.000000EBMTNoKyotoEBMT system without reranking
34bjtu_nlpzh-ja2016/08/12 12:50:3811380.7608400.7608400.760840----0.0000000.0000000.000000NMTNoRNN Encoder-Decoder with attention mechanism, single model
35TOSHIBAzh-ja2015/07/28 16:27:325250.7581100.7581100.7581100.0000000.0000000.0000000.0000000.0000000.0000000.000000SMT and RBMTYesRBMT with SPE(Statistical Post Editing) system
36Kyoto-Uzh-ja2014/09/01 21:33:232680.7576100.7576100.7576100.0000000.0000000.0000000.0000000.0000000.0000000.000000EBMTNoOur new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
37Kyoto-Uzh-ja2015/07/17 09:01:424900.7570700.7570700.7570700.0000000.0000000.0000000.0000000.0000000.0000000.000000EBMTNoWAT2015 baseline
38ORGANIZERzh-ja2014/07/11 20:04:10130.7548700.7548700.7548700.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoTree-to-String SMT (2014)
39ORGANIZERzh-ja2015/09/10 14:00:338790.7548700.7548700.7548700.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoTree-to-String SMT (2015)
40EHRzh-ja2015/09/04 11:44:268680.7541800.7541800.7541800.0000000.0000000.0000000.0000000.0000000.0000000.000000SMT and RBMTYesRBMT with user dictionary plus SPE.
41UT-KAYzh-ja2016/08/20 07:09:5412200.7538200.7538200.753820----0.0000000.0000000.000000NMTNoAn end-to-end NMT with 512 dimensional single-layer LSTMs, UNK replacement, and domain adaptation
42WASUIPSzh-ja2014/09/17 10:24:503830.7537500.7537500.7537500.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoOur baseline system (segmentation tools: kytea, moses: 2.1.1).
43WASUIPSzh-ja2014/09/17 10:26:433840.7536900.7536900.7536900.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTYesOur baseline system + additional quasi-parallel corpus (segmentation tools: kytea, moses: 2.1.1).
44ORGANIZERzh-ja2014/07/11 19:54:5880.7530100.7530100.7530100.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoPhrase-based SMT
45Sensezh-ja2014/08/26 15:17:492000.7528900.7528900.7528900.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoCharacter based SMT
46TOSHIBAzh-ja2015/07/23 15:14:535080.7528300.7528300.7528300.0000000.0000000.0000000.0000000.0000000.0000000.000000SMT and RBMTYesSystem combination SMT and RBMT(SPE) with RNNLM language model
47SAS_MTzh-ja2014/08/29 15:33:072320.7521700.7521700.7521700.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoSyntactic reordering phrase-based SMT (SAS token tool)
48ORGANIZERzh-ja2014/07/11 19:47:2740.7509500.7509500.7509500.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoHierarchical Phrase-based SMT (2014)
49Kyoto-Uzh-ja2014/08/31 23:42:412580.7503700.7503700.7503700.0000000.0000000.0000000.0000000.0000000.0000000.000000EBMTNoOur new baseline system after several modifications.
50Kyoto-Uzh-ja2014/08/19 09:31:081330.7503100.7503100.7503100.0000000.0000000.0000000.0000000.0000000.0000000.000000EBMTNoUsing n-best parses and RNNLM.
51Kyoto-Uzh-ja2014/08/19 10:21:371350.7482000.7482000.7482000.0000000.0000000.0000000.0000000.0000000.0000000.000000EBMTNoOur baseline system.
52BJTUNLPzh-ja2015/08/25 14:55:207690.7441300.7441300.7441300.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNo
53BJTUNLPzh-ja2015/09/01 21:08:108620.7441300.7441300.7441300.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoa dependency-to-string model for SMT
54WASUIPSzh-ja2014/09/17 12:00:463880.7440400.7440400.7440400.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTYesOur baseline system + additional quasi-parallel corpus (segmentation tools: stanford-ctb and juman, moses: 2.1.1).
55WASUIPSzh-ja2014/09/17 11:03:463870.7431400.7431400.7431400.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoOur baseline system (segmentation tools: stanford-ctb and juman, moses: 2.1.1).
56WASUIPSzh-ja2014/09/17 01:03:573740.7406500.7406500.7406500.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoOur baseline system (segmentation tools: urheen and mecab, moses: 2.1.1).
57WASUIPSzh-ja2014/09/17 01:05:383750.7406400.7406400.7406400.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTYesOur baseline system + additional quasi-parallel corpus (segmentation tools: urheen and mecab, moses: 2.1.1).
58WASUIPSzh-ja2014/09/17 00:46:073700.7346200.7346200.7346200.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTYesOur baseline system + additional quasi-parallel corpus (segmentation tools: urheen and mecab, moses: 1.0).
59Sensezh-ja2015/07/29 07:20:205330.7331900.7331900.7331900.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoBaseline-2015
60WASUIPSzh-ja2014/09/17 10:07:443790.7253600.7253600.7253600.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoOur baseline system (segmentation tools: kytea, moses: 1.0).
61WASUIPSzh-ja2014/09/17 10:10:473800.7252500.7252500.7252500.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTYesOur baseline system + additional quasi-parallel corpus (segmentation tools: kytea, moses: 1.0).
62WASUIPSzh-ja2014/09/17 00:43:383690.7116500.7116500.7116500.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoOur baseline system (segmentation tools: urheen and mecab, moses: 1.0).
63EHRzh-ja2015/09/02 17:00:168670.7073100.7073100.7073100.0000000.0000000.0000000.0000000.0000000.0000000.000000SMTNoPhrase based SMT with preordering.
64JAPIOzh-ja2016/08/19 16:44:4912080.6967700.6967700.696770----0.0000000.0000000.000000SMTYesPhrase-based SMT with Preordering + JAPIO corpus + rule-based posteditor
65EIWAzh-ja2014/08/20 11:56:001380.6933300.6933300.6933300.0000000.0000000.0000000.0000000.0000000.0000000.000000SMT and RBMTYesRBMT with user dictionary plus SPE(statistical post editing)
66ORGANIZERzh-ja2016/11/16 11:28:0013420.6928200.6928200.692820----0.0000000.0000000.000000NMTYesOnline A (2016/11/14)
67ORGANIZERzh-ja2016/07/26 11:54:1410430.6595400.6595400.659540----0.0000000.0000000.000000OtherYesOnline A (2016)
68ORGANIZERzh-ja2014/07/18 11:09:12360.6580600.6580600.6580600.0000000.0000000.0000000.0000000.0000000.0000000.000000OtherYesOnline A (2014)
69TOSHIBAzh-ja2015/08/17 12:11:526690.6540800.6540800.6540800.0000000.0000000.0000000.0000000.0000000.0000000.000000RBMTYesRBMT
70ORGANIZERzh-ja2015/08/25 18:58:087760.6498600.6498600.6498600.0000000.0000000.0000000.0000000.0000000.0000000.000000OtherYesOnline A (2015)
71ORGANIZERzh-ja2014/08/28 12:10:132150.6369300.6369300.6369300.0000000.0000000.0000000.0000000.0000000.0000000.000000OtherYesOnline B (2014)
72ORGANIZERzh-ja2015/09/11 10:09:238900.6282900.6282900.6282900.0000000.0000000.0000000.0000000.0000000.0000000.000000OtherYesOnline B (2015)
73ORGANIZERzh-ja2014/08/29 18:45:032390.6260700.6260700.6260700.0000000.0000000.0000000.0000000.0000000.0000000.000000RBMTNoRBMT A (2014)
74ORGANIZERzh-ja2015/09/10 14:30:568850.6260700.6260700.6260700.0000000.0000000.0000000.0000000.0000000.0000000.000000OtherYesRBMT A (2015)
75EIWAzh-ja2014/08/20 11:52:451370.6137300.6137300.6137300.0000000.0000000.0000000.0000000.0000000.0000000.000000RBMTYesRBMT plus user dictionary
76ORGANIZERzh-ja2014/08/29 18:48:292420.5867900.5867900.5867900.0000000.0000000.0000000.0000000.0000000.0000000.000000RBMTNoRBMT D
77TMUzh-ja2018/09/14 17:30:3323430.5124300.5124300.512430-----0.0000000.000000NMTYesUnsupervised NMT with sub-character information. Both ASPEC and JPC 4.0 data (zh-ja) were also used as monolingual data in the training.

Notice:

Back to top

HUMAN (WAT2022)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description

Notice:
Back to top

HUMAN (WAT2021)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description

Notice:
Back to top

HUMAN (WAT2020)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1Kyoto-U+ECNUzh-ja2020/09/17 18:41:3438134.210NMTYesensemble 9 models: structures(LSTM, Transformer, ConvS2S, Lightconv), training data(BT, out-of-domain parallel), S2S settings(deeper transformer, deep encoder shallow decoder)
2Kyoto-U+ECNUzh-ja2020/09/18 17:38:5539334.200NMTNowithout out-of-domain parallel data; others same as DataID:3813

Notice:
Back to top

HUMAN (WAT2019)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1KNU_Hyundaizh-ja2019/07/27 10:30:043179UnderwayNMTNoTransformer(base) + *Used ASPEC ja-en corpus* with relative position, bt, multi source, r2l rerank, 6-model ensemble
2srcbzh-ja2019/07/27 15:48:243210UnderwayNMTNoTransformer(Big) with relative position, sentence-wise smooth, deep transformer, back translation, ensemble of 7 models.

Notice:
Back to top

HUMAN (WAT2018)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1NICT-5zh-ja2018/09/10 14:14:05226722.750NMTNoMLNMT
2NICT-5zh-ja2018/08/22 18:51:44205211.000NMTNoMixed fine tuning by first pretraining on En-Ja ASPEC data and then continue on the En-Ja+Zh-Ja data. Transformer.

Notice:
Back to top

HUMAN (WAT2017)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1Kyoto-Uzh-ja2017/08/01 14:14:49172082.750NMTNoEnsemble of 7 shared BPE, averaged
2Kyoto-Uzh-ja2017/07/29 08:02:07157779.500NMTNoEnsemble of 5 Shared BPE 40k
3NICT-2zh-ja2017/07/26 14:08:45148179.000NMTNoNMT 6 Ensembles * Bi-directional Reranking
4ORGANIZERzh-ja2017/08/02 09:59:33174078.500NMTNoGoogle's "Attention Is All You Need"
5NICT-2zh-ja2017/07/26 13:58:44147778.000NMTNoNMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder

Notice:
Back to top

HUMAN (WAT2016)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1Kyoto-Uzh-ja2016/08/20 22:50:33125663.750NMTNovoc: 30k ensemble of 3 independent model + reverse rescoring
2Kyoto-Uzh-ja2016/08/20 22:48:16125556.000NMTNosrc: 200k tgt: 50k 2-layers self-ensembling
3bjtu_nlpzh-ja2016/08/12 12:50:38113849.000NMTNoRNN Encoder-Decoder with attention mechanism, single model
4UT-KAYzh-ja2016/08/20 07:12:52122147.250NMTNoEnsemble of our NMT models with and without domain adaptation
5UT-KAYzh-ja2016/08/20 07:09:54122041.000NMTNoAn end-to-end NMT with 512 dimensional single-layer LSTMs, UNK replacement, and domain adaptation
6NICT-2zh-ja2016/08/05 18:05:03109936.500SMTYesPhrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
7EHRzh-ja2016/07/31 17:06:57106332.500SMTYesLM-based merging of outputs of preordered word-based PBSMT(DL=6) and preordered character-based PBSMT(DL=6).
8ORGANIZERzh-ja2016/11/16 11:28:00134222.500NMTYesOnline A (2016/11/14)
9JAPIOzh-ja2016/08/19 16:44:49120816.500SMTYesPhrase-based SMT with Preordering + JAPIO corpus + rule-based posteditor
10ORGANIZERzh-ja2016/07/26 11:54:141043-51.250OtherYesOnline A (2016)

Notice:
Back to top

HUMAN (WAT2015)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1NAISTzh-ja2015/08/31 08:23:3083435.750SMTNoTravatar System with NeuralMT Reranking
2EHRzh-ja2015/08/19 11:23:3672025.750SMT and RBMTYesSystem combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
3NAISTzh-ja2015/08/31 08:26:3183525.750SMTNoTravatar System Baseline
4Kyoto-Uzh-ja2015/08/31 22:39:3684518.500EBMTNoKyotoEBMT system with bilingual RNNLM reranking
5TOSHIBAzh-ja2015/07/23 15:14:5350818.000SMT and RBMTYesSystem combination SMT and RBMT(SPE) with RNNLM language model
6ORGANIZERzh-ja2015/09/10 14:00:3387917.250SMTNoTree-to-String SMT (2015)
7Kyoto-Uzh-ja2015/08/31 22:38:2284416.750EBMTNoKyotoEBMT system without reranking
8BJTUNLPzh-ja2015/09/01 21:08:108626.500SMTNoa dependency-to-string model for SMT
9TOSHIBAzh-ja2015/07/28 16:27:32525-1.000SMT and RBMTYesRBMT with SPE(Statistical Post Editing) system
10ORGANIZERzh-ja2015/08/25 18:58:08776-19.000OtherYesOnline A (2015)
11ORGANIZERzh-ja2015/09/10 14:30:56885-28.000OtherYesRBMT A (2015)

Notice:
Back to top

HUMAN (WAT2014)


# Team Task Date/Time DataID HUMAN
Method
Other
Resources
System
Description
1NAISTzh-ja2014/07/31 11:42:3112050.750SMTNoTravatar-based Forest-to-String SMT System
2NAISTzh-ja2014/08/01 17:33:0112438.000SMTNoTravatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
3SAS_MTzh-ja2014/09/01 10:38:1326322.500SMTNoSyntactic reordering Hierarchical SMT (using SAS token tool)
4ORGANIZERzh-ja2014/07/11 20:04:101316.000SMTNoTree-to-String SMT (2014)
5EIWAzh-ja2014/08/20 11:56:0013815.000SMT and RBMTYesRBMT with user dictionary plus SPE(statistical post editing)
6Kyoto-Uzh-ja2014/09/01 21:33:232687.500EBMTNoOur new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
7Kyoto-Uzh-ja2014/08/31 23:42:412586.000EBMTNoOur new baseline system after several modifications.
8ORGANIZERzh-ja2014/07/11 19:47:2744.750SMTNoHierarchical Phrase-based SMT (2014)
9Sensezh-ja2014/08/26 15:17:49200-1.000SMTNoCharacter based SMT
10ORGANIZERzh-ja2014/07/18 11:09:1236-21.750OtherYesOnline A (2014)
11ORGANIZERzh-ja2014/08/29 18:45:03239-37.750RBMTNoRBMT A (2014)

Notice:
Back to top

EVALUATION RESULTS USAGE POLICY

When you use the WAT evaluation results for any purpose such as:
- writing technical papers,
- making presentations about your system,
- advertising your MT system to the customers,
you can use the information about translation directions, scores (including both automatic and human evaluations) and ranks of your system among others. You can also use the scores of the other systems, but you MUST anonymize the other system's names. In addition, you can show the links (URLs) to the WAT evaluation result pages.

NICT (National Institute of Information and Communications Technology)
Kyoto University
Last Modified: 2018-08-02