WAT

The Workshop on Asian Translation

Evaluation Results

BLEU

#	Team	Task	Date/Time	DataID	BLEU										Method	Other Resources	System Description
#	Team	Task	Date/Time	DataID	juman	kytea	mecab	moses- tokenizer	stanford- segmenter- ctb	stanford- segmenter- pku	indic- tokenizer	unuse	myseg	kmseg	Method	Other Resources	System Description
1	srcb	ja-en	2019/07/27 15:27:01	3205	-	-	-	30.92	-	-	-	-	-	-	NMT	No	Transformer (Big) with relative position, data augmentation, average checkpoints, Bayes re-ranking.
2	KNU_Hyundai	ja-en	2019/07/27 09:28:37	3173	-	-	-	30.88	-	-	-	-	-	-	NMT	No	Transformer Base, relative position, BT, r2l reranking, ensemble of 3 models
3	srcb	ja-en	2019/07/25 11:52:48	2919	-	-	-	30.62	-	-	-	-	-	-	NMT	No	Transformer (Big) with relative position, average checkpoints.
4	srcb	ja-en	2018/09/16 14:51:47	2474	-	-	-	30.59	-	-	-	-	0.00	0.00	NMT	No	Transformer with relative position, ensemble of 3 models.
5	NTT	ja-en	2019/07/28 11:18:59	3225	-	-	-	30.56	-	-	-	-	-	-	SMT	No	ASPEC first 1.5M ＋last 1.5M (fwd), 6 ensemble
6	NTT	ja-en	2019/07/28 15:11:04	3233	-	-	-	30.28	-	-	-	-	-	-	NMT	Yes	ParaCrawl + (ASPEC first 1.5M + Synthetic 1.5M) * 2 oversampling, fine-tune ASPEC, SINGLE MODEL
7	AISTAI	ja-en	2019/08/31 21:28:13	3361	-	-	-	29.71	-	-	-	-	-	-	NMT	No	Transformer, 1.5M sentences, relative position, ensemble of 4 models, by OpenNMT-py.
8	NICT-5	ja-en	2018/09/10 14:55:37	2273	-	-	-	29.65	-	-	-	-	0.00	0.00	NMT	No	MLNMT
9	NICT-2	ja-en	2019/07/26 22:37:22	3086	-	-	-	29.40	-	-	-	-	-	-	NMT	No	Transformer, ensemble of 4 models w/ long warm-up and self-training
10	NICT-5	ja-en	2019/07/16 17:11:38	2718	-	-	-	29.12	-	-	-	-	-	-	NMT	No	RSNMT 6 layer with distillation
11	AISTAI	ja-en	2019/08/01 10:44:49	3260	-	-	-	29.01	-	-	-	-	-	-	NMT	No	Transformer (big), 1.5M sentences, train_steps=300000, Averaged the last 10 ckpts, by Tensor2Tensor.
12	NICT-5	ja-en	2018/08/27 15:01:05	2174	-	-	-	28.63	-	-	-	-	0.00	0.00	NMT	No	Transformer vanilla model using 3M sentences.
13	NICT-2	ja-en	2019/07/26 22:33:30	3085	-	-	-	28.61	-	-	-	-	-	-	NMT	No	Transformer, sigle model w/ long warm-up and self-training
14	srcb	ja-en	2018/08/26 11:09:50	2152	-	-	-	28.46	-	-	-	-	0.00	0.00	NMT	No	Transformer, average checkpoints.
15	NTT	ja-en	2017/08/01 04:29:02	1681	-	-	-	28.36	-	-	-	0.00	0.00	0.00	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
16	NICT-5	ja-en	2021/03/18 23:22:53	4574	-	-	-	28.35	-	-	-	-	-	-	NMT	No	My NMT implementation. Beam size 8. LP 0.6
17	NTT	ja-en	2017/08/01 15:53:15	1732	-	-	-	28.15	-	-	-	0.00	0.00	0.00	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from training data)
18	ORGANIZER	ja-en	2017/08/02 01:03:08	1736	-	-	-	28.06	-	-	-	0.00	0.00	0.00	NMT	No	Google's "Attention Is All You Need"
19	Kyoto-U	ja-en	2017/08/01 16:38:21	1733	-	-	-	27.66	-	-	-	0.00	0.00	0.00	NMT	No	Ensemble of 4 BPE, averaged Coverage penalty
20	ykkd	ja-en	2019/07/26 12:26:03	2989	-	-	-	27.63	-	-	-	-	-	-	NMT	No	Fully Character-level,6 bi-LSTM (512*2) for Encoder, 6 LSTM for Decoder. Middle dense layer. Beam w/4.Length norm set to 0.2
21	NTT	ja-en	2017/08/01 14:50:51	1724	-	-	-	27.62	-	-	-	0.00	0.00	0.00	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
22	NAIST	ja-en	2016/08/27 00:05:38	1275	-	-	-	27.55	-	-	-	0.00	0.00	0.00	SMT	No	Neural MT w/ Lexicon and MinRisk Training 6 Ensemble
23	Kyoto-U	ja-en	2017/08/01 13:42:25	1717	-	-	-	27.53	-	-	-	0.00	0.00	0.00	NMT	No	Ensemble of 4 BPE averaged parameters
24	NTT	ja-en	2017/07/30 20:43:07	1616	-	-	-	27.43	-	-	-	0.00	0.00	0.00	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
25	NICT-5	ja-en	2019/07/26 18:02:47	3041	-	-	-	26.99	-	-	-	-	-	-	NMT	No	RSNMT 6 layer
26	ORGANIZER	ja-en	2018/08/14 11:07:47	1901	-	-	-	26.91	-	-	-	-	0.00	0.00	NMT	No	NMT with Attention
27	NICT-2	ja-en	2017/07/26 14:04:38	1480	-	-	-	26.76	-	-	-	0.00	0.00	0.00	NMT	No	NMT 6 Ensembles * Bi-directional Reranking
28	NAIST	ja-en	2016/08/09 16:14:05	1122	-	-	-	26.39	-	-	-	0.00	0.00	0.00	SMT	No	Neural MT w/ Lexicon and MinRisk Training 4 Ensemble
29	Kyoto-U	ja-en	2016/08/18 15:17:05	1182	-	-	-	26.22	-	-	-	0.00	0.00	0.00	NMT	No	Ensemble of 4 single-layer model (30k voc)
30	Osaka-U	ja-en	2018/09/15 23:05:12	2440	-	-	-	26.19	-	-	-	-	0.00	0.00	NMT	Yes	rewarding model
31	NAIST	ja-en	2016/08/20 15:33:12	1247	-	-	-	26.12	-	-	-	0.00	0.00	0.00	SMT	No	Neural MT w/ Lexicon 6 Ensemble
32	TMU	ja-en	2018/09/16 12:04:36	2464	-	-	-	25.85	-	-	-	-	0.00	0.00	NMT	No	Ensemble of 6 Baseline-NMT
33	TMU	ja-en	2018/09/16 17:02:15	2483	-	-	-	25.45	-	-	-	-	0.00	0.00	NMT	No	Ensemble of 6 NMT ( 2 Baseline + 2 Reconstructor + 2 GAN )
34	NAIST	ja-en	2015/08/14 17:46:43	655	-	-	-	25.41	-	-	0.00	0.00	0.00	0.00	SMT	No	Travatar System with NeuralMT Reranking and Parser Self Training
35	TMU	ja-en	2018/09/16 12:05:47	2465	-	-	-	25.17	-	-	-	-	0.00	0.00	NMT	No	GAN-NMT ( Single )
36	TMU	ja-en	2018/09/16 12:06:39	2466	-	-	-	24.98	-	-	-	-	0.00	0.00	NMT	No	Reconstructor-NMT ( Single )
37	TMU	ja-en	2018/09/16 11:53:24	2461	-	-	-	24.94	-	-	-	-	0.00	0.00	NMT	No	Baseline-NMT ( Single )
38	NICT-2	ja-en	2017/07/26 13:54:28	1476	-	-	-	24.79	-	-	-	0.00	0.00	0.00	NMT	No	NMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
39	NAIST	ja-en	2015/08/24 23:53:53	757	-	-	-	24.77	-	-	0.00	0.00	0.00	0.00	SMT	No	Travatar System with NeuralMT Reranking
40	Kyoto-U	ja-en	2016/08/20 15:07:47	1246	-	-	-	24.71	-	-	-	0.00	0.00	0.00	NMT	No	voc src:200k voc tgt: 52k + BPE 2-layer self-ensembling
41	TMU	ja-en	2017/08/04 11:16:36	1750	-	-	-	24.55	-	-	-	0.00	0.00	0.00	NMT	No	beam_size: 10, ensemble of different dropout rates.
42	NAIST	ja-en	2014/07/19 01:04:48	46	-	-	-	23.82	-	-	0.00	0.00	0.00	0.00	SMT	Yes	Travatar-based Forest-to-String SMT System with Extra Dictionaries
43	NAIST	ja-en	2014/08/01 17:35:16	125	-	-	-	23.47	-	-	0.00	0.00	0.00	0.00	SMT	No	Travatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
44	CUNI	ja-en	2017/07/31 22:30:52	1665	-	-	-	23.43	-	-	-	0.00	0.00	0.00	NMT	No	Bahdanau (2014) seq2seq with conditional GRU on byte-pair encoding, 1M sentences
45	NAIST	ja-en	2014/07/31 11:40:53	119	-	-	-	23.29	-	-	0.00	0.00	0.00	0.00	SMT	No	Travatar-based Forest-to-String SMT System
46	TMU	ja-en	2017/08/01 11:35:08	1703	-	-	-	23.03	-	-	-	0.00	0.00	0.00	NMT	No	baseline system with beam20
47	TOSHIBA	ja-en	2015/07/23 15:00:12	506	-	-	-	23.00	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	System combination SMT and RBMT(SPE) with RNNLM language model
48	TOSHIBA	ja-en	2015/07/28 16:44:27	529	-	-	-	22.89	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
49	Kyoto-U	ja-en	2015/08/30 13:02:18	829	-	-	-	22.89	-	-	0.00	0.00	0.00	0.00	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking
50	TMU	ja-en	2017/08/01 12:14:41	1712	-	-	-	22.87	-	-	-	0.00	0.00	0.00	NMT	No	the ensemble system of different dropout rate.
51	NAIST	ja-en	2015/08/25 13:02:45	766	-	-	-	22.62	-	-	0.00	0.00	0.00	0.00	SMT	No	Travatar System with Parser Self Training
52	NAIST	ja-en	2015/08/25 13:03:48	767	-	-	-	22.16	-	-	0.00	0.00	0.00	0.00	SMT	No	Travatar System Baseline
53	ORGANIZER	ja-en	2016/11/16 10:29:51	1333	-	-	-	22.04	-	-	-	0.00	0.00	0.00	NMT	Yes	Online D (2016/11/14)
54	NICT-2	ja-en	2016/08/05 17:50:25	1104	-	-	-	21.54	-	-	-	0.00	0.00	0.00	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
55	Kyoto-U	ja-en	2015/08/27 14:40:32	796	-	-	-	21.31	-	-	0.00	0.00	0.00	0.00	EBMT	No	KyotoEBMT system without reranking
56	Kyoto-U	ja-en	2016/08/19 01:31:01	1189	-	-	-	21.22	-	-	-	0.00	0.00	0.00	EBMT	No	KyotoEBMT 2016 w/o reranking
57	Kyoto-U	ja-en	2014/09/01 10:27:54	262	-	-	-	21.07	-	-	0.00	0.00	0.00	0.00	EBMT	No	Our new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
58	TMU	ja-en	2017/08/01 11:10:49	1695	-	-	-	21.00	-	-	-	0.00	0.00	0.00	NMT	No	our baseline system in 2017
59	TOSHIBA	ja-en	2014/08/29 18:48:24	241	-	-	-	20.61	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
60	Kyoto-U	ja-en	2014/08/31 23:36:50	256	-	-	-	20.60	-	-	0.00	0.00	0.00	0.00	EBMT	No	Our new baseline system after several modifications.
61	ORGANIZER	ja-en	2014/07/11 19:59:55	9	-	-	-	20.36	-	-	0.00	0.00	0.00	0.00	SMT	No	String-to-Tree SMT (2014)
62	ORGANIZER	ja-en	2015/09/10 13:41:03	877	-	-	-	20.36	-	-	0.00	0.00	0.00	0.00	SMT	No	String-to-Tree SMT (2015)
63	Kyoto-U	ja-en	2014/08/19 10:24:06	136	-	-	-	20.02	-	-	0.00	0.00	0.00	0.00	EBMT	No	Our baseline system using 3M parallel sentences.
64	EIWA	ja-en	2014/07/30 16:07:14	116	-	-	-	19.86	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	Combination of RBMT and SPE(statistical post editing)
65	NICT	ja-en	2015/07/16 13:27:58	488	-	-	-	18.98	-	-	0.00	0.00	0.00	0.00	SMT	No	our baseline (DL=6) + dependency-based pre-reordering [Ding+ 2015]
66	NICT	ja-en	2015/07/17 11:02:10	492	-	-	-	18.96	-	-	0.00	0.00	0.00	0.00	SMT	No	our baseline (DL=9) + reverse pre-reordering [Katz-Brown & Collins 2008]
67	Sense	ja-en	2014/08/23 05:34:03	164	-	-	-	18.82	-	-	0.00	0.00	0.00	0.00	SMT	No	Paraphrase max10
68	ORGANIZER	ja-en	2014/07/11 19:45:32	2	-	-	-	18.72	-	-	0.00	0.00	0.00	0.00	SMT	No	Hierarchical Phrase-based SMT (2014)
69	ORGANIZER	ja-en	2014/07/11 19:49:57	6	-	-	-	18.45	-	-	0.00	0.00	0.00	0.00	SMT	No	Phrase-based SMT
70	TMU	ja-en	2016/08/20 14:31:48	1234	-	-	-	18.45	-	-	-	0.00	0.00	0.00	NMT	No	6 ensemble
71	bjtu_nlp	ja-en	2016/08/17 19:51:24	1168	-	-	-	18.34	-	-	-	0.00	0.00	0.00	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
72	TMU	ja-en	2015/08/04 16:32:20	578	-	-	-	18.32	-	-	0.00	0.00	0.00	0.00	SMT	No	Our PBSMT baseline (2015)
73	TMU	ja-en	2016/08/20 07:39:02	1222	-	-	-	18.29	-	-	-	0.00	0.00	0.00	NMT	No	2016 our proposed method to control output voice
74	Sense	ja-en	2015/08/28 19:25:25	822	-	-	-	18.20	-	-	0.00	0.00	0.00	0.00	SMT	No	Baseline-2015 (train1 only)
75	NICT	ja-en	2015/07/17 08:51:45	489	-	-	-	18.09	-	-	0.00	0.00	0.00	0.00	SMT	No	our baseline: PB SMT in MOSES (DL=20) / SRILM / MeCab (IPA)
76	Sense	ja-en	2015/08/29 04:32:36	824	-	-	-	18.09	-	-	0.00	0.00	0.00	0.00	SMT	No	Baseline-2015 (train123)
77	NII	ja-en	2014/09/02 11:42:01	271	-	-	-	17.47	-	-	0.00	0.00	0.00	0.00	SMT	No	Our Baseline
78	Sense	ja-en	2015/07/28 22:23:43	530	-	-	-	17.04	-	-	0.00	0.00	0.00	0.00	SMT	No	Baseline-2015
79	NII	ja-en	2014/09/02 11:42:53	272	-	-	-	17.01	-	-	0.00	0.00	0.00	0.00	SMT	No	Our Baseline with Preordering
80	Sense	ja-en	2015/09/01 17:42:28	860	-	-	-	16.96	-	-	0.00	0.00	0.00	0.00	SMT	Yes	Passive JSTx1
81	ORGANIZER	ja-en	2016/07/26 11:37:38	1042	-	-	-	16.91	-	-	-	0.00	0.00	0.00	Other	Yes	Online D (2016)
82	ORGANIZER	ja-en	2015/08/25 18:57:25	775	-	-	-	16.85	-	-	0.00	0.00	0.00	0.00	Other	Yes	Online D (2015)
83	Sense	ja-en	2015/08/18 21:54:39	702	-	-	-	16.72	-	-	0.00	0.00	0.00	0.00	SMT	Yes	Passive JSTx1
84	Sense	ja-en	2015/09/01 17:42:58	861	-	-	-	16.61	-	-	0.00	0.00	0.00	0.00	SMT	Yes	Pervasive JSTx1
85	Sense	ja-en	2015/08/18 21:58:08	708	-	-	-	16.49	-	-	0.00	0.00	0.00	0.00	SMT	Yes	Pervasive JSTx1
86	TMU	ja-en	2014/09/07 23:32:49	301	-	-	-	15.95	-	-	0.00	0.00	0.00	0.00	SMT	No	Our baseline system with another preordering method
87	TMU	ja-en	2015/09/01 05:46:50	847	-	-	-	15.85	-	-	0.00	0.00	0.00	0.00	SMT	No	PBSMT with dependency based phrase segmentation
88	TOSHIBA	ja-en	2014/08/29 18:47:44	240	-	-	-	15.69	-	-	0.00	0.00	0.00	0.00	RBMT	Yes	RBMT system
89	TMU	ja-en	2014/09/07 23:28:04	300	-	-	-	15.55	-	-	0.00	0.00	0.00	0.00	SMT	No	Our baseline system with preordering method
90	TMU	ja-en	2014/09/09 19:14:42	307	-	-	-	15.40	-	-	0.00	0.00	0.00	0.00	SMT	No	Our baseline system
91	ORGANIZER	ja-en	2014/07/23 14:52:31	96	-	-	-	15.29	-	-	0.00	0.00	0.00	0.00	Other	Yes	RBMT D (2014)
92	ORGANIZER	ja-en	2015/09/10 14:38:13	887	-	-	-	15.29	-	-	0.00	0.00	0.00	0.00	Other	Yes	RBMT D (2015)
93	ORGANIZER	ja-en	2014/07/18 11:08:13	35	-	-	-	15.08	-	-	0.00	0.00	0.00	0.00	Other	Yes	Online D (2014)
94	ORGANIZER	ja-en	2014/07/21 11:53:48	76	-	-	-	14.82	-	-	0.00	0.00	0.00	0.00	Other	Yes	RBMT E
95	Osaka-U	ja-en	2018/09/16 13:11:48	2472	-	-	-	13.97	-	-	-	-	0.00	0.00	SMT	No	preordering with neural network
96	ORGANIZER	ja-en	2014/07/21 11:57:08	79	-	-	-	13.86	-	-	0.00	0.00	0.00	0.00	Other	Yes	RBMT F
97	ORGANIZER	ja-en	2014/07/22 11:22:40	87	-	-	-	10.64	-	-	0.00	0.00	0.00	0.00	Other	Yes	Online C (2014)
98	ORGANIZER	ja-en	2015/09/11 10:54:33	892	-	-	-	10.29	-	-	0.00	0.00	0.00	0.00	Other	Yes	Online C (2015)

Notice:

This table is sorted by the leftmost segmenters. You can change the segmenter used to sort by clicking each segmenter link.

RIBES

#	Team	Task	Date/Time	DataID	RIBES										Method	Other Resources	System Description
#	Team	Task	Date/Time	DataID	juman	kytea	mecab	moses- tokenizer	stanford- segmenter- ctb	stanford- segmenter- pku	indic- tokenizer	unuse	myseg	kmseg	Method	Other Resources	System Description
1	srcb	ja-en	2019/07/27 15:27:01	3205	-	-	-	0.778832	-	-	-	-	-	-	NMT	No	Transformer (Big) with relative position, data augmentation, average checkpoints, Bayes re-ranking.
2	srcb	ja-en	2018/09/16 14:51:47	2474	-	-	-	0.777896	-	-	-	-	0.000000	0.000000	NMT	No	Transformer with relative position, ensemble of 3 models.
3	srcb	ja-en	2019/07/25 11:52:48	2919	-	-	-	0.777801	-	-	-	-	-	-	NMT	No	Transformer (Big) with relative position, average checkpoints.
4	NICT-5	ja-en	2018/09/10 14:55:37	2273	-	-	-	0.774788	-	-	-	-	0.000000	0.000000	NMT	No	MLNMT
5	KNU_Hyundai	ja-en	2019/07/27 09:28:37	3173	-	-	-	0.774653	-	-	-	-	-	-	NMT	No	Transformer Base, relative position, BT, r2l reranking, ensemble of 3 models
6	NTT	ja-en	2019/07/28 11:18:59	3225	-	-	-	0.773281	-	-	-	-	-	-	SMT	No	ASPEC first 1.5M ＋last 1.5M (fwd), 6 ensemble
7	NICT-5	ja-en	2019/07/16 17:11:38	2718	-	-	-	0.772422	-	-	-	-	-	-	NMT	No	RSNMT 6 layer with distillation
8	NTT	ja-en	2019/07/28 15:11:04	3233	-	-	-	0.770096	-	-	-	-	-	-	NMT	Yes	ParaCrawl + (ASPEC first 1.5M + Synthetic 1.5M) * 2 oversampling, fine-tune ASPEC, SINGLE MODEL
9	NTT	ja-en	2017/08/01 15:53:15	1732	-	-	-	0.769430	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from training data)
10	AISTAI	ja-en	2019/08/31 21:28:13	3361	-	-	-	0.769105	-	-	-	-	-	-	NMT	No	Transformer, 1.5M sentences, relative position, ensemble of 4 models, by OpenNMT-py.
11	ykkd	ja-en	2019/07/26 12:26:03	2989	-	-	-	0.769061	-	-	-	-	-	-	NMT	No	Fully Character-level,6 bi-LSTM (512*2) for Encoder, 6 LSTM for Decoder. Middle dense layer. Beam w/4.Length norm set to 0.2
12	NTT	ja-en	2017/08/01 04:29:02	1681	-	-	-	0.768880	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
13	NICT-5	ja-en	2021/03/18 23:22:53	4574	-	-	-	0.768617	-	-	-	-	-	-	NMT	No	My NMT implementation. Beam size 8. LP 0.6
14	NAIST	ja-en	2016/08/27 00:05:38	1275	-	-	-	0.767661	-	-	-	0.000000	0.000000	0.000000	SMT	No	Neural MT w/ Lexicon and MinRisk Training 6 Ensemble
15	ORGANIZER	ja-en	2017/08/02 01:03:08	1736	-	-	-	0.767577	-	-	-	0.000000	0.000000	0.000000	NMT	No	Google's "Attention Is All You Need"
16	srcb	ja-en	2018/08/26 11:09:50	2152	-	-	-	0.767194	-	-	-	-	0.000000	0.000000	NMT	No	Transformer, average checkpoints.
17	NICT-5	ja-en	2018/08/27 15:01:05	2174	-	-	-	0.765933	-	-	-	-	0.000000	0.000000	NMT	No	Transformer vanilla model using 3M sentences.
18	Kyoto-U	ja-en	2017/08/01 16:38:21	1733	-	-	-	0.765464	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble of 4 BPE, averaged Coverage penalty
19	ORGANIZER	ja-en	2018/08/14 11:07:47	1901	-	-	-	0.764968	-	-	-	-	0.000000	0.000000	NMT	No	NMT with Attention
20	NTT	ja-en	2017/07/30 20:43:07	1616	-	-	-	0.764831	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
21	NICT-5	ja-en	2019/07/26 18:02:47	3041	-	-	-	0.764672	-	-	-	-	-	-	NMT	No	RSNMT 6 layer
22	NTT	ja-en	2017/08/01 14:50:51	1724	-	-	-	0.763248	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
23	NAIST	ja-en	2016/08/09 16:14:05	1122	-	-	-	0.762712	-	-	-	0.000000	0.000000	0.000000	SMT	No	Neural MT w/ Lexicon and MinRisk Training 4 Ensemble
24	TMU	ja-en	2018/09/16 12:04:36	2464	-	-	-	0.761450	-	-	-	-	0.000000	0.000000	NMT	No	Ensemble of 6 Baseline-NMT
25	Kyoto-U	ja-en	2017/08/01 13:42:25	1717	-	-	-	0.761403	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble of 4 BPE averaged parameters
26	NICT-2	ja-en	2019/07/26 22:37:22	3086	-	-	-	0.760796	-	-	-	-	-	-	NMT	No	Transformer, ensemble of 4 models w/ long warm-up and self-training
27	AISTAI	ja-en	2019/08/01 10:44:49	3260	-	-	-	0.760656	-	-	-	-	-	-	NMT	No	Transformer (big), 1.5M sentences, train_steps=300000, Averaged the last 10 ckpts, by Tensor2Tensor.
28	TMU	ja-en	2018/09/16 17:02:15	2483	-	-	-	0.759790	-	-	-	-	0.000000	0.000000	NMT	No	Ensemble of 6 NMT ( 2 Baseline + 2 Reconstructor + 2 GAN )
29	TMU	ja-en	2018/09/16 12:06:39	2466	-	-	-	0.759238	-	-	-	-	0.000000	0.000000	NMT	No	Reconstructor-NMT ( Single )
30	TMU	ja-en	2018/09/16 11:53:24	2461	-	-	-	0.757955	-	-	-	-	0.000000	0.000000	NMT	No	Baseline-NMT ( Single )
31	TMU	ja-en	2018/09/16 12:05:47	2465	-	-	-	0.757413	-	-	-	-	0.000000	0.000000	NMT	No	GAN-NMT ( Single )
32	NAIST	ja-en	2016/08/20 15:33:12	1247	-	-	-	0.756956	-	-	-	0.000000	0.000000	0.000000	SMT	No	Neural MT w/ Lexicon 6 Ensemble
33	Kyoto-U	ja-en	2016/08/18 15:17:05	1182	-	-	-	0.756601	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble of 4 single-layer model (30k voc)
34	NICT-2	ja-en	2019/07/26 22:33:30	3085	-	-	-	0.756346	-	-	-	-	-	-	NMT	No	Transformer, sigle model w/ long warm-up and self-training
35	Kyoto-U	ja-en	2016/08/20 15:07:47	1246	-	-	-	0.750802	-	-	-	0.000000	0.000000	0.000000	NMT	No	voc src:200k voc tgt: 52k + BPE 2-layer self-ensembling
36	Osaka-U	ja-en	2018/09/15 23:05:12	2440	-	-	-	0.749825	-	-	-	-	0.000000	0.000000	NMT	Yes	rewarding model
37	NAIST	ja-en	2015/08/14 17:46:43	655	-	-	-	0.749573	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System with NeuralMT Reranking and Parser Self Training
38	NICT-2	ja-en	2017/07/26 13:54:28	1476	-	-	-	0.747335	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
39	TMU	ja-en	2017/08/04 11:16:36	1750	-	-	-	0.744928	-	-	-	0.000000	0.000000	0.000000	NMT	No	beam_size: 10, ensemble of different dropout rates.
40	NAIST	ja-en	2015/08/24 23:53:53	757	-	-	-	0.743771	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System with NeuralMT Reranking
41	CUNI	ja-en	2017/07/31 22:30:52	1665	-	-	-	0.741699	-	-	-	0.000000	0.000000	0.000000	NMT	No	Bahdanau (2014) seq2seq with conditional GRU on byte-pair encoding, 1M sentences
42	NICT-2	ja-en	2017/07/26 14:04:38	1480	-	-	-	0.741329	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT 6 Ensembles * Bi-directional Reranking
43	TMU	ja-en	2017/08/01 11:35:08	1703	-	-	-	0.741175	-	-	-	0.000000	0.000000	0.000000	NMT	No	baseline system with beam20
44	TMU	ja-en	2017/08/01 12:14:41	1712	-	-	-	0.735908	-	-	-	0.000000	0.000000	0.000000	NMT	No	the ensemble system of different dropout rate.
45	ORGANIZER	ja-en	2016/11/16 10:29:51	1333	-	-	-	0.733483	-	-	-	0.000000	0.000000	0.000000	NMT	Yes	Online D (2016/11/14)
46	TMU	ja-en	2017/08/01 11:10:49	1695	-	-	-	0.725284	-	-	-	0.000000	0.000000	0.000000	NMT	No	our baseline system in 2017
47	Kyoto-U	ja-en	2015/08/30 13:02:18	829	-	-	-	0.724555	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking
48	NAIST	ja-en	2014/08/01 17:35:16	125	-	-	-	0.723670	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
49	NAIST	ja-en	2014/07/31 11:40:53	119	-	-	-	0.723541	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar-based Forest-to-String SMT System
50	NAIST	ja-en	2015/08/25 13:02:45	766	-	-	-	0.722798	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System with Parser Self Training
51	NAIST	ja-en	2014/07/19 01:04:48	46	-	-	-	0.722599	-	-	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Travatar-based Forest-to-String SMT System with Extra Dictionaries
52	TOSHIBA	ja-en	2015/07/28 16:44:27	529	-	-	-	0.718540	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
53	TOSHIBA	ja-en	2015/07/23 15:00:12	506	-	-	-	0.715795	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	System combination SMT and RBMT(SPE) with RNNLM language model
54	NAIST	ja-en	2015/08/25 13:03:48	767	-	-	-	0.713083	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System Baseline
55	TMU	ja-en	2016/08/20 14:31:48	1234	-	-	-	0.711542	-	-	-	0.000000	0.000000	0.000000	NMT	No	6 ensemble
56	TMU	ja-en	2016/08/20 07:39:02	1222	-	-	-	0.710613	-	-	-	0.000000	0.000000	0.000000	NMT	No	2016 our proposed method to control output voice
57	NICT-2	ja-en	2016/08/05 17:50:25	1104	-	-	-	0.708808	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
58	TOSHIBA	ja-en	2014/08/29 18:48:24	241	-	-	-	0.707936	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
59	EIWA	ja-en	2014/07/30 16:07:14	116	-	-	-	0.706686	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	Combination of RBMT and SPE(statistical post editing)
60	Kyoto-U	ja-en	2015/08/27 14:40:32	796	-	-	-	0.706480	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT system without reranking
61	Kyoto-U	ja-en	2016/08/19 01:31:01	1189	-	-	-	0.705700	-	-	-	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT 2016 w/o reranking
62	Kyoto-U	ja-en	2014/08/31 23:36:50	256	-	-	-	0.701154	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our new baseline system after several modifications.
63	Kyoto-U	ja-en	2014/09/01 10:27:54	262	-	-	-	0.698953	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
64	bjtu_nlp	ja-en	2016/08/17 19:51:24	1168	-	-	-	0.690455	-	-	-	0.000000	0.000000	0.000000	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
65	Kyoto-U	ja-en	2014/08/19 10:24:06	136	-	-	-	0.689829	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our baseline system using 3M parallel sentences.
66	TOSHIBA	ja-en	2014/08/29 18:47:44	240	-	-	-	0.687122	-	-	0.000000	0.000000	0.000000	0.000000	RBMT	Yes	RBMT system
67	NICT	ja-en	2015/07/17 11:02:10	492	-	-	-	0.684485	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	our baseline (DL=9) + reverse pre-reordering [Katz-Brown & Collins 2008]
68	ORGANIZER	ja-en	2014/07/23 14:52:31	96	-	-	-	0.683378	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT D (2014)
69	ORGANIZER	ja-en	2015/09/10 14:38:13	887	-	-	-	0.683378	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT D (2015)
70	ORGANIZER	ja-en	2014/07/11 19:59:55	9	-	-	-	0.678253	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	String-to-Tree SMT (2014)
71	ORGANIZER	ja-en	2015/09/10 13:41:03	877	-	-	-	0.678253	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	String-to-Tree SMT (2015)
72	ORGANIZER	ja-en	2016/07/26 11:37:38	1042	-	-	-	0.677412	-	-	-	0.000000	0.000000	0.000000	Other	Yes	Online D (2016)
73	ORGANIZER	ja-en	2015/08/25 18:57:25	775	-	-	-	0.676609	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online D (2015)
74	Osaka-U	ja-en	2018/09/16 13:11:48	2472	-	-	-	0.665391	-	-	-	-	0.000000	0.000000	SMT	No	preordering with neural network
75	ORGANIZER	ja-en	2014/07/21 11:53:48	76	-	-	-	0.663851	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT E
76	ORGANIZER	ja-en	2014/07/21 11:57:08	79	-	-	-	0.661387	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT F
77	NICT	ja-en	2015/07/16 13:27:58	488	-	-	-	0.659883	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	our baseline (DL=6) + dependency-based pre-reordering [Ding+ 2015]
78	ORGANIZER	ja-en	2014/07/11 19:45:32	2	-	-	-	0.651066	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Hierarchical Phrase-based SMT (2014)
79	TMU	ja-en	2014/09/07 23:32:49	301	-	-	-	0.648879	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Our baseline system with another preordering method
80	Sense	ja-en	2014/08/23 05:34:03	164	-	-	-	0.646204	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Paraphrase max10
81	ORGANIZER	ja-en	2014/07/11 19:49:57	6	-	-	-	0.645137	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Phrase-based SMT
82	TMU	ja-en	2014/09/07 23:28:04	300	-	-	-	0.644698	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Our baseline system with preordering method
83	ORGANIZER	ja-en	2014/07/18 11:08:13	35	-	-	-	0.643588	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online D (2014)
84	TMU	ja-en	2015/08/04 16:32:20	578	-	-	-	0.641456	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Our PBSMT baseline (2015)
85	NICT	ja-en	2015/07/17 08:51:45	489	-	-	-	0.639711	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	our baseline: PB SMT in MOSES (DL=20) / SRILM / MeCab (IPA)
86	Sense	ja-en	2015/08/29 04:32:36	824	-	-	-	0.633073	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-2015 (train123)
87	NII	ja-en	2014/09/02 11:42:01	271	-	-	-	0.630825	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Our Baseline
88	Sense	ja-en	2015/08/28 19:25:25	822	-	-	-	0.629066	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-2015 (train1 only)
89	TMU	ja-en	2015/09/01 05:46:50	847	-	-	-	0.628897	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	PBSMT with dependency based phrase segmentation
90	Sense	ja-en	2015/07/28 22:23:43	530	-	-	-	0.627006	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-2015
91	ORGANIZER	ja-en	2014/07/22 11:22:40	87	-	-	-	0.624827	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online C (2014)
92	ORGANIZER	ja-en	2015/09/11 10:54:33	892	-	-	-	0.622564	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online C (2015)
93	TMU	ja-en	2014/09/09 19:14:42	307	-	-	-	0.613119	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Our baseline system
94	NII	ja-en	2014/09/02 11:42:53	272	-	-	-	0.610833	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Our Baseline with Preordering
95	Sense	ja-en	2015/09/01 17:42:28	860	-	-	-	0.610775	-	-	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Passive JSTx1
96	Sense	ja-en	2015/08/18 21:54:39	702	-	-	-	0.609632	-	-	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Passive JSTx1
97	Sense	ja-en	2015/09/01 17:42:58	861	-	-	-	0.609008	-	-	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Pervasive JSTx1
98	Sense	ja-en	2015/08/18 21:58:08	708	-	-	-	0.600806	-	-	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Pervasive JSTx1

Notice:

This table is sorted by the leftmost segmenters. You can change the segmenter used to sort by clicking each segmenter link.

AMFM

#	Team	Task	Date/Time	DataID	AMFM										Method	Other Resources	System Description
#	Team	Task	Date/Time	DataID	unuse	unuse	unuse	unuse	unuse	unuse	unuse	unuse	unuse	unuse	Method	Other Resources	System Description
1	srcb	ja-en	2019/07/27 15:27:01	3205	-	-	-	0.630150	-	-	-	-	-	-	NMT	No	Transformer (Big) with relative position, data augmentation, average checkpoints, Bayes re-ranking.
2	srcb	ja-en	2019/07/25 11:52:48	2919	-	-	-	0.628070	-	-	-	-	-	-	NMT	No	Transformer (Big) with relative position, average checkpoints.
3	NTT	ja-en	2019/07/28 11:18:59	3225	-	-	-	0.626880	-	-	-	-	-	-	SMT	No	ASPEC first 1.5M ＋last 1.5M (fwd), 6 ensemble
4	AISTAI	ja-en	2019/08/31 21:28:13	3361	-	-	-	0.626300	-	-	-	-	-	-	NMT	No	Transformer, 1.5M sentences, relative position, ensemble of 4 models, by OpenNMT-py.
5	NTT	ja-en	2019/07/28 15:11:04	3233	-	-	-	0.626260	-	-	-	-	-	-	NMT	Yes	ParaCrawl + (ASPEC first 1.5M + Synthetic 1.5M) * 2 oversampling, fine-tune ASPEC, SINGLE MODEL
6	KNU_Hyundai	ja-en	2019/07/27 09:28:37	3173	-	-	-	0.622070	-	-	-	-	-	-	NMT	No	Transformer Base, relative position, BT, r2l reranking, ensemble of 3 models
7	AISTAI	ja-en	2019/08/01 10:44:49	3260	-	-	-	0.620640	-	-	-	-	-	-	NMT	No	Transformer (big), 1.5M sentences, train_steps=300000, Averaged the last 10 ckpts, by Tensor2Tensor.
8	ykkd	ja-en	2019/07/26 12:26:03	2989	-	-	-	0.619640	-	-	-	-	-	-	NMT	No	Fully Character-level,6 bi-LSTM (512*2) for Encoder, 6 LSTM for Decoder. Middle dense layer. Beam w/4.Length norm set to 0.2
9	srcb	ja-en	2018/09/16 14:51:47	2474	-	-	-	0.619390	-	-	-	-	0.000000	0.000000	NMT	No	Transformer with relative position, ensemble of 3 models.
10	NICT-5	ja-en	2019/07/16 17:11:38	2718	-	-	-	0.619040	-	-	-	-	-	-	NMT	No	RSNMT 6 layer with distillation
11	NICT-5	ja-en	2018/09/10 14:55:37	2273	-	-	-	0.612060	-	-	-	-	0.000000	0.000000	NMT	No	MLNMT
12	NAIST	ja-en	2015/08/14 17:46:43	655	0.000000	0.000000	0.000000	0.609430	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System with NeuralMT Reranking and Parser Self Training
13	NICT-5	ja-en	2019/07/26 18:02:47	3041	-	-	-	0.608450	-	-	-	-	-	-	NMT	No	RSNMT 6 layer
14	NICT-5	ja-en	2018/08/27 15:01:05	2174	-	-	-	0.608070	-	-	-	-	0.000000	0.000000	NMT	No	Transformer vanilla model using 3M sentences.
15	NAIST	ja-en	2015/08/24 23:53:53	757	0.000000	0.000000	0.000000	0.606600	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System with NeuralMT Reranking
16	NICT-2	ja-en	2019/07/26 22:37:22	3086	-	-	-	0.606190	-	-	-	-	-	-	NMT	No	Transformer, ensemble of 4 models w/ long warm-up and self-training
17	srcb	ja-en	2018/08/26 11:09:50	2152	-	-	-	0.605130	-	-	-	-	0.000000	0.000000	NMT	No	Transformer, average checkpoints.
18	TOSHIBA	ja-en	2015/07/23 15:00:12	506	0.000000	0.000000	0.000000	0.604760	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	System combination SMT and RBMT(SPE) with RNNLM language model
19	NAIST	ja-en	2014/07/19 01:04:48	46	0.000000	0.000000	0.000000	0.604180	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Travatar-based Forest-to-String SMT System with Extra Dictionaries
20	NAIST	ja-en	2014/07/31 11:40:53	119	0.000000	0.000000	0.000000	0.603490	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar-based Forest-to-String SMT System
21	Kyoto-U	ja-en	2015/08/30 13:02:18	829	0.000000	0.000000	0.000000	0.603210	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking
22	NAIST	ja-en	2014/08/01 17:35:16	125	0.000000	0.000000	0.000000	0.602780	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
23	NICT-2	ja-en	2019/07/26 22:33:30	3085	-	-	-	0.602770	-	-	-	-	-	-	NMT	No	Transformer, sigle model w/ long warm-up and self-training
24	NICT-5	ja-en	2021/03/18 23:22:53	4574	-	-	-	0.601460	-	-	-	-	-	-	NMT	No	My NMT implementation. Beam size 8. LP 0.6
25	TMU	ja-en	2018/09/16 12:04:36	2464	-	-	-	0.600730	-	-	-	-	0.000000	0.000000	NMT	No	Ensemble of 6 Baseline-NMT
26	NAIST	ja-en	2015/08/25 13:03:48	767	0.000000	0.000000	0.000000	0.600000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System Baseline
27	NTT	ja-en	2017/08/01 15:53:15	1732	-	-	-	0.599920	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from training data)
28	NAIST	ja-en	2015/08/25 13:02:45	766	0.000000	0.000000	0.000000	0.599800	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System with Parser Self Training
29	TMU	ja-en	2018/09/16 12:06:39	2466	-	-	-	0.599110	-	-	-	-	0.000000	0.000000	NMT	No	Reconstructor-NMT ( Single )
30	TMU	ja-en	2018/09/16 17:02:15	2483	-	-	-	0.598770	-	-	-	-	0.000000	0.000000	NMT	No	Ensemble of 6 NMT ( 2 Baseline + 2 Reconstructor + 2 GAN )
31	NTT	ja-en	2017/08/01 04:29:02	1681	-	-	-	0.597860	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
32	TOSHIBA	ja-en	2015/07/28 16:44:27	529	0.000000	0.000000	0.000000	0.597830	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
33	NTT	ja-en	2017/07/30 20:43:07	1616	-	-	-	0.597620	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
34	NTT	ja-en	2017/08/01 14:50:51	1724	-	-	-	0.597470	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
35	TMU	ja-en	2018/09/16 11:53:24	2461	-	-	-	0.596590	-	-	-	-	0.000000	0.000000	NMT	No	Baseline-NMT ( Single )
36	Kyoto-U	ja-en	2015/08/27 14:40:32	796	0.000000	0.000000	0.000000	0.596430	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT system without reranking
37	TMU	ja-en	2017/08/04 11:16:36	1750	-	-	-	0.596360	-	-	-	0.000000	0.000000	0.000000	NMT	No	beam_size: 10, ensemble of different dropout rates.
38	NICT-2	ja-en	2016/08/05 17:50:25	1104	-	-	-	0.595930	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
39	TMU	ja-en	2018/09/16 12:05:47	2465	-	-	-	0.595850	-	-	-	-	0.000000	0.000000	NMT	No	GAN-NMT ( Single )
40	ORGANIZER	ja-en	2017/08/02 01:03:08	1736	-	-	-	0.595580	-	-	-	0.000000	0.000000	0.000000	NMT	No	Google's "Attention Is All You Need"
41	ORGANIZER	ja-en	2018/08/14 11:07:47	1901	-	-	-	0.595370	-	-	-	-	0.000000	0.000000	NMT	No	NMT with Attention
42	TMU	ja-en	2017/08/01 11:35:08	1703	-	-	-	0.595260	-	-	-	0.000000	0.000000	0.000000	NMT	No	baseline system with beam20
43	Kyoto-U	ja-en	2016/08/19 01:31:01	1189	-	-	-	0.595240	-	-	-	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT 2016 w/o reranking
44	NAIST	ja-en	2016/08/27 00:05:38	1275	-	-	-	0.594150	-	-	-	0.000000	0.000000	0.000000	SMT	No	Neural MT w/ Lexicon and MinRisk Training 6 Ensemble
45	Kyoto-U	ja-en	2014/08/19 10:24:06	136	0.000000	0.000000	0.000000	0.593970	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our baseline system using 3M parallel sentences.
46	Kyoto-U	ja-en	2014/08/31 23:36:50	256	0.000000	0.000000	0.000000	0.593660	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our new baseline system after several modifications.
47	ORGANIZER	ja-en	2014/07/11 19:59:55	9	0.000000	0.000000	0.000000	0.593410	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	String-to-Tree SMT (2014)
48	ORGANIZER	ja-en	2015/09/10 13:41:03	877	0.000000	0.000000	0.000000	0.593410	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	String-to-Tree SMT (2015)
49	Sense	ja-en	2015/08/28 19:25:25	822	0.000000	0.000000	0.000000	0.592040	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-2015 (train1 only)
50	Kyoto-U	ja-en	2017/08/01 16:38:21	1733	-	-	-	0.591160	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble of 4 BPE, averaged Coverage penalty
51	TMU	ja-en	2015/08/04 16:32:20	578	0.000000	0.000000	0.000000	0.590980	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Our PBSMT baseline (2015)
52	ORGANIZER	ja-en	2014/07/11 19:49:57	6	0.000000	0.000000	0.000000	0.590950	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Phrase-based SMT
53	ORGANIZER	ja-en	2014/07/11 19:45:32	2	0.000000	0.000000	0.000000	0.588880	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Hierarchical Phrase-based SMT (2014)
54	Kyoto-U	ja-en	2014/09/01 10:27:54	262	0.000000	0.000000	0.000000	0.588480	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
55	TMU	ja-en	2017/08/01 12:14:41	1712	-	-	-	0.588360	-	-	-	0.000000	0.000000	0.000000	NMT	No	the ensemble system of different dropout rate.
56	Osaka-U	ja-en	2018/09/15 23:05:12	2440	-	-	-	0.588290	-	-	-	-	0.000000	0.000000	NMT	Yes	rewarding model
57	NICT	ja-en	2015/07/16 13:27:58	488	0.000000	0.000000	0.000000	0.587530	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	our baseline (DL=6) + dependency-based pre-reordering [Ding+ 2015]
58	Sense	ja-en	2014/08/23 05:34:03	164	0.000000	0.000000	0.000000	0.587520	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Paraphrase max10
59	NAIST	ja-en	2016/08/09 16:14:05	1122	-	-	-	0.587450	-	-	-	0.000000	0.000000	0.000000	SMT	No	Neural MT w/ Lexicon and MinRisk Training 4 Ensemble
60	TMU	ja-en	2017/08/01 11:10:49	1695	-	-	-	0.585710	-	-	-	0.000000	0.000000	0.000000	NMT	No	our baseline system in 2017
61	Kyoto-U	ja-en	2017/08/01 13:42:25	1717	-	-	-	0.585540	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble of 4 BPE averaged parameters
62	ORGANIZER	ja-en	2016/11/16 10:29:51	1333	-	-	-	0.584390	-	-	-	0.000000	0.000000	0.000000	NMT	Yes	Online D (2016/11/14)
63	CUNI	ja-en	2017/07/31 22:30:52	1665	-	-	-	0.583780	-	-	-	0.000000	0.000000	0.000000	NMT	No	Bahdanau (2014) seq2seq with conditional GRU on byte-pair encoding, 1M sentences
64	NII	ja-en	2014/09/02 11:42:01	271	0.000000	0.000000	0.000000	0.582800	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Our Baseline
65	Sense	ja-en	2015/09/01 17:42:58	861	0.000000	0.000000	0.000000	0.582370	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Pervasive JSTx1
66	Sense	ja-en	2015/08/18 21:58:08	708	0.000000	0.000000	0.000000	0.581540	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Pervasive JSTx1
67	TMU	ja-en	2014/09/09 19:14:42	307	0.000000	0.000000	0.000000	0.580500	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Our baseline system
68	Sense	ja-en	2015/09/01 17:42:28	860	0.000000	0.000000	0.000000	0.579790	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Passive JSTx1
69	Sense	ja-en	2015/08/29 04:32:36	824	0.000000	0.000000	0.000000	0.579280	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-2015 (train123)
70	Sense	ja-en	2015/08/18 21:54:39	702	0.000000	0.000000	0.000000	0.579210	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Passive JSTx1
71	TMU	ja-en	2014/09/07 23:32:49	301	0.000000	0.000000	0.000000	0.578360	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Our baseline system with another preordering method
72	NICT-2	ja-en	2017/07/26 14:04:38	1480	-	-	-	0.578150	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT 6 Ensembles * Bi-directional Reranking
73	EIWA	ja-en	2014/07/30 16:07:14	116	0.000000	0.000000	0.000000	0.576540	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	Combination of RBMT and SPE(statistical post editing)
74	NICT	ja-en	2015/07/17 11:02:10	492	0.000000	0.000000	0.000000	0.576510	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	our baseline (DL=9) + reverse pre-reordering [Katz-Brown & Collins 2008]
75	NICT-2	ja-en	2017/07/26 13:54:28	1476	-	-	-	0.574810	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
76	NII	ja-en	2014/09/02 11:42:53	272	0.000000	0.000000	0.000000	0.574030	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Our Baseline with Preordering
77	Osaka-U	ja-en	2018/09/16 13:11:48	2472	-	-	-	0.571400	-	-	-	-	0.000000	0.000000	SMT	No	preordering with neural network
78	NAIST	ja-en	2016/08/20 15:33:12	1247	-	-	-	0.571360	-	-	-	0.000000	0.000000	0.000000	SMT	No	Neural MT w/ Lexicon 6 Ensemble
79	TMU	ja-en	2015/09/01 05:46:50	847	0.000000	0.000000	0.000000	0.570430	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	PBSMT with dependency based phrase segmentation
80	TMU	ja-en	2016/08/20 07:39:02	1222	-	-	-	0.565270	-	-	-	0.000000	0.000000	0.000000	NMT	No	2016 our proposed method to control output voice
81	Sense	ja-en	2015/07/28 22:23:43	530	0.000000	0.000000	0.000000	0.564610	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-2015
82	ORGANIZER	ja-en	2016/07/26 11:37:38	1042	-	-	-	0.564270	-	-	-	0.000000	0.000000	0.000000	Other	Yes	Online D (2016)
83	ORGANIZER	ja-en	2014/07/18 11:08:13	35	0.000000	0.000000	0.000000	0.564170	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online D (2014)
84	NICT	ja-en	2015/07/17 08:51:45	489	0.000000	0.000000	0.000000	0.562920	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	our baseline: PB SMT in MOSES (DL=20) / SRILM / MeCab (IPA)
85	Kyoto-U	ja-en	2016/08/20 15:07:47	1246	-	-	-	0.562650	-	-	-	0.000000	0.000000	0.000000	NMT	No	voc src:200k voc tgt: 52k + BPE 2-layer self-ensembling
86	ORGANIZER	ja-en	2015/08/25 18:57:25	775	0.000000	0.000000	0.000000	0.562270	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online D (2015)
87	ORGANIZER	ja-en	2014/07/21 11:53:48	76	0.000000	0.000000	0.000000	0.561620	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT E
88	TMU	ja-en	2014/09/07 23:28:04	300	0.000000	0.000000	0.000000	0.561450	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Our baseline system with preordering method
89	Kyoto-U	ja-en	2016/08/18 15:17:05	1182	-	-	-	0.558540	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble of 4 single-layer model (30k voc)
90	ORGANIZER	ja-en	2014/07/21 11:57:08	79	0.000000	0.000000	0.000000	0.556840	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT F
91	TOSHIBA	ja-en	2014/08/29 18:47:44	240	0.000000	0.000000	0.000000	0.552980	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	RBMT	Yes	RBMT system
92	TOSHIBA	ja-en	2014/08/29 18:48:24	241	0.000000	0.000000	0.000000	0.551740	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
93	ORGANIZER	ja-en	2014/07/23 14:52:31	96	0.000000	0.000000	0.000000	0.551690	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT D (2014)
94	ORGANIZER	ja-en	2015/09/10 14:38:13	887	0.000000	0.000000	0.000000	0.551690	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT D (2015)
95	TMU	ja-en	2016/08/20 14:31:48	1234	-	-	-	0.546880	-	-	-	0.000000	0.000000	0.000000	NMT	No	6 ensemble
96	bjtu_nlp	ja-en	2016/08/17 19:51:24	1168	-	-	-	0.505730	-	-	-	0.000000	0.000000	0.000000	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
97	ORGANIZER	ja-en	2014/07/22 11:22:40	87	0.000000	0.000000	0.000000	0.466480	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online C (2014)
98	ORGANIZER	ja-en	2015/09/11 10:54:33	892	0.000000	0.000000	0.000000	0.453370	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online C (2015)

Notice:

This table is sorted by the leftmost segmenters. You can change the segmenter used to sort by clicking each segmenter link.
Adequacy-Fluency Metrics (AMFM) is a two-dimensional automatic evaluation metric for machine translation, designed to operate at the sentence level. It is based on adequacy and fluency, to decouple semantic and syntactic components of the translation process to provide a balanced view on translation quality.
AMFM is calculated without tokenizers.
The detail of AMFM is shown on the following paper: "Adequacy–Fluency Metrics: Evaluating MT in the Continuous Space Model Framework" [pdf]. Invited Talk in WAT2015 also helps understanding [slide].

HUMAN (WAT2022)

Notice:

HUMAN (WAT2022) is the result of the Pairwise Crowdsourcing Evaluation on WAT2022.
HUMAN (WAT2022) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2021)

Notice:

HUMAN (WAT2021) is the result of the Pairwise Crowdsourcing Evaluation on WAT2021.
HUMAN (WAT2021) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2020)

Notice:

HUMAN (WAT2020) is the result of the Pairwise Crowdsourcing Evaluation on WAT2020.
HUMAN (WAT2020) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2019)

Notice:

HUMAN (WAT2019) is the result of the Pairwise Crowdsourcing Evaluation on WAT2019.
HUMAN (WAT2019) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2018)

Notice:

HUMAN (WAT2018) is the result of the Pairwise Crowdsourcing Evaluation on WAT2018.
HUMAN (WAT2018) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2017)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	Kyoto-U	ja-en	2017/08/01 13:42:25	1717	77.750	NMT	No	Ensemble of 4 BPE averaged parameters
2	NTT	ja-en	2017/08/01 04:29:02	1681	77.250	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
3	ORGANIZER	ja-en	2017/08/02 01:03:08	1736	75.250	NMT	No	Google's "Attention Is All You Need"
4	NTT	ja-en	2017/07/30 20:43:07	1616	75.000	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
5	Kyoto-U	ja-en	2017/08/01 16:38:21	1733	74.500	NMT	No	Ensemble of 4 BPE, averaged Coverage penalty
6	NICT-2	ja-en	2017/07/26 14:04:38	1480	69.750	NMT	No	NMT 6 Ensembles * Bi-directional Reranking
7	NICT-2	ja-en	2017/07/26 13:54:28	1476	68.750	NMT	No	NMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
8	CUNI	ja-en	2017/07/31 22:30:52	1665	66.000	NMT	No	Bahdanau (2014) seq2seq with conditional GRU on byte-pair encoding, 1M sentences
9	TMU	ja-en	2017/08/01 11:35:08	1703	61.000	NMT	No	baseline system with beam20
10	TMU	ja-en	2017/08/01 11:10:49	1695	56.750	NMT	No	our baseline system in 2017

Notice:

HUMAN (WAT2017) is the result of the Pairwise Crowdsourcing Evaluation on WAT2017.
HUMAN (WAT2017) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2016)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	ORGANIZER	ja-en	2016/11/16 10:29:51	1333	63.000	NMT	Yes	Online D (2016/11/14)
2	NAIST	ja-en	2016/08/09 16:14:05	1122	48.250	SMT	No	Neural MT w/ Lexicon and MinRisk Training 4 Ensemble
3	NAIST	ja-en	2016/08/20 15:33:12	1247	47.500	SMT	No	Neural MT w/ Lexicon 6 Ensemble
4	Kyoto-U	ja-en	2016/08/20 15:07:47	1246	47.000	NMT	No	voc src:200k voc tgt: 52k + BPE 2-layer self-ensembling
5	Kyoto-U	ja-en	2016/08/18 15:17:05	1182	44.250	NMT	No	Ensemble of 4 single-layer model (30k voc)
6	ORGANIZER	ja-en	2016/07/26 11:37:38	1042	28.000	Other	Yes	Online D (2016)
7	TMU	ja-en	2016/08/20 14:31:48	1234	25.000	NMT	No	6 ensemble
8	bjtu_nlp	ja-en	2016/08/17 19:51:24	1168	19.250	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
9	TMU	ja-en	2016/08/20 07:39:02	1222	16.000	NMT	No	2016 our proposed method to control output voice
10	NICT-2	ja-en	2016/08/05 17:50:25	1104	Underway	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM

Notice:

HUMAN (WAT2016) is the result of the Pairwise Crowdsourcing Evaluation on WAT2016.
HUMAN (WAT2016) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2015)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	NAIST	ja-en	2015/08/14 17:46:43	655	35.500	SMT	No	Travatar System with NeuralMT Reranking and Parser Self Training
2	Kyoto-U	ja-en	2015/08/30 13:02:18	829	32.500	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking
3	TOSHIBA	ja-en	2015/07/28 16:44:27	529	25.000	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
4	TOSHIBA	ja-en	2015/07/23 15:00:12	506	21.250	SMT and RBMT	Yes	System combination SMT and RBMT(SPE) with RNNLM language model
5	ORGANIZER	ja-en	2015/09/10 14:38:13	887	16.750	Other	Yes	RBMT D (2015)
6	Kyoto-U	ja-en	2015/08/27 14:40:32	796	16.500	EBMT	No	KyotoEBMT system without reranking
7	NICT	ja-en	2015/07/16 13:27:58	488	16.000	SMT	No	our baseline (DL=6) + dependency-based pre-reordering [Ding+ 2015]
8	NAIST	ja-en	2015/08/25 13:02:45	766	11.750	SMT	No	Travatar System with Parser Self Training
9	ORGANIZER	ja-en	2015/09/10 13:41:03	877	7.000	SMT	No	String-to-Tree SMT (2015)
10	NICT	ja-en	2015/07/17 11:02:10	492	6.500	SMT	No	our baseline (DL=9) + reverse pre-reordering [Katz-Brown & Collins 2008]
11	ORGANIZER	ja-en	2015/08/25 18:57:25	775	0.250	Other	Yes	Online D (2015)
12	Sense	ja-en	2015/09/01 17:42:28	860	-7.750	SMT	Yes	Passive JSTx1
13	Sense	ja-en	2015/09/01 17:42:58	861	-12.750	SMT	Yes	Pervasive JSTx1
14	TMU	ja-en	2015/09/01 05:46:50	847	-25.500	SMT	No	PBSMT with dependency based phrase segmentation

Notice:

HUMAN (WAT2015) is the result of the Pairwise Crowdsourcing Evaluation on WAT2015.
HUMAN (WAT2015) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.
The detail of the evaluation can be found in the pdf document (PDF file).

HUMAN (WAT2014)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	NAIST	ja-en	2014/07/19 01:04:48	46	40.500	SMT	Yes	Travatar-based Forest-to-String SMT System with Extra Dictionaries
2	NAIST	ja-en	2014/07/31 11:40:53	119	37.500	SMT	No	Travatar-based Forest-to-String SMT System
3	ORGANIZER	ja-en	2014/07/11 19:59:55	9	25.500	SMT	No	String-to-Tree SMT (2014)
4	Kyoto-U	ja-en	2014/09/01 10:27:54	262	25.000	EBMT	No	Our new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
5	TOSHIBA	ja-en	2014/08/29 18:48:24	241	23.250	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
6	ORGANIZER	ja-en	2014/07/23 14:52:31	96	23.000	Other	Yes	RBMT D (2014)
7	EIWA	ja-en	2014/07/30 16:07:14	116	22.500	SMT and RBMT	Yes	Combination of RBMT and SPE(statistical post editing)
8	Kyoto-U	ja-en	2014/08/31 23:36:50	256	21.250	EBMT	No	Our new baseline system after several modifications.
9	TOSHIBA	ja-en	2014/08/29 18:47:44	240	20.250	RBMT	Yes	RBMT system
10	ORGANIZER	ja-en	2014/07/18 11:08:13	35	13.750	Other	Yes	Online D (2014)
11	ORGANIZER	ja-en	2014/07/11 19:45:32	2	7.750	SMT	No	Hierarchical Phrase-based SMT (2014)
12	Sense	ja-en	2014/08/23 05:34:03	164	1.250	SMT	No	Paraphrase max10
13	NII	ja-en	2014/09/02 11:42:01	271	-5.750	SMT	No	Our Baseline
14	NII	ja-en	2014/09/02 11:42:53	272	-14.250	SMT	No	Our Baseline with Preordering
15	TMU	ja-en	2014/09/07 23:32:49	301	-17.000	SMT	No	Our baseline system with another preordering method
16	TMU	ja-en	2014/09/07 23:28:04	300	-17.250	SMT	No	Our baseline system with preordering method

Notice:

HUMAN (WAT2014) is the result of the Pairwise Crowdsourcing Evaluation on WAT2014.
HUMAN (WAT2014) was evaluated by 3 different workers and the final decision is made by the voting of the judgements.
The detail of the evaluation can be found in the pdf document (PDF file).

EVALUATION RESULTS USAGE POLICY

When you use the WAT evaluation results for any purpose such as:
- writing technical papers,
- making presentations about your system,
- advertising your MT system to the customers,
you can use the information about translation directions, scores (including both automatic and human evaluations) and ranks of your system among others. You can also use the scores of the other systems, but you MUST anonymize the other system's names. In addition, you can show the links (URLs) to the WAT evaluation result pages.

NICT (National Institute of Information and Communications Technology)
Kyoto University
Last Modified: 2018-08-02

WAT The Workshop on Asian Translation Evaluation Results

BLEU

RIBES

AMFM

HUMAN (WAT2022)

HUMAN (WAT2021)

HUMAN (WAT2020)

HUMAN (WAT2019)

HUMAN (WAT2018)

HUMAN (WAT2017)

HUMAN (WAT2016)

HUMAN (WAT2015)

HUMAN (WAT2014)

EVALUATION RESULTS USAGE POLICY

WAT

The Workshop on Asian Translation

Evaluation Results