WAT

The Workshop on Asian Translation

Evaluation Results

BLEU

#	Team	Task	Date/Time	DataID	BLEU										Method	Other Resources	System Description
#	Team	Task	Date/Time	DataID	juman	kytea	mecab	moses- tokenizer	stanford- segmenter- ctb	stanford- segmenter- pku	indic- tokenizer	unuse	myseg	kmseg	Method	Other Resources	System Description
1	NICT-5	ja-en	2019/07/16 17:11:38	2718	-	-	-	29.12	-	-	-	-	-	-	NMT	No	RSNMT 6 layer with distillation
2	srcb	ja-en	2019/07/25 11:52:48	2919	-	-	-	30.62	-	-	-	-	-	-	NMT	No	Transformer (Big) with relative position, average checkpoints.
3	ykkd	ja-en	2019/07/26 12:26:03	2989	-	-	-	27.63	-	-	-	-	-	-	NMT	No	Fully Character-level,6 bi-LSTM (512*2) for Encoder, 6 LSTM for Decoder. Middle dense layer. Beam w/4.Length norm set to 0.2
4	NICT-5	ja-en	2019/07/26 18:02:47	3041	-	-	-	26.99	-	-	-	-	-	-	NMT	No	RSNMT 6 layer
5	NICT-2	ja-en	2019/07/26 22:33:30	3085	-	-	-	28.61	-	-	-	-	-	-	NMT	No	Transformer, sigle model w/ long warm-up and self-training
6	NICT-2	ja-en	2019/07/26 22:37:22	3086	-	-	-	29.40	-	-	-	-	-	-	NMT	No	Transformer, ensemble of 4 models w/ long warm-up and self-training
7	KNU_Hyundai	ja-en	2019/07/27 09:28:37	3173	-	-	-	30.88	-	-	-	-	-	-	NMT	No	Transformer Base, relative position, BT, r2l reranking, ensemble of 3 models
8	srcb	ja-en	2019/07/27 15:27:01	3205	-	-	-	30.92	-	-	-	-	-	-	NMT	No	Transformer (Big) with relative position, data augmentation, average checkpoints, Bayes re-ranking.
9	NTT	ja-en	2019/07/28 11:18:59	3225	-	-	-	30.56	-	-	-	-	-	-	SMT	No	ASPEC first 1.5M ＋last 1.5M (fwd), 6 ensemble
10	NTT	ja-en	2019/07/28 15:11:04	3233	-	-	-	30.28	-	-	-	-	-	-	NMT	Yes	ParaCrawl + (ASPEC first 1.5M + Synthetic 1.5M) * 2 oversampling, fine-tune ASPEC, SINGLE MODEL
11	AISTAI	ja-en	2019/08/01 10:44:49	3260	-	-	-	29.01	-	-	-	-	-	-	NMT	No	Transformer (big), 1.5M sentences, train_steps=300000, Averaged the last 10 ckpts, by Tensor2Tensor.
12	AISTAI	ja-en	2019/08/31 21:28:13	3361	-	-	-	29.71	-	-	-	-	-	-	NMT	No	Transformer, 1.5M sentences, relative position, ensemble of 4 models, by OpenNMT-py.
13	NICT-5	ja-en	2021/03/18 23:22:53	4574	-	-	-	28.35	-	-	-	-	-	-	NMT	No	My NMT implementation. Beam size 8. LP 0.6
14	ORGANIZER	ja-en	2014/07/11 19:45:32	2	-	-	-	18.72	-	-	0.00	0.00	0.00	0.00	SMT	No	Hierarchical Phrase-based SMT (2014)
15	ORGANIZER	ja-en	2014/07/11 19:49:57	6	-	-	-	18.45	-	-	0.00	0.00	0.00	0.00	SMT	No	Phrase-based SMT
16	ORGANIZER	ja-en	2014/07/11 19:59:55	9	-	-	-	20.36	-	-	0.00	0.00	0.00	0.00	SMT	No	String-to-Tree SMT (2014)
17	ORGANIZER	ja-en	2014/07/18 11:08:13	35	-	-	-	15.08	-	-	0.00	0.00	0.00	0.00	Other	Yes	Online D (2014)
18	NAIST	ja-en	2014/07/19 01:04:48	46	-	-	-	23.82	-	-	0.00	0.00	0.00	0.00	SMT	Yes	Travatar-based Forest-to-String SMT System with Extra Dictionaries
19	ORGANIZER	ja-en	2014/07/21 11:53:48	76	-	-	-	14.82	-	-	0.00	0.00	0.00	0.00	Other	Yes	RBMT E
20	ORGANIZER	ja-en	2014/07/21 11:57:08	79	-	-	-	13.86	-	-	0.00	0.00	0.00	0.00	Other	Yes	RBMT F
21	ORGANIZER	ja-en	2014/07/22 11:22:40	87	-	-	-	10.64	-	-	0.00	0.00	0.00	0.00	Other	Yes	Online C (2014)
22	ORGANIZER	ja-en	2014/07/23 14:52:31	96	-	-	-	15.29	-	-	0.00	0.00	0.00	0.00	Other	Yes	RBMT D (2014)
23	EIWA	ja-en	2014/07/30 16:07:14	116	-	-	-	19.86	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	Combination of RBMT and SPE(statistical post editing)
24	NAIST	ja-en	2014/07/31 11:40:53	119	-	-	-	23.29	-	-	0.00	0.00	0.00	0.00	SMT	No	Travatar-based Forest-to-String SMT System
25	NAIST	ja-en	2014/08/01 17:35:16	125	-	-	-	23.47	-	-	0.00	0.00	0.00	0.00	SMT	No	Travatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
26	Kyoto-U	ja-en	2014/08/19 10:24:06	136	-	-	-	20.02	-	-	0.00	0.00	0.00	0.00	EBMT	No	Our baseline system using 3M parallel sentences.
27	Sense	ja-en	2014/08/23 05:34:03	164	-	-	-	18.82	-	-	0.00	0.00	0.00	0.00	SMT	No	Paraphrase max10
28	TOSHIBA	ja-en	2014/08/29 18:47:44	240	-	-	-	15.69	-	-	0.00	0.00	0.00	0.00	RBMT	Yes	RBMT system
29	TOSHIBA	ja-en	2014/08/29 18:48:24	241	-	-	-	20.61	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
30	Kyoto-U	ja-en	2014/08/31 23:36:50	256	-	-	-	20.60	-	-	0.00	0.00	0.00	0.00	EBMT	No	Our new baseline system after several modifications.
31	Kyoto-U	ja-en	2014/09/01 10:27:54	262	-	-	-	21.07	-	-	0.00	0.00	0.00	0.00	EBMT	No	Our new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
32	NII	ja-en	2014/09/02 11:42:01	271	-	-	-	17.47	-	-	0.00	0.00	0.00	0.00	SMT	No	Our Baseline
33	NII	ja-en	2014/09/02 11:42:53	272	-	-	-	17.01	-	-	0.00	0.00	0.00	0.00	SMT	No	Our Baseline with Preordering
34	TMU	ja-en	2014/09/07 23:28:04	300	-	-	-	15.55	-	-	0.00	0.00	0.00	0.00	SMT	No	Our baseline system with preordering method
35	TMU	ja-en	2014/09/07 23:32:49	301	-	-	-	15.95	-	-	0.00	0.00	0.00	0.00	SMT	No	Our baseline system with another preordering method
36	TMU	ja-en	2014/09/09 19:14:42	307	-	-	-	15.40	-	-	0.00	0.00	0.00	0.00	SMT	No	Our baseline system
37	NICT	ja-en	2015/07/16 13:27:58	488	-	-	-	18.98	-	-	0.00	0.00	0.00	0.00	SMT	No	our baseline (DL=6) + dependency-based pre-reordering [Ding+ 2015]
38	NICT	ja-en	2015/07/17 08:51:45	489	-	-	-	18.09	-	-	0.00	0.00	0.00	0.00	SMT	No	our baseline: PB SMT in MOSES (DL=20) / SRILM / MeCab (IPA)
39	NICT	ja-en	2015/07/17 11:02:10	492	-	-	-	18.96	-	-	0.00	0.00	0.00	0.00	SMT	No	our baseline (DL=9) + reverse pre-reordering [Katz-Brown & Collins 2008]
40	TOSHIBA	ja-en	2015/07/23 15:00:12	506	-	-	-	23.00	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	System combination SMT and RBMT(SPE) with RNNLM language model
41	TOSHIBA	ja-en	2015/07/28 16:44:27	529	-	-	-	22.89	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
42	Sense	ja-en	2015/07/28 22:23:43	530	-	-	-	17.04	-	-	0.00	0.00	0.00	0.00	SMT	No	Baseline-2015
43	TMU	ja-en	2015/08/04 16:32:20	578	-	-	-	18.32	-	-	0.00	0.00	0.00	0.00	SMT	No	Our PBSMT baseline (2015)
44	NAIST	ja-en	2015/08/14 17:46:43	655	-	-	-	25.41	-	-	0.00	0.00	0.00	0.00	SMT	No	Travatar System with NeuralMT Reranking and Parser Self Training
45	Sense	ja-en	2015/08/18 21:54:39	702	-	-	-	16.72	-	-	0.00	0.00	0.00	0.00	SMT	Yes	Passive JSTx1
46	Sense	ja-en	2015/08/18 21:58:08	708	-	-	-	16.49	-	-	0.00	0.00	0.00	0.00	SMT	Yes	Pervasive JSTx1
47	NAIST	ja-en	2015/08/24 23:53:53	757	-	-	-	24.77	-	-	0.00	0.00	0.00	0.00	SMT	No	Travatar System with NeuralMT Reranking
48	NAIST	ja-en	2015/08/25 13:02:45	766	-	-	-	22.62	-	-	0.00	0.00	0.00	0.00	SMT	No	Travatar System with Parser Self Training
49	NAIST	ja-en	2015/08/25 13:03:48	767	-	-	-	22.16	-	-	0.00	0.00	0.00	0.00	SMT	No	Travatar System Baseline
50	ORGANIZER	ja-en	2015/08/25 18:57:25	775	-	-	-	16.85	-	-	0.00	0.00	0.00	0.00	Other	Yes	Online D (2015)
51	Kyoto-U	ja-en	2015/08/27 14:40:32	796	-	-	-	21.31	-	-	0.00	0.00	0.00	0.00	EBMT	No	KyotoEBMT system without reranking
52	Sense	ja-en	2015/08/28 19:25:25	822	-	-	-	18.20	-	-	0.00	0.00	0.00	0.00	SMT	No	Baseline-2015 (train1 only)
53	Sense	ja-en	2015/08/29 04:32:36	824	-	-	-	18.09	-	-	0.00	0.00	0.00	0.00	SMT	No	Baseline-2015 (train123)
54	Kyoto-U	ja-en	2015/08/30 13:02:18	829	-	-	-	22.89	-	-	0.00	0.00	0.00	0.00	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking
55	TMU	ja-en	2015/09/01 05:46:50	847	-	-	-	15.85	-	-	0.00	0.00	0.00	0.00	SMT	No	PBSMT with dependency based phrase segmentation
56	Sense	ja-en	2015/09/01 17:42:28	860	-	-	-	16.96	-	-	0.00	0.00	0.00	0.00	SMT	Yes	Passive JSTx1
57	Sense	ja-en	2015/09/01 17:42:58	861	-	-	-	16.61	-	-	0.00	0.00	0.00	0.00	SMT	Yes	Pervasive JSTx1
58	ORGANIZER	ja-en	2015/09/10 13:41:03	877	-	-	-	20.36	-	-	0.00	0.00	0.00	0.00	SMT	No	String-to-Tree SMT (2015)
59	ORGANIZER	ja-en	2015/09/10 14:38:13	887	-	-	-	15.29	-	-	0.00	0.00	0.00	0.00	Other	Yes	RBMT D (2015)
60	ORGANIZER	ja-en	2015/09/11 10:54:33	892	-	-	-	10.29	-	-	0.00	0.00	0.00	0.00	Other	Yes	Online C (2015)
61	ORGANIZER	ja-en	2016/07/26 11:37:38	1042	-	-	-	16.91	-	-	-	0.00	0.00	0.00	Other	Yes	Online D (2016)
62	NICT-2	ja-en	2016/08/05 17:50:25	1104	-	-	-	21.54	-	-	-	0.00	0.00	0.00	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
63	NAIST	ja-en	2016/08/09 16:14:05	1122	-	-	-	26.39	-	-	-	0.00	0.00	0.00	SMT	No	Neural MT w/ Lexicon and MinRisk Training 4 Ensemble
64	bjtu_nlp	ja-en	2016/08/17 19:51:24	1168	-	-	-	18.34	-	-	-	0.00	0.00	0.00	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
65	Kyoto-U	ja-en	2016/08/18 15:17:05	1182	-	-	-	26.22	-	-	-	0.00	0.00	0.00	NMT	No	Ensemble of 4 single-layer model (30k voc)
66	Kyoto-U	ja-en	2016/08/19 01:31:01	1189	-	-	-	21.22	-	-	-	0.00	0.00	0.00	EBMT	No	KyotoEBMT 2016 w/o reranking
67	TMU	ja-en	2016/08/20 07:39:02	1222	-	-	-	18.29	-	-	-	0.00	0.00	0.00	NMT	No	2016 our proposed method to control output voice
68	TMU	ja-en	2016/08/20 14:31:48	1234	-	-	-	18.45	-	-	-	0.00	0.00	0.00	NMT	No	6 ensemble
69	Kyoto-U	ja-en	2016/08/20 15:07:47	1246	-	-	-	24.71	-	-	-	0.00	0.00	0.00	NMT	No	voc src:200k voc tgt: 52k + BPE 2-layer self-ensembling
70	NAIST	ja-en	2016/08/20 15:33:12	1247	-	-	-	26.12	-	-	-	0.00	0.00	0.00	SMT	No	Neural MT w/ Lexicon 6 Ensemble
71	NAIST	ja-en	2016/08/27 00:05:38	1275	-	-	-	27.55	-	-	-	0.00	0.00	0.00	SMT	No	Neural MT w/ Lexicon and MinRisk Training 6 Ensemble
72	ORGANIZER	ja-en	2016/11/16 10:29:51	1333	-	-	-	22.04	-	-	-	0.00	0.00	0.00	NMT	Yes	Online D (2016/11/14)
73	NICT-2	ja-en	2017/07/26 13:54:28	1476	-	-	-	24.79	-	-	-	0.00	0.00	0.00	NMT	No	NMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
74	NICT-2	ja-en	2017/07/26 14:04:38	1480	-	-	-	26.76	-	-	-	0.00	0.00	0.00	NMT	No	NMT 6 Ensembles * Bi-directional Reranking
75	NTT	ja-en	2017/07/30 20:43:07	1616	-	-	-	27.43	-	-	-	0.00	0.00	0.00	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
76	CUNI	ja-en	2017/07/31 22:30:52	1665	-	-	-	23.43	-	-	-	0.00	0.00	0.00	NMT	No	Bahdanau (2014) seq2seq with conditional GRU on byte-pair encoding, 1M sentences
77	NTT	ja-en	2017/08/01 04:29:02	1681	-	-	-	28.36	-	-	-	0.00	0.00	0.00	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
78	TMU	ja-en	2017/08/01 11:10:49	1695	-	-	-	21.00	-	-	-	0.00	0.00	0.00	NMT	No	our baseline system in 2017
79	TMU	ja-en	2017/08/01 11:35:08	1703	-	-	-	23.03	-	-	-	0.00	0.00	0.00	NMT	No	baseline system with beam20
80	TMU	ja-en	2017/08/01 12:14:41	1712	-	-	-	22.87	-	-	-	0.00	0.00	0.00	NMT	No	the ensemble system of different dropout rate.
81	Kyoto-U	ja-en	2017/08/01 13:42:25	1717	-	-	-	27.53	-	-	-	0.00	0.00	0.00	NMT	No	Ensemble of 4 BPE averaged parameters
82	NTT	ja-en	2017/08/01 14:50:51	1724	-	-	-	27.62	-	-	-	0.00	0.00	0.00	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
83	NTT	ja-en	2017/08/01 15:53:15	1732	-	-	-	28.15	-	-	-	0.00	0.00	0.00	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from training data)
84	Kyoto-U	ja-en	2017/08/01 16:38:21	1733	-	-	-	27.66	-	-	-	0.00	0.00	0.00	NMT	No	Ensemble of 4 BPE, averaged Coverage penalty
85	ORGANIZER	ja-en	2017/08/02 01:03:08	1736	-	-	-	28.06	-	-	-	0.00	0.00	0.00	NMT	No	Google's "Attention Is All You Need"
86	TMU	ja-en	2017/08/04 11:16:36	1750	-	-	-	24.55	-	-	-	0.00	0.00	0.00	NMT	No	beam_size: 10, ensemble of different dropout rates.
87	ORGANIZER	ja-en	2018/08/14 11:07:47	1901	-	-	-	26.91	-	-	-	-	0.00	0.00	NMT	No	NMT with Attention
88	srcb	ja-en	2018/08/26 11:09:50	2152	-	-	-	28.46	-	-	-	-	0.00	0.00	NMT	No	Transformer, average checkpoints.
89	NICT-5	ja-en	2018/08/27 15:01:05	2174	-	-	-	28.63	-	-	-	-	0.00	0.00	NMT	No	Transformer vanilla model using 3M sentences.
90	NICT-5	ja-en	2018/09/10 14:55:37	2273	-	-	-	29.65	-	-	-	-	0.00	0.00	NMT	No	MLNMT
91	Osaka-U	ja-en	2018/09/15 23:05:12	2440	-	-	-	26.19	-	-	-	-	0.00	0.00	NMT	Yes	rewarding model
92	TMU	ja-en	2018/09/16 11:53:24	2461	-	-	-	24.94	-	-	-	-	0.00	0.00	NMT	No	Baseline-NMT ( Single )
93	TMU	ja-en	2018/09/16 12:04:36	2464	-	-	-	25.85	-	-	-	-	0.00	0.00	NMT	No	Ensemble of 6 Baseline-NMT
94	TMU	ja-en	2018/09/16 12:05:47	2465	-	-	-	25.17	-	-	-	-	0.00	0.00	NMT	No	GAN-NMT ( Single )
95	TMU	ja-en	2018/09/16 12:06:39	2466	-	-	-	24.98	-	-	-	-	0.00	0.00	NMT	No	Reconstructor-NMT ( Single )
96	Osaka-U	ja-en	2018/09/16 13:11:48	2472	-	-	-	13.97	-	-	-	-	0.00	0.00	SMT	No	preordering with neural network
97	srcb	ja-en	2018/09/16 14:51:47	2474	-	-	-	30.59	-	-	-	-	0.00	0.00	NMT	No	Transformer with relative position, ensemble of 3 models.
98	TMU	ja-en	2018/09/16 17:02:15	2483	-	-	-	25.45	-	-	-	-	0.00	0.00	NMT	No	Ensemble of 6 NMT ( 2 Baseline + 2 Reconstructor + 2 GAN )

Notice:

This table is sorted by the leftmost segmenters. You can change the segmenter used to sort by clicking each segmenter link.

RIBES

#	Team	Task	Date/Time	DataID	RIBES										Method	Other Resources	System Description
#	Team	Task	Date/Time	DataID	juman	kytea	mecab	moses- tokenizer	stanford- segmenter- ctb	stanford- segmenter- pku	indic- tokenizer	unuse	myseg	kmseg	Method	Other Resources	System Description
1	NICT-5	ja-en	2019/07/16 17:11:38	2718	-	-	-	0.772422	-	-	-	-	-	-	NMT	No	RSNMT 6 layer with distillation
2	srcb	ja-en	2019/07/25 11:52:48	2919	-	-	-	0.777801	-	-	-	-	-	-	NMT	No	Transformer (Big) with relative position, average checkpoints.
3	ykkd	ja-en	2019/07/26 12:26:03	2989	-	-	-	0.769061	-	-	-	-	-	-	NMT	No	Fully Character-level,6 bi-LSTM (512*2) for Encoder, 6 LSTM for Decoder. Middle dense layer. Beam w/4.Length norm set to 0.2
4	NICT-5	ja-en	2019/07/26 18:02:47	3041	-	-	-	0.764672	-	-	-	-	-	-	NMT	No	RSNMT 6 layer
5	NICT-2	ja-en	2019/07/26 22:33:30	3085	-	-	-	0.756346	-	-	-	-	-	-	NMT	No	Transformer, sigle model w/ long warm-up and self-training
6	NICT-2	ja-en	2019/07/26 22:37:22	3086	-	-	-	0.760796	-	-	-	-	-	-	NMT	No	Transformer, ensemble of 4 models w/ long warm-up and self-training
7	KNU_Hyundai	ja-en	2019/07/27 09:28:37	3173	-	-	-	0.774653	-	-	-	-	-	-	NMT	No	Transformer Base, relative position, BT, r2l reranking, ensemble of 3 models
8	srcb	ja-en	2019/07/27 15:27:01	3205	-	-	-	0.778832	-	-	-	-	-	-	NMT	No	Transformer (Big) with relative position, data augmentation, average checkpoints, Bayes re-ranking.
9	NTT	ja-en	2019/07/28 11:18:59	3225	-	-	-	0.773281	-	-	-	-	-	-	SMT	No	ASPEC first 1.5M ＋last 1.5M (fwd), 6 ensemble
10	NTT	ja-en	2019/07/28 15:11:04	3233	-	-	-	0.770096	-	-	-	-	-	-	NMT	Yes	ParaCrawl + (ASPEC first 1.5M + Synthetic 1.5M) * 2 oversampling, fine-tune ASPEC, SINGLE MODEL
11	AISTAI	ja-en	2019/08/01 10:44:49	3260	-	-	-	0.760656	-	-	-	-	-	-	NMT	No	Transformer (big), 1.5M sentences, train_steps=300000, Averaged the last 10 ckpts, by Tensor2Tensor.
12	AISTAI	ja-en	2019/08/31 21:28:13	3361	-	-	-	0.769105	-	-	-	-	-	-	NMT	No	Transformer, 1.5M sentences, relative position, ensemble of 4 models, by OpenNMT-py.
13	NICT-5	ja-en	2021/03/18 23:22:53	4574	-	-	-	0.768617	-	-	-	-	-	-	NMT	No	My NMT implementation. Beam size 8. LP 0.6
14	ORGANIZER	ja-en	2014/07/11 19:45:32	2	-	-	-	0.651066	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Hierarchical Phrase-based SMT (2014)
15	ORGANIZER	ja-en	2014/07/11 19:49:57	6	-	-	-	0.645137	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Phrase-based SMT
16	ORGANIZER	ja-en	2014/07/11 19:59:55	9	-	-	-	0.678253	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	String-to-Tree SMT (2014)
17	ORGANIZER	ja-en	2014/07/18 11:08:13	35	-	-	-	0.643588	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online D (2014)
18	NAIST	ja-en	2014/07/19 01:04:48	46	-	-	-	0.722599	-	-	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Travatar-based Forest-to-String SMT System with Extra Dictionaries
19	ORGANIZER	ja-en	2014/07/21 11:53:48	76	-	-	-	0.663851	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT E
20	ORGANIZER	ja-en	2014/07/21 11:57:08	79	-	-	-	0.661387	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT F
21	ORGANIZER	ja-en	2014/07/22 11:22:40	87	-	-	-	0.624827	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online C (2014)
22	ORGANIZER	ja-en	2014/07/23 14:52:31	96	-	-	-	0.683378	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT D (2014)
23	EIWA	ja-en	2014/07/30 16:07:14	116	-	-	-	0.706686	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	Combination of RBMT and SPE(statistical post editing)
24	NAIST	ja-en	2014/07/31 11:40:53	119	-	-	-	0.723541	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar-based Forest-to-String SMT System
25	NAIST	ja-en	2014/08/01 17:35:16	125	-	-	-	0.723670	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
26	Kyoto-U	ja-en	2014/08/19 10:24:06	136	-	-	-	0.689829	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our baseline system using 3M parallel sentences.
27	Sense	ja-en	2014/08/23 05:34:03	164	-	-	-	0.646204	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Paraphrase max10
28	TOSHIBA	ja-en	2014/08/29 18:47:44	240	-	-	-	0.687122	-	-	0.000000	0.000000	0.000000	0.000000	RBMT	Yes	RBMT system
29	TOSHIBA	ja-en	2014/08/29 18:48:24	241	-	-	-	0.707936	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
30	Kyoto-U	ja-en	2014/08/31 23:36:50	256	-	-	-	0.701154	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our new baseline system after several modifications.
31	Kyoto-U	ja-en	2014/09/01 10:27:54	262	-	-	-	0.698953	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
32	NII	ja-en	2014/09/02 11:42:01	271	-	-	-	0.630825	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Our Baseline
33	NII	ja-en	2014/09/02 11:42:53	272	-	-	-	0.610833	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Our Baseline with Preordering
34	TMU	ja-en	2014/09/07 23:28:04	300	-	-	-	0.644698	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Our baseline system with preordering method
35	TMU	ja-en	2014/09/07 23:32:49	301	-	-	-	0.648879	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Our baseline system with another preordering method
36	TMU	ja-en	2014/09/09 19:14:42	307	-	-	-	0.613119	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Our baseline system
37	NICT	ja-en	2015/07/16 13:27:58	488	-	-	-	0.659883	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	our baseline (DL=6) + dependency-based pre-reordering [Ding+ 2015]
38	NICT	ja-en	2015/07/17 08:51:45	489	-	-	-	0.639711	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	our baseline: PB SMT in MOSES (DL=20) / SRILM / MeCab (IPA)
39	NICT	ja-en	2015/07/17 11:02:10	492	-	-	-	0.684485	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	our baseline (DL=9) + reverse pre-reordering [Katz-Brown & Collins 2008]
40	TOSHIBA	ja-en	2015/07/23 15:00:12	506	-	-	-	0.715795	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	System combination SMT and RBMT(SPE) with RNNLM language model
41	TOSHIBA	ja-en	2015/07/28 16:44:27	529	-	-	-	0.718540	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
42	Sense	ja-en	2015/07/28 22:23:43	530	-	-	-	0.627006	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-2015
43	TMU	ja-en	2015/08/04 16:32:20	578	-	-	-	0.641456	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Our PBSMT baseline (2015)
44	NAIST	ja-en	2015/08/14 17:46:43	655	-	-	-	0.749573	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System with NeuralMT Reranking and Parser Self Training
45	Sense	ja-en	2015/08/18 21:54:39	702	-	-	-	0.609632	-	-	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Passive JSTx1
46	Sense	ja-en	2015/08/18 21:58:08	708	-	-	-	0.600806	-	-	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Pervasive JSTx1
47	NAIST	ja-en	2015/08/24 23:53:53	757	-	-	-	0.743771	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System with NeuralMT Reranking
48	NAIST	ja-en	2015/08/25 13:02:45	766	-	-	-	0.722798	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System with Parser Self Training
49	NAIST	ja-en	2015/08/25 13:03:48	767	-	-	-	0.713083	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System Baseline
50	ORGANIZER	ja-en	2015/08/25 18:57:25	775	-	-	-	0.676609	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online D (2015)
51	Kyoto-U	ja-en	2015/08/27 14:40:32	796	-	-	-	0.706480	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT system without reranking
52	Sense	ja-en	2015/08/28 19:25:25	822	-	-	-	0.629066	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-2015 (train1 only)
53	Sense	ja-en	2015/08/29 04:32:36	824	-	-	-	0.633073	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-2015 (train123)
54	Kyoto-U	ja-en	2015/08/30 13:02:18	829	-	-	-	0.724555	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking
55	TMU	ja-en	2015/09/01 05:46:50	847	-	-	-	0.628897	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	PBSMT with dependency based phrase segmentation
56	Sense	ja-en	2015/09/01 17:42:28	860	-	-	-	0.610775	-	-	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Passive JSTx1
57	Sense	ja-en	2015/09/01 17:42:58	861	-	-	-	0.609008	-	-	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Pervasive JSTx1
58	ORGANIZER	ja-en	2015/09/10 13:41:03	877	-	-	-	0.678253	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	String-to-Tree SMT (2015)
59	ORGANIZER	ja-en	2015/09/10 14:38:13	887	-	-	-	0.683378	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT D (2015)
60	ORGANIZER	ja-en	2015/09/11 10:54:33	892	-	-	-	0.622564	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online C (2015)
61	ORGANIZER	ja-en	2016/07/26 11:37:38	1042	-	-	-	0.677412	-	-	-	0.000000	0.000000	0.000000	Other	Yes	Online D (2016)
62	NICT-2	ja-en	2016/08/05 17:50:25	1104	-	-	-	0.708808	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
63	NAIST	ja-en	2016/08/09 16:14:05	1122	-	-	-	0.762712	-	-	-	0.000000	0.000000	0.000000	SMT	No	Neural MT w/ Lexicon and MinRisk Training 4 Ensemble
64	bjtu_nlp	ja-en	2016/08/17 19:51:24	1168	-	-	-	0.690455	-	-	-	0.000000	0.000000	0.000000	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
65	Kyoto-U	ja-en	2016/08/18 15:17:05	1182	-	-	-	0.756601	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble of 4 single-layer model (30k voc)
66	Kyoto-U	ja-en	2016/08/19 01:31:01	1189	-	-	-	0.705700	-	-	-	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT 2016 w/o reranking
67	TMU	ja-en	2016/08/20 07:39:02	1222	-	-	-	0.710613	-	-	-	0.000000	0.000000	0.000000	NMT	No	2016 our proposed method to control output voice
68	TMU	ja-en	2016/08/20 14:31:48	1234	-	-	-	0.711542	-	-	-	0.000000	0.000000	0.000000	NMT	No	6 ensemble
69	Kyoto-U	ja-en	2016/08/20 15:07:47	1246	-	-	-	0.750802	-	-	-	0.000000	0.000000	0.000000	NMT	No	voc src:200k voc tgt: 52k + BPE 2-layer self-ensembling
70	NAIST	ja-en	2016/08/20 15:33:12	1247	-	-	-	0.756956	-	-	-	0.000000	0.000000	0.000000	SMT	No	Neural MT w/ Lexicon 6 Ensemble
71	NAIST	ja-en	2016/08/27 00:05:38	1275	-	-	-	0.767661	-	-	-	0.000000	0.000000	0.000000	SMT	No	Neural MT w/ Lexicon and MinRisk Training 6 Ensemble
72	ORGANIZER	ja-en	2016/11/16 10:29:51	1333	-	-	-	0.733483	-	-	-	0.000000	0.000000	0.000000	NMT	Yes	Online D (2016/11/14)
73	NICT-2	ja-en	2017/07/26 13:54:28	1476	-	-	-	0.747335	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
74	NICT-2	ja-en	2017/07/26 14:04:38	1480	-	-	-	0.741329	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT 6 Ensembles * Bi-directional Reranking
75	NTT	ja-en	2017/07/30 20:43:07	1616	-	-	-	0.764831	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
76	CUNI	ja-en	2017/07/31 22:30:52	1665	-	-	-	0.741699	-	-	-	0.000000	0.000000	0.000000	NMT	No	Bahdanau (2014) seq2seq with conditional GRU on byte-pair encoding, 1M sentences
77	NTT	ja-en	2017/08/01 04:29:02	1681	-	-	-	0.768880	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
78	TMU	ja-en	2017/08/01 11:10:49	1695	-	-	-	0.725284	-	-	-	0.000000	0.000000	0.000000	NMT	No	our baseline system in 2017
79	TMU	ja-en	2017/08/01 11:35:08	1703	-	-	-	0.741175	-	-	-	0.000000	0.000000	0.000000	NMT	No	baseline system with beam20
80	TMU	ja-en	2017/08/01 12:14:41	1712	-	-	-	0.735908	-	-	-	0.000000	0.000000	0.000000	NMT	No	the ensemble system of different dropout rate.
81	Kyoto-U	ja-en	2017/08/01 13:42:25	1717	-	-	-	0.761403	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble of 4 BPE averaged parameters
82	NTT	ja-en	2017/08/01 14:50:51	1724	-	-	-	0.763248	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
83	NTT	ja-en	2017/08/01 15:53:15	1732	-	-	-	0.769430	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from training data)
84	Kyoto-U	ja-en	2017/08/01 16:38:21	1733	-	-	-	0.765464	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble of 4 BPE, averaged Coverage penalty
85	ORGANIZER	ja-en	2017/08/02 01:03:08	1736	-	-	-	0.767577	-	-	-	0.000000	0.000000	0.000000	NMT	No	Google's "Attention Is All You Need"
86	TMU	ja-en	2017/08/04 11:16:36	1750	-	-	-	0.744928	-	-	-	0.000000	0.000000	0.000000	NMT	No	beam_size: 10, ensemble of different dropout rates.
87	ORGANIZER	ja-en	2018/08/14 11:07:47	1901	-	-	-	0.764968	-	-	-	-	0.000000	0.000000	NMT	No	NMT with Attention
88	srcb	ja-en	2018/08/26 11:09:50	2152	-	-	-	0.767194	-	-	-	-	0.000000	0.000000	NMT	No	Transformer, average checkpoints.
89	NICT-5	ja-en	2018/08/27 15:01:05	2174	-	-	-	0.765933	-	-	-	-	0.000000	0.000000	NMT	No	Transformer vanilla model using 3M sentences.
90	NICT-5	ja-en	2018/09/10 14:55:37	2273	-	-	-	0.774788	-	-	-	-	0.000000	0.000000	NMT	No	MLNMT
91	Osaka-U	ja-en	2018/09/15 23:05:12	2440	-	-	-	0.749825	-	-	-	-	0.000000	0.000000	NMT	Yes	rewarding model
92	TMU	ja-en	2018/09/16 11:53:24	2461	-	-	-	0.757955	-	-	-	-	0.000000	0.000000	NMT	No	Baseline-NMT ( Single )
93	TMU	ja-en	2018/09/16 12:04:36	2464	-	-	-	0.761450	-	-	-	-	0.000000	0.000000	NMT	No	Ensemble of 6 Baseline-NMT
94	TMU	ja-en	2018/09/16 12:05:47	2465	-	-	-	0.757413	-	-	-	-	0.000000	0.000000	NMT	No	GAN-NMT ( Single )
95	TMU	ja-en	2018/09/16 12:06:39	2466	-	-	-	0.759238	-	-	-	-	0.000000	0.000000	NMT	No	Reconstructor-NMT ( Single )
96	Osaka-U	ja-en	2018/09/16 13:11:48	2472	-	-	-	0.665391	-	-	-	-	0.000000	0.000000	SMT	No	preordering with neural network
97	srcb	ja-en	2018/09/16 14:51:47	2474	-	-	-	0.777896	-	-	-	-	0.000000	0.000000	NMT	No	Transformer with relative position, ensemble of 3 models.
98	TMU	ja-en	2018/09/16 17:02:15	2483	-	-	-	0.759790	-	-	-	-	0.000000	0.000000	NMT	No	Ensemble of 6 NMT ( 2 Baseline + 2 Reconstructor + 2 GAN )

Notice:

This table is sorted by the leftmost segmenters. You can change the segmenter used to sort by clicking each segmenter link.

AMFM

#	Team	Task	Date/Time	DataID	AMFM										Method	Other Resources	System Description
#	Team	Task	Date/Time	DataID	unuse	unuse	unuse	unuse	unuse	unuse	unuse	unuse	unuse	unuse	Method	Other Resources	System Description
1	NICT-5	ja-en	2019/07/16 17:11:38	2718	-	-	-	0.619040	-	-	-	-	-	-	NMT	No	RSNMT 6 layer with distillation
2	srcb	ja-en	2019/07/25 11:52:48	2919	-	-	-	0.628070	-	-	-	-	-	-	NMT	No	Transformer (Big) with relative position, average checkpoints.
3	ykkd	ja-en	2019/07/26 12:26:03	2989	-	-	-	0.619640	-	-	-	-	-	-	NMT	No	Fully Character-level,6 bi-LSTM (512*2) for Encoder, 6 LSTM for Decoder. Middle dense layer. Beam w/4.Length norm set to 0.2
4	NICT-5	ja-en	2019/07/26 18:02:47	3041	-	-	-	0.608450	-	-	-	-	-	-	NMT	No	RSNMT 6 layer
5	NICT-2	ja-en	2019/07/26 22:33:30	3085	-	-	-	0.602770	-	-	-	-	-	-	NMT	No	Transformer, sigle model w/ long warm-up and self-training
6	NICT-2	ja-en	2019/07/26 22:37:22	3086	-	-	-	0.606190	-	-	-	-	-	-	NMT	No	Transformer, ensemble of 4 models w/ long warm-up and self-training
7	KNU_Hyundai	ja-en	2019/07/27 09:28:37	3173	-	-	-	0.622070	-	-	-	-	-	-	NMT	No	Transformer Base, relative position, BT, r2l reranking, ensemble of 3 models
8	srcb	ja-en	2019/07/27 15:27:01	3205	-	-	-	0.630150	-	-	-	-	-	-	NMT	No	Transformer (Big) with relative position, data augmentation, average checkpoints, Bayes re-ranking.
9	NTT	ja-en	2019/07/28 11:18:59	3225	-	-	-	0.626880	-	-	-	-	-	-	SMT	No	ASPEC first 1.5M ＋last 1.5M (fwd), 6 ensemble
10	NTT	ja-en	2019/07/28 15:11:04	3233	-	-	-	0.626260	-	-	-	-	-	-	NMT	Yes	ParaCrawl + (ASPEC first 1.5M + Synthetic 1.5M) * 2 oversampling, fine-tune ASPEC, SINGLE MODEL
11	AISTAI	ja-en	2019/08/01 10:44:49	3260	-	-	-	0.620640	-	-	-	-	-	-	NMT	No	Transformer (big), 1.5M sentences, train_steps=300000, Averaged the last 10 ckpts, by Tensor2Tensor.
12	AISTAI	ja-en	2019/08/31 21:28:13	3361	-	-	-	0.626300	-	-	-	-	-	-	NMT	No	Transformer, 1.5M sentences, relative position, ensemble of 4 models, by OpenNMT-py.
13	NICT-5	ja-en	2021/03/18 23:22:53	4574	-	-	-	0.601460	-	-	-	-	-	-	NMT	No	My NMT implementation. Beam size 8. LP 0.6
14	ORGANIZER	ja-en	2014/07/11 19:45:32	2	0.000000	0.000000	0.000000	0.588880	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Hierarchical Phrase-based SMT (2014)
15	ORGANIZER	ja-en	2014/07/11 19:49:57	6	0.000000	0.000000	0.000000	0.590950	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Phrase-based SMT
16	ORGANIZER	ja-en	2014/07/11 19:59:55	9	0.000000	0.000000	0.000000	0.593410	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	String-to-Tree SMT (2014)
17	ORGANIZER	ja-en	2014/07/18 11:08:13	35	0.000000	0.000000	0.000000	0.564170	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online D (2014)
18	NAIST	ja-en	2014/07/19 01:04:48	46	0.000000	0.000000	0.000000	0.604180	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Travatar-based Forest-to-String SMT System with Extra Dictionaries
19	ORGANIZER	ja-en	2014/07/21 11:53:48	76	0.000000	0.000000	0.000000	0.561620	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT E
20	ORGANIZER	ja-en	2014/07/21 11:57:08	79	0.000000	0.000000	0.000000	0.556840	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT F
21	ORGANIZER	ja-en	2014/07/22 11:22:40	87	0.000000	0.000000	0.000000	0.466480	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online C (2014)
22	ORGANIZER	ja-en	2014/07/23 14:52:31	96	0.000000	0.000000	0.000000	0.551690	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT D (2014)
23	EIWA	ja-en	2014/07/30 16:07:14	116	0.000000	0.000000	0.000000	0.576540	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	Combination of RBMT and SPE(statistical post editing)
24	NAIST	ja-en	2014/07/31 11:40:53	119	0.000000	0.000000	0.000000	0.603490	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar-based Forest-to-String SMT System
25	NAIST	ja-en	2014/08/01 17:35:16	125	0.000000	0.000000	0.000000	0.602780	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
26	Kyoto-U	ja-en	2014/08/19 10:24:06	136	0.000000	0.000000	0.000000	0.593970	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our baseline system using 3M parallel sentences.
27	Sense	ja-en	2014/08/23 05:34:03	164	0.000000	0.000000	0.000000	0.587520	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Paraphrase max10
28	TOSHIBA	ja-en	2014/08/29 18:47:44	240	0.000000	0.000000	0.000000	0.552980	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	RBMT	Yes	RBMT system
29	TOSHIBA	ja-en	2014/08/29 18:48:24	241	0.000000	0.000000	0.000000	0.551740	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
30	Kyoto-U	ja-en	2014/08/31 23:36:50	256	0.000000	0.000000	0.000000	0.593660	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our new baseline system after several modifications.
31	Kyoto-U	ja-en	2014/09/01 10:27:54	262	0.000000	0.000000	0.000000	0.588480	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
32	NII	ja-en	2014/09/02 11:42:01	271	0.000000	0.000000	0.000000	0.582800	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Our Baseline
33	NII	ja-en	2014/09/02 11:42:53	272	0.000000	0.000000	0.000000	0.574030	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Our Baseline with Preordering
34	TMU	ja-en	2014/09/07 23:28:04	300	0.000000	0.000000	0.000000	0.561450	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Our baseline system with preordering method
35	TMU	ja-en	2014/09/07 23:32:49	301	0.000000	0.000000	0.000000	0.578360	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Our baseline system with another preordering method
36	TMU	ja-en	2014/09/09 19:14:42	307	0.000000	0.000000	0.000000	0.580500	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Our baseline system
37	NICT	ja-en	2015/07/16 13:27:58	488	0.000000	0.000000	0.000000	0.587530	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	our baseline (DL=6) + dependency-based pre-reordering [Ding+ 2015]
38	NICT	ja-en	2015/07/17 08:51:45	489	0.000000	0.000000	0.000000	0.562920	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	our baseline: PB SMT in MOSES (DL=20) / SRILM / MeCab (IPA)
39	NICT	ja-en	2015/07/17 11:02:10	492	0.000000	0.000000	0.000000	0.576510	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	our baseline (DL=9) + reverse pre-reordering [Katz-Brown & Collins 2008]
40	TOSHIBA	ja-en	2015/07/23 15:00:12	506	0.000000	0.000000	0.000000	0.604760	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	System combination SMT and RBMT(SPE) with RNNLM language model
41	TOSHIBA	ja-en	2015/07/28 16:44:27	529	0.000000	0.000000	0.000000	0.597830	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
42	Sense	ja-en	2015/07/28 22:23:43	530	0.000000	0.000000	0.000000	0.564610	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-2015
43	TMU	ja-en	2015/08/04 16:32:20	578	0.000000	0.000000	0.000000	0.590980	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Our PBSMT baseline (2015)
44	NAIST	ja-en	2015/08/14 17:46:43	655	0.000000	0.000000	0.000000	0.609430	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System with NeuralMT Reranking and Parser Self Training
45	Sense	ja-en	2015/08/18 21:54:39	702	0.000000	0.000000	0.000000	0.579210	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Passive JSTx1
46	Sense	ja-en	2015/08/18 21:58:08	708	0.000000	0.000000	0.000000	0.581540	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Pervasive JSTx1
47	NAIST	ja-en	2015/08/24 23:53:53	757	0.000000	0.000000	0.000000	0.606600	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System with NeuralMT Reranking
48	NAIST	ja-en	2015/08/25 13:02:45	766	0.000000	0.000000	0.000000	0.599800	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System with Parser Self Training
49	NAIST	ja-en	2015/08/25 13:03:48	767	0.000000	0.000000	0.000000	0.600000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System Baseline
50	ORGANIZER	ja-en	2015/08/25 18:57:25	775	0.000000	0.000000	0.000000	0.562270	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online D (2015)
51	Kyoto-U	ja-en	2015/08/27 14:40:32	796	0.000000	0.000000	0.000000	0.596430	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT system without reranking
52	Sense	ja-en	2015/08/28 19:25:25	822	0.000000	0.000000	0.000000	0.592040	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-2015 (train1 only)
53	Sense	ja-en	2015/08/29 04:32:36	824	0.000000	0.000000	0.000000	0.579280	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-2015 (train123)
54	Kyoto-U	ja-en	2015/08/30 13:02:18	829	0.000000	0.000000	0.000000	0.603210	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking
55	TMU	ja-en	2015/09/01 05:46:50	847	0.000000	0.000000	0.000000	0.570430	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	PBSMT with dependency based phrase segmentation
56	Sense	ja-en	2015/09/01 17:42:28	860	0.000000	0.000000	0.000000	0.579790	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Passive JSTx1
57	Sense	ja-en	2015/09/01 17:42:58	861	0.000000	0.000000	0.000000	0.582370	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Pervasive JSTx1
58	ORGANIZER	ja-en	2015/09/10 13:41:03	877	0.000000	0.000000	0.000000	0.593410	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	String-to-Tree SMT (2015)
59	ORGANIZER	ja-en	2015/09/10 14:38:13	887	0.000000	0.000000	0.000000	0.551690	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT D (2015)
60	ORGANIZER	ja-en	2015/09/11 10:54:33	892	0.000000	0.000000	0.000000	0.453370	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online C (2015)
61	ORGANIZER	ja-en	2016/07/26 11:37:38	1042	-	-	-	0.564270	-	-	-	0.000000	0.000000	0.000000	Other	Yes	Online D (2016)
62	NICT-2	ja-en	2016/08/05 17:50:25	1104	-	-	-	0.595930	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
63	NAIST	ja-en	2016/08/09 16:14:05	1122	-	-	-	0.587450	-	-	-	0.000000	0.000000	0.000000	SMT	No	Neural MT w/ Lexicon and MinRisk Training 4 Ensemble
64	bjtu_nlp	ja-en	2016/08/17 19:51:24	1168	-	-	-	0.505730	-	-	-	0.000000	0.000000	0.000000	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
65	Kyoto-U	ja-en	2016/08/18 15:17:05	1182	-	-	-	0.558540	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble of 4 single-layer model (30k voc)
66	Kyoto-U	ja-en	2016/08/19 01:31:01	1189	-	-	-	0.595240	-	-	-	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT 2016 w/o reranking
67	TMU	ja-en	2016/08/20 07:39:02	1222	-	-	-	0.565270	-	-	-	0.000000	0.000000	0.000000	NMT	No	2016 our proposed method to control output voice
68	TMU	ja-en	2016/08/20 14:31:48	1234	-	-	-	0.546880	-	-	-	0.000000	0.000000	0.000000	NMT	No	6 ensemble
69	Kyoto-U	ja-en	2016/08/20 15:07:47	1246	-	-	-	0.562650	-	-	-	0.000000	0.000000	0.000000	NMT	No	voc src:200k voc tgt: 52k + BPE 2-layer self-ensembling
70	NAIST	ja-en	2016/08/20 15:33:12	1247	-	-	-	0.571360	-	-	-	0.000000	0.000000	0.000000	SMT	No	Neural MT w/ Lexicon 6 Ensemble
71	NAIST	ja-en	2016/08/27 00:05:38	1275	-	-	-	0.594150	-	-	-	0.000000	0.000000	0.000000	SMT	No	Neural MT w/ Lexicon and MinRisk Training 6 Ensemble
72	ORGANIZER	ja-en	2016/11/16 10:29:51	1333	-	-	-	0.584390	-	-	-	0.000000	0.000000	0.000000	NMT	Yes	Online D (2016/11/14)
73	NICT-2	ja-en	2017/07/26 13:54:28	1476	-	-	-	0.574810	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
74	NICT-2	ja-en	2017/07/26 14:04:38	1480	-	-	-	0.578150	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT 6 Ensembles * Bi-directional Reranking
75	NTT	ja-en	2017/07/30 20:43:07	1616	-	-	-	0.597620	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
76	CUNI	ja-en	2017/07/31 22:30:52	1665	-	-	-	0.583780	-	-	-	0.000000	0.000000	0.000000	NMT	No	Bahdanau (2014) seq2seq with conditional GRU on byte-pair encoding, 1M sentences
77	NTT	ja-en	2017/08/01 04:29:02	1681	-	-	-	0.597860	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
78	TMU	ja-en	2017/08/01 11:10:49	1695	-	-	-	0.585710	-	-	-	0.000000	0.000000	0.000000	NMT	No	our baseline system in 2017
79	TMU	ja-en	2017/08/01 11:35:08	1703	-	-	-	0.595260	-	-	-	0.000000	0.000000	0.000000	NMT	No	baseline system with beam20
80	TMU	ja-en	2017/08/01 12:14:41	1712	-	-	-	0.588360	-	-	-	0.000000	0.000000	0.000000	NMT	No	the ensemble system of different dropout rate.
81	Kyoto-U	ja-en	2017/08/01 13:42:25	1717	-	-	-	0.585540	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble of 4 BPE averaged parameters
82	NTT	ja-en	2017/08/01 14:50:51	1724	-	-	-	0.597470	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
83	NTT	ja-en	2017/08/01 15:53:15	1732	-	-	-	0.599920	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from training data)
84	Kyoto-U	ja-en	2017/08/01 16:38:21	1733	-	-	-	0.591160	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble of 4 BPE, averaged Coverage penalty
85	ORGANIZER	ja-en	2017/08/02 01:03:08	1736	-	-	-	0.595580	-	-	-	0.000000	0.000000	0.000000	NMT	No	Google's "Attention Is All You Need"
86	TMU	ja-en	2017/08/04 11:16:36	1750	-	-	-	0.596360	-	-	-	0.000000	0.000000	0.000000	NMT	No	beam_size: 10, ensemble of different dropout rates.
87	ORGANIZER	ja-en	2018/08/14 11:07:47	1901	-	-	-	0.595370	-	-	-	-	0.000000	0.000000	NMT	No	NMT with Attention
88	srcb	ja-en	2018/08/26 11:09:50	2152	-	-	-	0.605130	-	-	-	-	0.000000	0.000000	NMT	No	Transformer, average checkpoints.
89	NICT-5	ja-en	2018/08/27 15:01:05	2174	-	-	-	0.608070	-	-	-	-	0.000000	0.000000	NMT	No	Transformer vanilla model using 3M sentences.
90	NICT-5	ja-en	2018/09/10 14:55:37	2273	-	-	-	0.612060	-	-	-	-	0.000000	0.000000	NMT	No	MLNMT
91	Osaka-U	ja-en	2018/09/15 23:05:12	2440	-	-	-	0.588290	-	-	-	-	0.000000	0.000000	NMT	Yes	rewarding model
92	TMU	ja-en	2018/09/16 11:53:24	2461	-	-	-	0.596590	-	-	-	-	0.000000	0.000000	NMT	No	Baseline-NMT ( Single )
93	TMU	ja-en	2018/09/16 12:04:36	2464	-	-	-	0.600730	-	-	-	-	0.000000	0.000000	NMT	No	Ensemble of 6 Baseline-NMT
94	TMU	ja-en	2018/09/16 12:05:47	2465	-	-	-	0.595850	-	-	-	-	0.000000	0.000000	NMT	No	GAN-NMT ( Single )
95	TMU	ja-en	2018/09/16 12:06:39	2466	-	-	-	0.599110	-	-	-	-	0.000000	0.000000	NMT	No	Reconstructor-NMT ( Single )
96	Osaka-U	ja-en	2018/09/16 13:11:48	2472	-	-	-	0.571400	-	-	-	-	0.000000	0.000000	SMT	No	preordering with neural network
97	srcb	ja-en	2018/09/16 14:51:47	2474	-	-	-	0.619390	-	-	-	-	0.000000	0.000000	NMT	No	Transformer with relative position, ensemble of 3 models.
98	TMU	ja-en	2018/09/16 17:02:15	2483	-	-	-	0.598770	-	-	-	-	0.000000	0.000000	NMT	No	Ensemble of 6 NMT ( 2 Baseline + 2 Reconstructor + 2 GAN )

Notice:

This table is sorted by the leftmost segmenters. You can change the segmenter used to sort by clicking each segmenter link.
Adequacy-Fluency Metrics (AMFM) is a two-dimensional automatic evaluation metric for machine translation, designed to operate at the sentence level. It is based on adequacy and fluency, to decouple semantic and syntactic components of the translation process to provide a balanced view on translation quality.
AMFM is calculated without tokenizers.
The detail of AMFM is shown on the following paper: "Adequacy–Fluency Metrics: Evaluating MT in the Continuous Space Model Framework" [pdf]. Invited Talk in WAT2015 also helps understanding [slide].

HUMAN (WAT2022)

Notice:

HUMAN (WAT2022) is the result of the Pairwise Crowdsourcing Evaluation on WAT2022.
HUMAN (WAT2022) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2021)

Notice:

HUMAN (WAT2021) is the result of the Pairwise Crowdsourcing Evaluation on WAT2021.
HUMAN (WAT2021) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2020)

Notice:

HUMAN (WAT2020) is the result of the Pairwise Crowdsourcing Evaluation on WAT2020.
HUMAN (WAT2020) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2019)

Notice:

HUMAN (WAT2019) is the result of the Pairwise Crowdsourcing Evaluation on WAT2019.
HUMAN (WAT2019) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2018)

Notice:

HUMAN (WAT2018) is the result of the Pairwise Crowdsourcing Evaluation on WAT2018.
HUMAN (WAT2018) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2017)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	Kyoto-U	ja-en	2017/08/01 13:42:25	1717	77.750	NMT	No	Ensemble of 4 BPE averaged parameters
2	NTT	ja-en	2017/08/01 04:29:02	1681	77.250	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
3	ORGANIZER	ja-en	2017/08/02 01:03:08	1736	75.250	NMT	No	Google's "Attention Is All You Need"
4	NTT	ja-en	2017/07/30 20:43:07	1616	75.000	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
5	Kyoto-U	ja-en	2017/08/01 16:38:21	1733	74.500	NMT	No	Ensemble of 4 BPE, averaged Coverage penalty
6	NICT-2	ja-en	2017/07/26 14:04:38	1480	69.750	NMT	No	NMT 6 Ensembles * Bi-directional Reranking
7	NICT-2	ja-en	2017/07/26 13:54:28	1476	68.750	NMT	No	NMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
8	CUNI	ja-en	2017/07/31 22:30:52	1665	66.000	NMT	No	Bahdanau (2014) seq2seq with conditional GRU on byte-pair encoding, 1M sentences
9	TMU	ja-en	2017/08/01 11:35:08	1703	61.000	NMT	No	baseline system with beam20
10	TMU	ja-en	2017/08/01 11:10:49	1695	56.750	NMT	No	our baseline system in 2017

Notice:

HUMAN (WAT2017) is the result of the Pairwise Crowdsourcing Evaluation on WAT2017.
HUMAN (WAT2017) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2016)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	ORGANIZER	ja-en	2016/11/16 10:29:51	1333	63.000	NMT	Yes	Online D (2016/11/14)
2	NAIST	ja-en	2016/08/09 16:14:05	1122	48.250	SMT	No	Neural MT w/ Lexicon and MinRisk Training 4 Ensemble
3	NAIST	ja-en	2016/08/20 15:33:12	1247	47.500	SMT	No	Neural MT w/ Lexicon 6 Ensemble
4	Kyoto-U	ja-en	2016/08/20 15:07:47	1246	47.000	NMT	No	voc src:200k voc tgt: 52k + BPE 2-layer self-ensembling
5	Kyoto-U	ja-en	2016/08/18 15:17:05	1182	44.250	NMT	No	Ensemble of 4 single-layer model (30k voc)
6	ORGANIZER	ja-en	2016/07/26 11:37:38	1042	28.000	Other	Yes	Online D (2016)
7	TMU	ja-en	2016/08/20 14:31:48	1234	25.000	NMT	No	6 ensemble
8	bjtu_nlp	ja-en	2016/08/17 19:51:24	1168	19.250	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
9	TMU	ja-en	2016/08/20 07:39:02	1222	16.000	NMT	No	2016 our proposed method to control output voice
10	NICT-2	ja-en	2016/08/05 17:50:25	1104	Underway	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM

Notice:

HUMAN (WAT2016) is the result of the Pairwise Crowdsourcing Evaluation on WAT2016.
HUMAN (WAT2016) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2015)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	NAIST	ja-en	2015/08/14 17:46:43	655	35.500	SMT	No	Travatar System with NeuralMT Reranking and Parser Self Training
2	Kyoto-U	ja-en	2015/08/30 13:02:18	829	32.500	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking
3	TOSHIBA	ja-en	2015/07/28 16:44:27	529	25.000	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
4	TOSHIBA	ja-en	2015/07/23 15:00:12	506	21.250	SMT and RBMT	Yes	System combination SMT and RBMT(SPE) with RNNLM language model
5	ORGANIZER	ja-en	2015/09/10 14:38:13	887	16.750	Other	Yes	RBMT D (2015)
6	Kyoto-U	ja-en	2015/08/27 14:40:32	796	16.500	EBMT	No	KyotoEBMT system without reranking
7	NICT	ja-en	2015/07/16 13:27:58	488	16.000	SMT	No	our baseline (DL=6) + dependency-based pre-reordering [Ding+ 2015]
8	NAIST	ja-en	2015/08/25 13:02:45	766	11.750	SMT	No	Travatar System with Parser Self Training
9	ORGANIZER	ja-en	2015/09/10 13:41:03	877	7.000	SMT	No	String-to-Tree SMT (2015)
10	NICT	ja-en	2015/07/17 11:02:10	492	6.500	SMT	No	our baseline (DL=9) + reverse pre-reordering [Katz-Brown & Collins 2008]
11	ORGANIZER	ja-en	2015/08/25 18:57:25	775	0.250	Other	Yes	Online D (2015)
12	Sense	ja-en	2015/09/01 17:42:28	860	-7.750	SMT	Yes	Passive JSTx1
13	Sense	ja-en	2015/09/01 17:42:58	861	-12.750	SMT	Yes	Pervasive JSTx1
14	TMU	ja-en	2015/09/01 05:46:50	847	-25.500	SMT	No	PBSMT with dependency based phrase segmentation

Notice:

HUMAN (WAT2015) is the result of the Pairwise Crowdsourcing Evaluation on WAT2015.
HUMAN (WAT2015) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.
The detail of the evaluation can be found in the pdf document (PDF file).

HUMAN (WAT2014)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	NAIST	ja-en	2014/07/19 01:04:48	46	40.500	SMT	Yes	Travatar-based Forest-to-String SMT System with Extra Dictionaries
2	NAIST	ja-en	2014/07/31 11:40:53	119	37.500	SMT	No	Travatar-based Forest-to-String SMT System
3	ORGANIZER	ja-en	2014/07/11 19:59:55	9	25.500	SMT	No	String-to-Tree SMT (2014)
4	Kyoto-U	ja-en	2014/09/01 10:27:54	262	25.000	EBMT	No	Our new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
5	TOSHIBA	ja-en	2014/08/29 18:48:24	241	23.250	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
6	ORGANIZER	ja-en	2014/07/23 14:52:31	96	23.000	Other	Yes	RBMT D (2014)
7	EIWA	ja-en	2014/07/30 16:07:14	116	22.500	SMT and RBMT	Yes	Combination of RBMT and SPE(statistical post editing)
8	Kyoto-U	ja-en	2014/08/31 23:36:50	256	21.250	EBMT	No	Our new baseline system after several modifications.
9	TOSHIBA	ja-en	2014/08/29 18:47:44	240	20.250	RBMT	Yes	RBMT system
10	ORGANIZER	ja-en	2014/07/18 11:08:13	35	13.750	Other	Yes	Online D (2014)
11	ORGANIZER	ja-en	2014/07/11 19:45:32	2	7.750	SMT	No	Hierarchical Phrase-based SMT (2014)
12	Sense	ja-en	2014/08/23 05:34:03	164	1.250	SMT	No	Paraphrase max10
13	NII	ja-en	2014/09/02 11:42:01	271	-5.750	SMT	No	Our Baseline
14	NII	ja-en	2014/09/02 11:42:53	272	-14.250	SMT	No	Our Baseline with Preordering
15	TMU	ja-en	2014/09/07 23:32:49	301	-17.000	SMT	No	Our baseline system with another preordering method
16	TMU	ja-en	2014/09/07 23:28:04	300	-17.250	SMT	No	Our baseline system with preordering method

Notice:

HUMAN (WAT2014) is the result of the Pairwise Crowdsourcing Evaluation on WAT2014.
HUMAN (WAT2014) was evaluated by 3 different workers and the final decision is made by the voting of the judgements.
The detail of the evaluation can be found in the pdf document (PDF file).

EVALUATION RESULTS USAGE POLICY

When you use the WAT evaluation results for any purpose such as:
- writing technical papers,
- making presentations about your system,
- advertising your MT system to the customers,
you can use the information about translation directions, scores (including both automatic and human evaluations) and ranks of your system among others. You can also use the scores of the other systems, but you MUST anonymize the other system's names. In addition, you can show the links (URLs) to the WAT evaluation result pages.

NICT (National Institute of Information and Communications Technology)
Kyoto University
Last Modified: 2018-08-02

WAT The Workshop on Asian Translation Evaluation Results

BLEU

RIBES

AMFM

HUMAN (WAT2022)

HUMAN (WAT2021)

HUMAN (WAT2020)

HUMAN (WAT2019)

HUMAN (WAT2018)

HUMAN (WAT2017)

HUMAN (WAT2016)

HUMAN (WAT2015)

HUMAN (WAT2014)

EVALUATION RESULTS USAGE POLICY

WAT

The Workshop on Asian Translation

Evaluation Results