WAT

The Workshop on Asian Translation

Evaluation Results

BLEU

#	Team	Task	Date/Time	DataID	BLEU										Method	Other Resources	System Description
#	Team	Task	Date/Time	DataID	juman	kytea	mecab	moses- tokenizer	stanford- segmenter- ctb	stanford- segmenter- pku	indic- tokenizer	unuse	myseg	kmseg	Method	Other Resources	System Description
1	NTT	en-ja	2019/07/28 15:44:12	3236	45.83	47.63	46.57	-	-	-	-	-	-	-	NMT	No	ASPEC first 1.5M + Synthetic 1.5M, 6 ensemble
2	srcb	en-ja	2019/07/27 15:50:19	3212	45.71	47.55	46.29	-	-	-	-	-	-	-	NMT	No	Transformer(Big) with relative position, sentence-wise smooth, deep transformer, back translation, ensemble of 10 models, rerank.
3	NTT	en-ja	2019/07/28 15:55:18	3239	45.30	47.13	46.04	-	-	-	-	-	-	-	NMT	Yes	ParaCrawl + (ASPEC first 1.5M + Synthetic 1.5M) * 2 oversampling, fine-tune ASPEC, SINGLE MODEL
4	srcb	en-ja	2019/07/27 15:28:30	3206	45.30	47.12	45.93	-	-	-	-	-	-	-	NMT	No	Transformer(Big) with relative position, sentence-wise smooth, deep transformer, back translation, ensemble of 7 models, re-rank.
5	NICT-2	en-ja	2019/07/27 10:57:26	3182	44.61	46.59	45.66	-	-	-	-	-	-	-	NMT	No	Transformer, ensemble of 4 models w/ long warm-up and self-training
6	NICT-2	en-ja	2019/07/27 10:56:02	3181	43.79	45.82	44.83	-	-	-	-	-	-	-	NMT	No	Transformer, sigle model w/ long warm-up and self-training
7	KNU_Hyundai	en-ja	2019/07/27 09:19:31	3172	44.08	45.88	44.78	-	-	-	-	-	-	-	NMT	No	Transformer Base, relative position, BT, r2l reranking, ensemble of 3 models
8	AISTAI	en-ja	2019/08/30 20:23:24	3357	43.76	45.70	44.52	-	-	-	-	-	-	-	NMT	No	Transformer, 1.5M sentences, relative position, ensemble of 3 models, by OpenNMT-py.
9	srcb	en-ja	2019/07/25 11:43:06	2918	43.60	45.49	44.29	-	-	-	-	-	-	-	NMT	No	Transformer (Big) with relative position, sentence-wise smooth
10	srcb	en-ja	2018/09/16 15:52:17	2480	43.43	44.94	44.05	-	-	-	-	-	0.00	0.00	NMT	No	Transformer with relative position, ensemble of 4 models, rerank.
11	AISTAI	en-ja	2019/09/05 08:18:45	3373	42.92	44.90	43.67	-	-	-	-	-	-	-	NMT	No	Transformer (big), 1.5M sentences, train_steps=300000, Averaged the last 20 ckpts, by Tensor2Tensor.
12	NICT-5	en-ja	2018/09/03 16:49:57	2219	42.87	44.42	43.49	-	-	-	-	-	0.00	0.00	NMT	No	Big Bidirectional Transformer. 1.5M sentences only.
13	AISTAI	en-ja	2019/07/29 07:33:56	3251	42.64	44.17	43.34	-	-	-	-	-	-	-	NMT	No	Transformer (big). 1.5M sentences, train_steps=131000 only. Averaged the last 10 ckpts.
14	srcb	en-ja	2018/09/16 15:26:37	2479	42.49	44.11	43.20	-	-	-	-	-	0.00	0.00	NMT	No	Transformer with relative position, ensemble of 3 models.
15	NICT-5	en-ja	2018/08/22 18:43:00	2048	41.91	43.50	42.60	-	-	-	-	-	0.00	0.00	NMT	No	Tensor2Tensor's transformer implementation. Used recurrently stacked layers model logic. Layer recurrence logic: 1-2-3-1-2-3-1-2-3-1-2-3. Codename: Megarecurrence.
16	Kyoto-U	en-ja	2017/09/04 22:31:18	1759	41.53	43.26	42.13	-	-	-	-	0.00	0.00	0.00	SMT	No	Syscomb of AIAYN and KNMT
17	ORGANIZER	en-ja	2017/08/02 01:04:49	1737	40.79	42.55	41.50	-	-	-	-	0.00	0.00	0.00	NMT	No	Google's "Attention Is All You Need"
18	NICT-2	en-ja	2017/07/26 14:03:00	1479	40.17	42.25	41.17	-	-	-	-	0.00	0.00	0.00	NMT	No	NMT 6 Ensembles * Bi-directional Reranking
19	NTT	en-ja	2017/08/01 15:42:13	1729	40.32	42.80	40.95	-	-	-	-	0.00	0.00	0.00	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
20	NICT-5	en-ja	2018/08/22 18:48:51	2050	40.02	41.74	40.77	-	-	-	-	-	0.00	0.00	NMT	No	RSNMT 6 layer
21	NTT	en-ja	2017/08/01 07:12:25	1684	39.80	42.27	40.47	-	-	-	-	0.00	0.00	0.00	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
22	NTT	en-ja	2017/08/01 02:22:56	1673	38.87	41.42	39.71	-	-	-	-	0.00	0.00	0.00	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
23	Kyoto-U	en-ja	2017/08/01 15:49:51	1731	38.72	40.65	39.37	-	-	-	-	0.00	0.00	0.00	NMT	No	Ensemble of 4 BPE Averaged
24	UT-KAY	en-ja	2017/06/27 16:58:42	1369	38.32	41.12	39.32	-	-	-	-	0.00	0.00	0.00	NMT	No	Hashimoto and Tsuruoka (2017), https://arxiv.org/abs/1702.02265
25	Osaka-U	en-ja	2018/09/15 23:02:05	2439	38.01	40.00	39.10	-	-	-	-	-	0.00	0.00	NMT	Yes	rewarding model
26	NAIST-NICT	en-ja	2017/07/27 21:50:15	1507	38.25	40.29	39.05	-	-	-	-	0.00	0.00	0.00	NMT	No	Single Model: SPM16k/16k, BiLSTM Encoder 51222, UniLSTM Decoder 512*2, Adjusted Search
27	EHR	en-ja	2018/09/08 12:42:03	2245	37.97	40.00	38.66	-	-	-	-	-	0.00	0.00	NMT	No	SMT reranked NMT
28	NTT	en-ja	2017/07/30 16:16:25	1608	37.90	40.48	38.61	-	-	-	-	0.00	0.00	0.00	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
29	NICT-2	en-ja	2017/07/26 13:52:35	1475	36.85	38.94	37.87	-	-	-	-	0.00	0.00	0.00	NMT	No	NMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
30	NAIST-NICT	en-ja	2017/07/27 21:49:04	1506	36.47	38.54	37.30	-	-	-	-	0.00	0.00	0.00	NMT	No	Single Model: SPM16k/16k, BiLSTM Encoder 51222, UniLSTM Decoder 512*2, One-best Search
31	ORGANIZER	en-ja	2018/08/14 10:58:49	1900	36.37	38.48	37.15	-	-	-	-	-	0.00	0.00	NMT	No	NMT with Attention
32	UT-IIS	en-ja	2017/08/01 12:03:22	1710	36.26	38.93	37.06	-	-	-	-	0.00	0.00	0.00	NMT	No	NMT 8 Ensembles with beam search, Sentence Piece, Embedding layer initialization
33	TMU	en-ja	2017/08/04 11:06:51	1747	35.73	38.34	37.00	-	-	-	-	0.00	0.00	0.00	NMT	No	beam_size: 10, ensemble of different dropout rates.
34	Kyoto-U	en-ja	2016/08/18 03:45:51	1172	36.19	38.20	36.78	-	-	-	-	0.00	0.00	0.00	NMT	No	BPE tgt/src: 52k 2-layer lstm self-ensemble of 3
35	NAIST	en-ja	2015/08/25 12:39:23	761	35.83	38.17	36.61	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Travatar System with NeuralMT Reranking
36	TMU	en-ja	2018/09/16 12:44:23	2469	35.08	37.69	36.14	-	-	-	-	-	0.00	0.00	NMT	No	Ensemble of 6 Baseline-NMT
37	Sense	en-ja	2018/08/24 11:38:23	2092	35.29	37.21	36.02	-	-	-	-	-	0.00	0.00	NMT	No
38	NAIST	en-ja	2014/07/31 11:38:37	118	35.03	37.16	35.81	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Travatar-based Forest-to-String SMT System
39	NAIST	en-ja	2014/08/01 17:37:23	126	34.84	37.15	35.67	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Travatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
40	TMU	en-ja	2017/08/02 10:20:14	1741	34.58	36.95	35.63	-	-	-	-	0.00	0.00	0.00	NMT	No	the ensemble system of different dropout rate.
41	TMU	en-ja	2018/09/16 17:25:36	2484	34.38	37.06	35.44	-	-	-	-	-	0.00	0.00	NMT	No	Ensemble of 6 NMT ( 2 Baseline + 2 Reconstructor + 2 GAN )
42	NICT-2	en-ja	2016/08/05 17:47:05	1097	34.67	36.86	35.37	-	-	-	-	0.00	0.00	0.00	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
43	TMU	en-ja	2017/08/01 11:56:31	1709	34.05	36.69	35.32	-	-	-	-	0.00	0.00	0.00	NMT	No	baseline system with beam20
44	naver	en-ja	2015/08/31 14:18:18	837	34.60	36.14	35.30	-	-	-	0.00	0.00	0.00	0.00	SMT	No	SMT t2s + Spell correction + NMT reranking
45	TMU	en-ja	2018/09/16 12:10:49	2467	34.02	36.69	35.17	-	-	-	-	-	0.00	0.00	NMT	No	Baseline-NMT ( Single )
46	NAIST	en-ja	2015/08/25 12:47:49	763	34.38	36.58	35.16	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Travatar System Baseline
47	TMU	en-ja	2018/09/16 12:11:58	2468	34.00	36.65	35.09	-	-	-	-	-	0.00	0.00	NMT	No	GAN-NMT ( Single )
48	TMU	en-ja	2018/09/16 16:56:49	2482	33.86	36.58	34.89	-	-	-	-	-	0.00	0.00	NMT	No	Reconstructor-NMT ( Single )
49	UT-AKY	en-ja	2016/08/20 12:34:43	1228	33.57	36.95	34.65	-	-	-	-	0.00	0.00	0.00	NMT	No	tree-to-seq NMT model (word-based decoder)
50	W	en-ja	2015/08/26 16:00:46	786	33.23	36.21	34.05	-	-	-	0.00	0.00	0.00	0.00	SMT	No	NMT, LSTM Search, 5 ensembles, beam size 20, UNK replacing, System Combination with NMT score (Pick top-1k results from NMT)
51	Kyoto-U	en-ja	2015/08/30 19:59:18	832	33.06	35.57	33.99	-	-	-	0.00	0.00	0.00	0.00	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking
52	naver	en-ja	2015/08/31 14:12:03	836	33.14	35.75	33.93	-	-	-	0.00	0.00	0.00	0.00	SMT	No	NMT only
53	TMU	en-ja	2017/08/01 11:42:03	1704	32.65	35.05	33.72	-	-	-	-	0.00	0.00	0.00	NMT	No	our baseline system in 2017
54	naver	en-ja	2015/08/25 16:20:30	770	32.76	34.53	33.47	-	-	-	0.00	0.00	0.00	0.00	SMT	No	SMT t2s + Spell correction
55	W	en-ja	2014/08/26 16:17:15	202	32.69	35.04	33.40	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Weblio Pre-reordering SMT System (with forest inputs)
56	naver	en-ja	2015/08/04 16:48:32	581	32.63	34.46	33.34	-	-	-	0.00	0.00	0.00	0.00	SMT	No	SMT t2s
57	W	en-ja	2014/08/16 00:57:16	132	32.53	34.87	33.26	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Weblio Pre-reordering SMT System Baseline
58	TOKYOMT	en-ja	2016/08/19 23:18:46	1217	32.03	34.77	32.98	-	-	-	-	0.00	0.00	0.00	NMT	No	Combination of NMT and T2S
59	TOSHIBA	en-ja	2015/07/28 16:24:37	524	32.06	34.17	32.76	-	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
60	TOSHIBA	en-ja	2015/07/30 11:22:11	540	31.82	33.94	32.54	-	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	System combination SMT and RBMT(SPE) with RNNLM language model
61	EHR	en-ja	2016/08/15 11:21:51	1140	31.32	33.58	32.28	-	-	-	-	0.00	0.00	0.00	SMT	No	PBSMT with preordering (DL=6)
62	ORGANIZER	en-ja	2014/07/11 20:03:21	12	31.05	33.44	32.10	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Tree-to-String SMT (2014)
63	ORGANIZER	en-ja	2015/09/10 13:29:35	875	31.05	33.44	32.10	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Tree-to-String SMT (2015)
64	Kyoto-U	en-ja	2016/08/19 10:18:09	1201	31.03	33.40	31.88	-	-	-	-	0.00	0.00	0.00	EBMT	No	KyotoEBMT 2016 w/o reranking
65	bjtu_nlp	en-ja	2016/08/16 11:21:07	1143	31.18	33.47	31.80	-	-	-	-	0.00	0.00	0.00	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
66	EHR	en-ja	2015/09/09 12:21:34	873	30.88	33.20	31.74	-	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	System combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
67	Kyoto-U	en-ja	2014/09/01 21:06:32	267	31.09	33.55	31.73	-	-	-	0.00	0.00	0.00	0.00	EBMT	No	Our new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
68	Kyoto-U	en-ja	2015/08/27 22:56:13	805	30.69	33.25	31.71	-	-	-	0.00	0.00	0.00	0.00	EBMT	No	KyotoEBMT system without reranking
69	W	en-ja	2015/08/28 14:30:56	813	30.72	34.19	31.57	-	-	-	0.00	0.00	0.00	0.00	Other	No	NMT, LSTM Search, Beam Size 20, Ensemble of 2 models, UNK replacing
70	SAS_MT	en-ja	2014/09/01 10:39:27	264	30.47	33.00	31.47	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Syntactic reordering Hierarchical SMT （using part of data）
71	EHR	en-ja	2015/09/09 12:22:58	874	30.46	32.56	31.28	-	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	RBMT with user dictionary plus SPE.
72	TOKYOMT	en-ja	2016/08/12 11:21:43	1131	30.21	33.38	31.24	-	-	-	-	0.00	0.00	0.00	NMT	No	char 1 , ens 2 , version 1
73	UT-AKY	en-ja	2016/08/20 09:25:15	1224	30.14	33.20	31.09	-	-	-	-	0.00	0.00	0.00	NMT	No	tree-to-seq NMT model (character-based decoder)
74	ORGANIZER	en-ja	2014/09/16 13:36:35	367	30.19	32.56	30.94	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Hierarchical Phrase-based SMT (2014)
75	Kyoto-U	en-ja	2014/08/25 14:16:15	186	30.25	32.78	30.84	-	-	-	0.00	0.00	0.00	0.00	EBMT	No	Using n-best parses and RNNLM.
76	EHR	en-ja	2015/08/22 12:28:15	742	29.78	32.36	30.71	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Phrase based SMT with preordering.
77	Kyoto-U	en-ja	2014/08/31 10:33:23	253	29.76	32.46	30.46	-	-	-	0.00	0.00	0.00	0.00	EBMT	No	Our new baseline system after several modifications.
78	Kyoto-U	en-ja	2014/08/19 10:16:02	134	28.93	31.61	29.59	-	-	-	0.00	0.00	0.00	0.00	EBMT	No	Our baseline system using 3M parallel sentences.
79	Sense	en-ja	2015/08/28 19:22:24	821	27.91	30.03	28.78	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Baseline-2015 (train1 only)
80	Sense	en-ja	2014/08/25 01:07:27	184	27.92	30.18	28.66	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Baseline SMT
81	ORGANIZER	en-ja	2014/07/11 19:49:03	5	27.48	29.80	28.27	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Phrase-based SMT
82	ORGANIZER	en-ja	2016/11/16 10:50:13	1334	26.19	28.22	26.68	-	-	-	-	0.00	0.00	0.00	NMT	Yes	Online A (2016/11/14)
83	Sense	en-ja	2015/07/28 22:26:55	531	25.22	27.22	25.90	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Baseline-2015
84	Sense	en-ja	2015/08/18 22:04:09	715	24.43	26.58	25.36	-	-	-	0.00	0.00	0.00	0.00	SMT	Yes	Passive JSTx3
85	Sense	en-ja	2015/08/18 21:52:01	700	24.13	26.24	24.96	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Baseline-dictmt
86	Osaka-U	en-ja	2018/09/16 13:06:45	2470	23.24	25.50	24.26	-	-	-	-	-	0.00	0.00	SMT	No	preordering with neural network
87	JAPIO	en-ja	2016/08/17 12:50:50	1165	20.52	22.56	21.05	-	-	-	-	0.00	0.00	0.00	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus + rule-based posteditor
88	ORGANIZER	en-ja	2014/07/18 11:02:25	34	19.66	21.63	20.17	-	-	-	0.00	0.00	0.00	0.00	Other	Yes	Online A (2014)
89	ORGANIZER	en-ja	2016/07/26 11:31:47	1041	18.28	19.81	18.51	-	-	-	-	0.00	0.00	0.00	Other	Yes	Online A (2016)
90	ORGANIZER	en-ja	2015/08/25 18:54:29	774	18.22	19.77	18.46	-	-	-	0.00	0.00	0.00	0.00	Other	Yes	Online A (2015)
91	ORGANIZER	en-ja	2015/09/10 19:02:38	889	17.80	19.52	18.11	-	-	-	0.00	0.00	0.00	0.00	Other	Yes	Online B (2015)
92	ORGANIZER	en-ja	2014/07/22 13:30:13	91	17.04	18.67	17.36	-	-	-	0.00	0.00	0.00	0.00	Other	Yes	Online B (2014)
93	ORGANIZER	en-ja	2014/07/21 11:38:12	66	13.18	14.85	13.48	-	-	-	0.00	0.00	0.00	0.00	Other	Yes	RBMT B (2014)
94	ORGANIZER	en-ja	2015/09/10 14:26:28	883	13.18	14.85	13.48	-	-	-	0.00	0.00	0.00	0.00	Other	Yes	RBMT B (2015)
95	ORGANIZER	en-ja	2014/07/21 11:40:50	68	12.86	14.43	13.16	-	-	-	0.00	0.00	0.00	0.00	Other	Yes	RBMT A
96	ORGANIZER	en-ja	2014/07/23 14:50:44	95	12.19	13.32	12.14	-	-	-	0.00	0.00	0.00	0.00	Other	Yes	RBMT C

Notice:

This table is sorted by the leftmost segmenters. You can change the segmenter used to sort by clicking each segmenter link.

RIBES

#	Team	Task	Date/Time	DataID	RIBES										Method	Other Resources	System Description
#	Team	Task	Date/Time	DataID	juman	kytea	mecab	moses- tokenizer	stanford- segmenter- ctb	stanford- segmenter- pku	indic- tokenizer	unuse	myseg	kmseg	Method	Other Resources	System Description
1	NTT	en-ja	2019/07/28 15:44:12	3236	0.861994	0.865640	0.868175	-	-	-	-	-	-	-	NMT	No	ASPEC first 1.5M + Synthetic 1.5M, 6 ensemble
2	NTT	en-ja	2019/07/28 15:55:18	3239	0.859673	0.863532	0.866183	-	-	-	-	-	-	-	NMT	Yes	ParaCrawl + (ASPEC first 1.5M + Synthetic 1.5M) * 2 oversampling, fine-tune ASPEC, SINGLE MODEL
3	KNU_Hyundai	en-ja	2019/07/27 09:19:31	3172	0.857060	0.859885	0.863400	-	-	-	-	-	-	-	NMT	No	Transformer Base, relative position, BT, r2l reranking, ensemble of 3 models
4	srcb	en-ja	2019/07/27 15:28:30	3206	0.856245	0.859025	0.863177	-	-	-	-	-	-	-	NMT	No	Transformer(Big) with relative position, sentence-wise smooth, deep transformer, back translation, ensemble of 7 models, re-rank.
5	srcb	en-ja	2019/07/27 15:50:19	3212	0.857506	0.860642	0.862819	-	-	-	-	-	-	-	NMT	No	Transformer(Big) with relative position, sentence-wise smooth, deep transformer, back translation, ensemble of 10 models, rerank.
6	AISTAI	en-ja	2019/08/30 20:23:24	3357	0.856082	0.859166	0.862434	-	-	-	-	-	-	-	NMT	No	Transformer, 1.5M sentences, relative position, ensemble of 3 models, by OpenNMT-py.
7	NICT-2	en-ja	2019/07/27 10:57:26	3182	0.852970	0.856755	0.859059	-	-	-	-	-	-	-	NMT	No	Transformer, ensemble of 4 models w/ long warm-up and self-training
8	NICT-2	en-ja	2019/07/27 10:56:02	3181	0.850458	0.853909	0.857275	-	-	-	-	-	-	-	NMT	No	Transformer, sigle model w/ long warm-up and self-training
9	srcb	en-ja	2018/09/16 15:26:37	2479	0.850318	0.852209	0.857017	-	-	-	-	-	0.000000	0.000000	NMT	No	Transformer with relative position, ensemble of 3 models.
10	AISTAI	en-ja	2019/07/29 07:33:56	3251	0.849129	0.851400	0.856177	-	-	-	-	-	-	-	NMT	No	Transformer (big). 1.5M sentences, train_steps=131000 only. Averaged the last 10 ckpts.
11	srcb	en-ja	2019/07/25 11:43:06	2918	0.848929	0.852948	0.855857	-	-	-	-	-	-	-	NMT	No	Transformer (Big) with relative position, sentence-wise smooth
12	srcb	en-ja	2018/09/16 15:52:17	2480	0.847678	0.850180	0.855073	-	-	-	-	-	0.000000	0.000000	NMT	No	Transformer with relative position, ensemble of 4 models, rerank.
13	AISTAI	en-ja	2019/09/05 08:18:45	3373	0.847863	0.851673	0.854716	-	-	-	-	-	-	-	NMT	No	Transformer (big), 1.5M sentences, train_steps=300000, Averaged the last 20 ckpts, by Tensor2Tensor.
14	NICT-5	en-ja	2018/09/03 16:49:57	2219	0.847134	0.849399	0.853634	-	-	-	-	-	0.000000	0.000000	NMT	No	Big Bidirectional Transformer. 1.5M sentences only.
15	ORGANIZER	en-ja	2017/08/02 01:04:49	1737	0.844896	0.847559	0.851471	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Google's "Attention Is All You Need"
16	NICT-2	en-ja	2017/07/26 14:03:00	1479	0.842206	0.848170	0.849929	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT 6 Ensembles * Bi-directional Reranking
17	NICT-5	en-ja	2018/08/22 18:43:00	2048	0.840776	0.845042	0.849326	-	-	-	-	-	0.000000	0.000000	NMT	No	Tensor2Tensor's transformer implementation. Used recurrently stacked layers model logic. Layer recurrence logic: 1-2-3-1-2-3-1-2-3-1-2-3. Codename: Megarecurrence.
18	NICT-5	en-ja	2018/08/22 18:48:51	2050	0.841011	0.844360	0.848283	-	-	-	-	-	0.000000	0.000000	NMT	No	RSNMT 6 layer
19	Kyoto-U	en-ja	2017/09/04 22:31:18	1759	0.841000	0.843725	0.847819	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Syscomb of AIAYN and KNMT
20	NTT	en-ja	2017/08/01 15:42:13	1729	0.838594	0.841769	0.846486	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
21	NTT	en-ja	2017/08/01 07:12:25	1684	0.835806	0.839981	0.844326	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
22	NAIST-NICT	en-ja	2017/07/27 21:50:15	1507	0.834492	0.839321	0.842337	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: SPM16k/16k, BiLSTM Encoder 51222, UniLSTM Decoder 512*2, Adjusted Search
23	NTT	en-ja	2017/08/01 02:22:56	1673	0.833541	0.838245	0.841739	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
24	Kyoto-U	en-ja	2017/08/01 15:49:51	1731	0.832472	0.835870	0.839646	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble of 4 BPE Averaged
25	UT-KAY	en-ja	2017/06/27 16:58:42	1369	0.829600	0.833920	0.838260	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Hashimoto and Tsuruoka (2017), https://arxiv.org/abs/1702.02265
26	EHR	en-ja	2018/09/08 12:42:03	2245	0.828746	0.833333	0.837806	-	-	-	-	-	0.000000	0.000000	NMT	No	SMT reranked NMT
27	UT-IIS	en-ja	2017/08/01 12:03:22	1710	0.827891	0.832054	0.836394	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT 8 Ensembles with beam search, Sentence Piece, Embedding layer initialization
28	NTT	en-ja	2017/07/30 16:16:25	1608	0.828193	0.832006	0.836381	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
29	NICT-2	en-ja	2017/07/26 13:52:35	1475	0.826791	0.834448	0.835255	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
30	ORGANIZER	en-ja	2018/08/14 10:58:49	1900	0.824985	0.831183	0.833207	-	-	-	-	-	0.000000	0.000000	NMT	No	NMT with Attention
31	Osaka-U	en-ja	2018/09/15 23:02:05	2439	0.825061	0.829328	0.833200	-	-	-	-	-	0.000000	0.000000	NMT	Yes	rewarding model
32	TMU	en-ja	2017/08/04 11:06:51	1747	0.824801	0.829580	0.832569	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	beam_size: 10, ensemble of different dropout rates.
33	TMU	en-ja	2018/09/16 12:44:23	2469	0.823653	0.829156	0.831219	-	-	-	-	-	0.000000	0.000000	NMT	No	Ensemble of 6 Baseline-NMT
34	NAIST-NICT	en-ja	2017/07/27 21:49:04	1506	0.821989	0.827225	0.830116	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: SPM16k/16k, BiLSTM Encoder 51222, UniLSTM Decoder 512*2, One-best Search
35	TMU	en-ja	2018/09/16 17:25:36	2484	0.820961	0.826779	0.829178	-	-	-	-	-	0.000000	0.000000	NMT	No	Ensemble of 6 NMT ( 2 Baseline + 2 Reconstructor + 2 GAN )
36	Kyoto-U	en-ja	2016/08/18 03:45:51	1172	0.819836	0.823878	0.828956	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	BPE tgt/src: 52k 2-layer lstm self-ensemble of 3
37	TMU	en-ja	2018/09/16 12:11:58	2468	0.819148	0.825367	0.827650	-	-	-	-	-	0.000000	0.000000	NMT	No	GAN-NMT ( Single )
38	UT-AKY	en-ja	2016/08/20 12:34:43	1228	0.816984	0.824456	0.827647	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	tree-to-seq NMT model (word-based decoder)
39	TMU	en-ja	2018/09/16 12:10:49	2467	0.817948	0.824239	0.827386	-	-	-	-	-	0.000000	0.000000	NMT	No	Baseline-NMT ( Single )
40	TMU	en-ja	2018/09/16 16:56:49	2482	0.817153	0.824066	0.826013	-	-	-	-	-	0.000000	0.000000	NMT	No	Reconstructor-NMT ( Single )
41	TMU	en-ja	2017/08/02 10:20:14	1741	0.817280	0.824332	0.825683	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	the ensemble system of different dropout rate.
42	TMU	en-ja	2017/08/01 11:56:31	1709	0.812926	0.818443	0.821563	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	baseline system with beam20
43	NAIST	en-ja	2015/08/25 12:39:23	761	0.811479	0.813827	0.820337	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System with NeuralMT Reranking
44	TOKYOMT	en-ja	2016/08/12 11:21:43	1131	0.809691	0.817258	0.819951	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	char 1 , ens 2 , version 1
45	TOKYOMT	en-ja	2016/08/19 23:18:46	1217	0.808189	0.814452	0.818130	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Combination of NMT and T2S
46	naver	en-ja	2015/08/31 14:12:03	836	0.807280	0.811487	0.817343	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	NMT only
47	UT-AKY	en-ja	2016/08/20 09:25:15	1224	0.806025	0.814490	0.815836	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	tree-to-seq NMT model (character-based decoder)
48	W	en-ja	2015/08/26 16:00:46	786	0.804722	0.809065	0.814337	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	NMT, LSTM Search, 5 ensembles, beam size 20, UNK replacing, System Combination with NMT score (Pick top-1k results from NMT)
49	NAIST	en-ja	2014/08/01 17:37:23	126	0.801742	0.807010	0.811081	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
50	TMU	en-ja	2017/08/01 11:42:03	1704	0.802262	0.809649	0.811057	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	our baseline system in 2017
51	naver	en-ja	2015/08/31 14:18:18	837	0.799966	0.803154	0.808787	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	SMT t2s + Spell correction + NMT reranking
52	W	en-ja	2015/08/28 14:30:56	813	0.796863	0.802666	0.807186	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	No	NMT, LSTM Search, Beam Size 20, Ensemble of 2 models, UNK replacing
53	NAIST	en-ja	2014/07/31 11:38:37	118	0.796079	0.801520	0.806581	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar-based Forest-to-String SMT System
54	Sense	en-ja	2018/08/24 11:38:23	2092	0.796214	0.800727	0.803453	-	-	-	-	-	0.000000	0.000000	NMT	No
55	NAIST	en-ja	2015/08/25 12:47:49	763	0.792447	0.796489	0.802228	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System Baseline
56	Kyoto-U	en-ja	2015/08/30 19:59:18	832	0.789514	0.797182	0.799979	-	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking
57	W	en-ja	2014/08/26 16:17:15	202	0.785015	0.790066	0.795027	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Weblio Pre-reordering SMT System (with forest inputs)
58	naver	en-ja	2015/08/25 16:20:30	770	0.784174	0.788811	0.794452	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	SMT t2s + Spell correction
59	naver	en-ja	2015/08/04 16:48:32	581	0.783310	0.788165	0.793723	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	SMT t2s
60	NICT-2	en-ja	2016/08/05 17:47:05	1097	0.784335	0.790993	0.793409	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
61	W	en-ja	2014/08/16 00:57:16	132	0.782066	0.786902	0.792616	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Weblio Pre-reordering SMT System Baseline
62	bjtu_nlp	en-ja	2016/08/16 11:21:07	1143	0.780510	0.787497	0.791088	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
63	ORGANIZER	en-ja	2016/11/16 10:50:13	1334	0.776787	0.780217	0.782674	-	-	-	-	0.000000	0.000000	0.000000	NMT	Yes	Online A (2016/11/14)
64	Kyoto-U	en-ja	2016/08/19 10:18:09	1201	0.771240	0.779774	0.781960	-	-	-	-	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT 2016 w/o reranking
65	TOSHIBA	en-ja	2015/07/28 16:24:37	524	0.770989	0.778570	0.780467	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
66	TOSHIBA	en-ja	2015/07/30 11:22:11	540	0.769865	0.776841	0.779574	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	System combination SMT and RBMT(SPE) with RNNLM language model
67	Kyoto-U	en-ja	2015/08/27 22:56:13	805	0.767778	0.776672	0.778358	-	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT system without reranking
68	EHR	en-ja	2015/09/09 12:21:34	873	0.765700	0.774819	0.778269	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	System combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
69	EHR	en-ja	2015/09/09 12:22:58	874	0.768563	0.776526	0.778113	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with user dictionary plus SPE.
70	EHR	en-ja	2016/08/15 11:21:51	1140	0.759914	0.771427	0.775023	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	PBSMT with preordering (DL=6)
71	SAS_MT	en-ja	2014/09/01 10:39:27	264	0.759415	0.770948	0.771605	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Syntactic reordering Hierarchical SMT （using part of data）
72	Kyoto-U	en-ja	2014/09/01 21:06:32	267	0.759608	0.770908	0.771545	-	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
73	EHR	en-ja	2015/08/22 12:28:15	742	0.753576	0.766044	0.768105	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Phrase based SMT with preordering.
74	Kyoto-U	en-ja	2014/08/25 14:16:15	186	0.755629	0.765251	0.766495	-	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	Using n-best parses and RNNLM.
75	Kyoto-U	en-ja	2014/08/31 10:33:23	253	0.752058	0.764049	0.766435	-	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our new baseline system after several modifications.
76	ORGANIZER	en-ja	2014/07/11 20:03:21	12	0.748883	0.758031	0.760516	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Tree-to-String SMT (2014)
77	ORGANIZER	en-ja	2015/09/10 13:29:35	875	0.748883	0.758031	0.760516	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Tree-to-String SMT (2015)
78	Kyoto-U	en-ja	2014/08/19 10:16:02	134	0.743969	0.755744	0.756545	-	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our baseline system using 3M parallel sentences.
79	ORGANIZER	en-ja	2014/09/16 13:36:35	367	0.734705	0.746978	0.747722	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Hierarchical Phrase-based SMT (2014)
80	JAPIO	en-ja	2016/08/17 12:50:50	1165	0.723467	0.728584	0.731474	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus + rule-based posteditor
81	Osaka-U	en-ja	2018/09/16 13:06:45	2470	0.716889	0.726469	0.729323	-	-	-	-	-	0.000000	0.000000	SMT	No	preordering with neural network
82	ORGANIZER	en-ja	2014/07/18 11:02:25	34	0.718019	0.723486	0.725848	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online A (2014)
83	ORGANIZER	en-ja	2016/07/26 11:31:47	1041	0.706639	0.715222	0.718559	-	-	-	-	0.000000	0.000000	0.000000	Other	Yes	Online A (2016)
84	ORGANIZER	en-ja	2015/08/25 18:54:29	774	0.705882	0.713960	0.718150	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online A (2015)
85	ORGANIZER	en-ja	2015/09/10 19:02:38	889	0.693359	0.701966	0.703859	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online B (2015)
86	Sense	en-ja	2014/08/25 01:07:27	184	0.690464	0.700583	0.703049	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline SMT
87	Sense	en-ja	2015/08/28 19:22:24	821	0.689313	0.697817	0.700994	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-2015 (train1 only)
88	ORGANIZER	en-ja	2014/07/22 13:30:13	91	0.687797	0.693390	0.698126	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online B (2014)
89	ORGANIZER	en-ja	2014/07/11 19:49:03	5	0.683735	0.691926	0.695390	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Phrase-based SMT
90	ORGANIZER	en-ja	2014/07/21 11:38:12	66	0.671958	0.680748	0.682683	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT B (2014)
91	ORGANIZER	en-ja	2015/09/10 14:26:28	883	0.671958	0.680748	0.682683	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT B (2015)
92	ORGANIZER	en-ja	2014/07/21 11:40:50	68	0.670167	0.676464	0.678934	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT A
93	ORGANIZER	en-ja	2014/07/23 14:50:44	95	0.668372	0.672645	0.676018	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT C
94	Sense	en-ja	2015/07/28 22:26:55	531	0.646288	0.653814	0.659505	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-2015
95	Sense	en-ja	2015/08/18 21:52:01	700	0.637378	0.642789	0.647831	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-dictmt
96	Sense	en-ja	2015/08/18 22:04:09	715	0.635933	0.641517	0.646682	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Passive JSTx3

Notice:

This table is sorted by the leftmost segmenters. You can change the segmenter used to sort by clicking each segmenter link.

AMFM

#	Team	Task	Date/Time	DataID	AMFM										Method	Other Resources	System Description
#	Team	Task	Date/Time	DataID	unuse	unuse	unuse	unuse	unuse	unuse	unuse	unuse	unuse	unuse	Method	Other Resources	System Description
1	srcb	en-ja	2018/09/16 15:26:37	2479	0.781000	0.781000	0.781000	-	-	-	-	-	0.000000	0.000000	NMT	No	Transformer with relative position, ensemble of 3 models.
2	srcb	en-ja	2018/09/16 15:52:17	2480	0.779820	0.779820	0.779820	-	-	-	-	-	0.000000	0.000000	NMT	No	Transformer with relative position, ensemble of 4 models, rerank.
3	NICT-5	en-ja	2018/09/03 16:49:57	2219	0.779560	0.779560	0.779560	-	-	-	-	-	0.000000	0.000000	NMT	No	Big Bidirectional Transformer. 1.5M sentences only.
4	NTT	en-ja	2019/07/28 15:44:12	3236	0.774950	0.774950	0.774950	-	-	-	-	-	-	-	NMT	No	ASPEC first 1.5M + Synthetic 1.5M, 6 ensemble
5	NTT	en-ja	2019/07/28 15:55:18	3239	0.772590	0.772590	0.772590	-	-	-	-	-	-	-	NMT	Yes	ParaCrawl + (ASPEC first 1.5M + Synthetic 1.5M) * 2 oversampling, fine-tune ASPEC, SINGLE MODEL
6	NICT-5	en-ja	2018/08/22 18:43:00	2048	0.771400	0.771400	0.771400	-	-	-	-	-	0.000000	0.000000	NMT	No	Tensor2Tensor's transformer implementation. Used recurrently stacked layers model logic. Layer recurrence logic: 1-2-3-1-2-3-1-2-3-1-2-3. Codename: Megarecurrence.
7	Kyoto-U	en-ja	2017/09/04 22:31:18	1759	0.771020	0.771020	0.771020	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Syscomb of AIAYN and KNMT
8	NICT-5	en-ja	2018/08/22 18:48:51	2050	0.770560	0.770560	0.770560	-	-	-	-	-	0.000000	0.000000	NMT	No	RSNMT 6 layer
9	NAIST-NICT	en-ja	2017/07/27 21:50:15	1507	0.770480	0.770480	0.770480	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: SPM16k/16k, BiLSTM Encoder 51222, UniLSTM Decoder 512*2, Adjusted Search
10	srcb	en-ja	2019/07/27 15:50:19	3212	0.770440	0.770440	0.770440	-	-	-	-	-	-	-	NMT	No	Transformer(Big) with relative position, sentence-wise smooth, deep transformer, back translation, ensemble of 10 models, rerank.
11	ORGANIZER	en-ja	2017/08/02 01:04:49	1737	0.768630	0.768630	0.768630	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Google's "Attention Is All You Need"
12	NICT-2	en-ja	2017/07/26 14:03:00	1479	0.765580	0.765580	0.765580	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT 6 Ensembles * Bi-directional Reranking
13	srcb	en-ja	2019/07/27 15:28:30	3206	0.765330	0.765330	0.765330	-	-	-	-	-	-	-	NMT	No	Transformer(Big) with relative position, sentence-wise smooth, deep transformer, back translation, ensemble of 7 models, re-rank.
14	NTT	en-ja	2017/08/01 02:22:56	1673	0.763770	0.763770	0.763770	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
15	NAIST-NICT	en-ja	2017/07/27 21:49:04	1506	0.763310	0.763310	0.763310	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: SPM16k/16k, BiLSTM Encoder 51222, UniLSTM Decoder 512*2, One-best Search
16	Osaka-U	en-ja	2018/09/15 23:02:05	2439	0.763140	0.763140	0.763140	-	-	-	-	-	0.000000	0.000000	NMT	Yes	rewarding model
17	NTT	en-ja	2017/08/01 15:42:13	1729	0.762170	0.762170	0.762170	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
18	NICT-2	en-ja	2019/07/27 10:57:26	3182	0.761340	0.761340	0.761340	-	-	-	-	-	-	-	NMT	No	Transformer, ensemble of 4 models w/ long warm-up and self-training
19	AISTAI	en-ja	2019/08/30 20:23:24	3357	0.760410	0.760410	0.760410	-	-	-	-	-	-	-	NMT	No	Transformer, 1.5M sentences, relative position, ensemble of 3 models, by OpenNMT-py.
20	KNU_Hyundai	en-ja	2019/07/27 09:19:31	3172	0.760050	0.760050	0.760050	-	-	-	-	-	-	-	NMT	No	Transformer Base, relative position, BT, r2l reranking, ensemble of 3 models
21	AISTAI	en-ja	2019/09/05 08:18:45	3373	0.759940	0.759940	0.759940	-	-	-	-	-	-	-	NMT	No	Transformer (big), 1.5M sentences, train_steps=300000, Averaged the last 20 ckpts, by Tensor2Tensor.
22	ORGANIZER	en-ja	2018/08/14 10:58:49	1900	0.759910	0.759910	0.759910	-	-	-	-	-	0.000000	0.000000	NMT	No	NMT with Attention
23	NICT-2	en-ja	2017/07/26 13:52:35	1475	0.759570	0.759570	0.759570	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
24	srcb	en-ja	2019/07/25 11:43:06	2918	0.759510	0.759510	0.759510	-	-	-	-	-	-	-	NMT	No	Transformer (Big) with relative position, sentence-wise smooth
25	NICT-2	en-ja	2019/07/27 10:56:02	3181	0.759260	0.759260	0.759260	-	-	-	-	-	-	-	NMT	No	Transformer, sigle model w/ long warm-up and self-training
26	EHR	en-ja	2018/09/08 12:42:03	2245	0.758750	0.758750	0.758750	-	-	-	-	-	0.000000	0.000000	NMT	No	SMT reranked NMT
27	NAIST	en-ja	2014/08/01 17:37:23	126	0.758740	0.758740	0.758740	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
28	NTT	en-ja	2017/08/01 07:12:25	1684	0.757740	0.757740	0.757740	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
29	AISTAI	en-ja	2019/07/29 07:33:56	3251	0.757590	0.757590	0.757590	-	-	-	-	-	-	-	NMT	No	Transformer (big). 1.5M sentences, train_steps=131000 only. Averaged the last 10 ckpts.
30	NTT	en-ja	2017/07/30 16:16:25	1608	0.756480	0.756480	0.756480	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
31	Kyoto-U	en-ja	2017/08/01 15:49:51	1731	0.754220	0.754220	0.754220	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble of 4 BPE Averaged
32	UT-KAY	en-ja	2017/06/27 16:58:42	1369	0.753090	0.753090	0.753090	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Hashimoto and Tsuruoka (2017), https://arxiv.org/abs/1702.02265
33	NICT-2	en-ja	2016/08/05 17:47:05	1097	0.753080	0.753080	0.753080	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
34	TMU	en-ja	2018/09/16 12:44:23	2469	0.753040	0.753040	0.753040	-	-	-	-	-	0.000000	0.000000	NMT	No	Ensemble of 6 Baseline-NMT
35	NAIST	en-ja	2014/07/31 11:38:37	118	0.752850	0.752850	0.752850	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar-based Forest-to-String SMT System
36	TMU	en-ja	2018/09/16 17:25:36	2484	0.752420	0.752420	0.752420	-	-	-	-	-	0.000000	0.000000	NMT	No	Ensemble of 6 NMT ( 2 Baseline + 2 Reconstructor + 2 GAN )
37	NAIST	en-ja	2015/08/25 12:47:49	763	0.752260	0.752260	0.752260	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System Baseline
38	TMU	en-ja	2018/09/16 16:56:49	2482	0.752100	0.752100	0.752100	-	-	-	-	-	0.000000	0.000000	NMT	No	Reconstructor-NMT ( Single )
39	TMU	en-ja	2017/08/02 10:20:14	1741	0.751660	0.751660	0.751660	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	the ensemble system of different dropout rate.
40	TMU	en-ja	2018/09/16 12:11:58	2468	0.750350	0.750350	0.750350	-	-	-	-	-	0.000000	0.000000	NMT	No	GAN-NMT ( Single )
41	TMU	en-ja	2017/08/04 11:06:51	1747	0.749410	0.749410	0.749410	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	beam_size: 10, ensemble of different dropout rates.
42	TMU	en-ja	2018/09/16 12:10:49	2467	0.749190	0.749190	0.749190	-	-	-	-	-	0.000000	0.000000	NMT	No	Baseline-NMT ( Single )
43	NAIST	en-ja	2015/08/25 12:39:23	761	0.748870	0.748870	0.748870	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System with NeuralMT Reranking
44	W	en-ja	2014/08/26 16:17:15	202	0.747650	0.747650	0.747650	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Weblio Pre-reordering SMT System (with forest inputs)
45	Kyoto-U	en-ja	2016/08/19 10:18:09	1201	0.747510	0.747510	0.747510	-	-	-	-	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT 2016 w/o reranking
46	UT-IIS	en-ja	2017/08/01 12:03:22	1710	0.746910	0.746910	0.746910	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT 8 Ensembles with beam search, Sentence Piece, Embedding layer initialization
47	EHR	en-ja	2016/08/15 11:21:51	1140	0.746720	0.746720	0.746720	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	PBSMT with preordering (DL=6)
48	W	en-ja	2014/08/16 00:57:16	132	0.746570	0.746570	0.746570	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Weblio Pre-reordering SMT System Baseline
49	TMU	en-ja	2017/08/01 11:56:31	1709	0.744890	0.744890	0.744890	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	baseline system with beam20
50	TOSHIBA	en-ja	2015/07/30 11:22:11	540	0.744660	0.744660	0.744660	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	System combination SMT and RBMT(SPE) with RNNLM language model
51	naver	en-ja	2015/08/31 14:18:18	837	0.744630	0.744630	0.744630	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	SMT t2s + Spell correction + NMT reranking
52	ORGANIZER	en-ja	2014/07/11 20:03:21	12	0.744370	0.744370	0.744370	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Tree-to-String SMT (2014)
53	ORGANIZER	en-ja	2015/09/10 13:29:35	875	0.744370	0.744370	0.744370	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Tree-to-String SMT (2015)
54	ORGANIZER	en-ja	2014/09/16 13:36:35	367	0.743900	0.743900	0.743900	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Hierarchical Phrase-based SMT (2014)
55	Kyoto-U	en-ja	2015/08/30 19:59:18	832	0.743710	0.743710	0.743710	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking
56	TOSHIBA	en-ja	2015/07/28 16:24:37	524	0.742100	0.742100	0.742100	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
57	Kyoto-U	en-ja	2014/08/31 10:33:23	253	0.741710	0.741710	0.741710	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our new baseline system after several modifications.
58	Sense	en-ja	2018/08/24 11:38:23	2092	0.741090	0.741090	0.741090	-	-	-	-	-	0.000000	0.000000	NMT	No
59	Kyoto-U	en-ja	2015/08/27 22:56:13	805	0.741050	0.741050	0.741050	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT system without reranking
60	TMU	en-ja	2017/08/01 11:42:03	1704	0.740620	0.740620	0.740620	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	our baseline system in 2017
61	SAS_MT	en-ja	2014/09/01 10:39:27	264	0.740580	0.740580	0.740580	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Syntactic reordering Hierarchical SMT （using part of data）
62	naver	en-ja	2015/08/25 16:20:30	770	0.740210	0.740210	0.740210	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	SMT t2s + Spell correction
63	Kyoto-U	en-ja	2016/08/18 03:45:51	1172	0.738700	0.738700	0.738700	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	BPE tgt/src: 52k 2-layer lstm self-ensemble of 3
64	Kyoto-U	en-ja	2014/09/01 21:06:32	267	0.738680	0.738680	0.738680	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
65	Kyoto-U	en-ja	2014/08/19 10:16:02	134	0.737890	0.737890	0.737890	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our baseline system using 3M parallel sentences.
66	naver	en-ja	2015/08/04 16:48:32	581	0.737590	0.737590	0.737590	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	SMT t2s
67	Sense	en-ja	2015/08/28 19:22:24	821	0.737560	0.737560	0.737560	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-2015 (train1 only)
68	ORGANIZER	en-ja	2014/07/11 19:49:03	5	0.736380	0.736380	0.736380	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Phrase-based SMT
69	Sense	en-ja	2014/08/25 01:07:27	184	0.733360	0.733360	0.733360	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline SMT
70	Kyoto-U	en-ja	2014/08/25 14:16:15	186	0.732380	0.732380	0.732380	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	Using n-best parses and RNNLM.
71	Sense	en-ja	2015/07/28 22:26:55	531	0.731620	0.731620	0.731620	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-2015
72	UT-AKY	en-ja	2016/08/20 12:34:43	1228	0.731440	0.731440	0.731440	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	tree-to-seq NMT model (word-based decoder)
73	Sense	en-ja	2015/08/18 22:04:09	715	0.731400	0.731400	0.731400	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Passive JSTx3
74	Sense	en-ja	2015/08/18 21:52:01	700	0.729200	0.729200	0.729200	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-dictmt
75	ORGANIZER	en-ja	2016/11/16 10:50:13	1334	0.727040	0.727040	0.727040	-	-	-	-	0.000000	0.000000	0.000000	NMT	Yes	Online A (2016/11/14)
76	naver	en-ja	2015/08/31 14:12:03	836	0.725860	0.725860	0.725860	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	NMT only
77	W	en-ja	2015/08/26 16:00:46	786	0.725370	0.725370	0.725370	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	NMT, LSTM Search, 5 ensembles, beam size 20, UNK replacing, System Combination with NMT score (Pick top-1k results from NMT)
78	TOKYOMT	en-ja	2016/08/19 23:18:46	1217	0.720810	0.720810	0.720810	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Combination of NMT and T2S
79	UT-AKY	en-ja	2016/08/20 09:25:15	1224	0.708140	0.708140	0.708140	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	tree-to-seq NMT model (character-based decoder)
80	W	en-ja	2015/08/28 14:30:56	813	0.705340	0.705340	0.705340	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	No	NMT, LSTM Search, Beam Size 20, Ensemble of 2 models, UNK replacing
81	TOKYOMT	en-ja	2016/08/12 11:21:43	1131	0.705210	0.705210	0.705210	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	char 1 , ens 2 , version 1
82	Osaka-U	en-ja	2018/09/16 13:06:45	2470	0.705050	0.705050	0.705050	-	-	-	-	-	0.000000	0.000000	SMT	No	preordering with neural network
83	bjtu_nlp	en-ja	2016/08/16 11:21:07	1143	0.704340	0.704340	0.704340	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
84	ORGANIZER	en-ja	2014/07/18 11:02:25	34	0.695420	0.695420	0.695420	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online A (2014)
85	ORGANIZER	en-ja	2015/08/25 18:54:29	774	0.677200	0.677200	0.677200	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online A (2015)
86	ORGANIZER	en-ja	2016/07/26 11:31:47	1041	0.677020	0.677020	0.677020	-	-	-	-	0.000000	0.000000	0.000000	Other	Yes	Online A (2016)
87	JAPIO	en-ja	2016/08/17 12:50:50	1165	0.660790	0.660790	0.660790	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus + rule-based posteditor
88	ORGANIZER	en-ja	2015/09/10 19:02:38	889	0.646160	0.646160	0.646160	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online B (2015)
89	ORGANIZER	en-ja	2014/07/22 13:30:13	91	0.643070	0.643070	0.643070	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online B (2014)
90	EHR	en-ja	2015/09/09 12:21:34	873	0.630410	0.630410	0.630410	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	System combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
91	ORGANIZER	en-ja	2014/07/21 11:40:50	68	0.626940	0.626940	0.626940	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT A
92	ORGANIZER	en-ja	2014/07/21 11:38:12	66	0.622930	0.622930	0.622930	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT B (2014)
93	ORGANIZER	en-ja	2015/09/10 14:26:28	883	0.622930	0.622930	0.622930	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT B (2015)
94	EHR	en-ja	2015/08/22 12:28:15	742	0.620500	0.620500	0.620500	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Phrase based SMT with preordering.
95	EHR	en-ja	2015/09/09 12:22:58	874	0.604090	0.604090	0.604090	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with user dictionary plus SPE.
96	ORGANIZER	en-ja	2014/07/23 14:50:44	95	0.594380	0.594380	0.594380	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT C

Notice:

This table is sorted by the leftmost segmenters. You can change the segmenter used to sort by clicking each segmenter link.
Adequacy-Fluency Metrics (AMFM) is a two-dimensional automatic evaluation metric for machine translation, designed to operate at the sentence level. It is based on adequacy and fluency, to decouple semantic and syntactic components of the translation process to provide a balanced view on translation quality.
AMFM is calculated without tokenizers.
The detail of AMFM is shown on the following paper: "Adequacy–Fluency Metrics: Evaluating MT in the Continuous Space Model Framework" [pdf]. Invited Talk in WAT2015 also helps understanding [slide].

HUMAN (WAT2022)

Notice:

HUMAN (WAT2022) is the result of the Pairwise Crowdsourcing Evaluation on WAT2022.
HUMAN (WAT2022) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2021)

Notice:

HUMAN (WAT2021) is the result of the Pairwise Crowdsourcing Evaluation on WAT2021.
HUMAN (WAT2021) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2020)

Notice:

HUMAN (WAT2020) is the result of the Pairwise Crowdsourcing Evaluation on WAT2020.
HUMAN (WAT2020) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2019)

Notice:

HUMAN (WAT2019) is the result of the Pairwise Crowdsourcing Evaluation on WAT2019.
HUMAN (WAT2019) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2018)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	NICT-5	en-ja	2018/09/03 16:49:57	2219	28.500	NMT	No	Big Bidirectional Transformer. 1.5M sentences only.
2	srcb	en-ja	2018/09/16 15:26:37	2479	25.000	NMT	No	Transformer with relative position, ensemble of 3 models.
3	NICT-5	en-ja	2018/08/22 18:43:00	2048	20.250	NMT	No	Tensor2Tensor's transformer implementation. Used recurrently stacked layers model logic. Layer recurrence logic: 1-2-3-1-2-3-1-2-3-1-2-3. Codename: Megarecurrence.
4	Osaka-U	en-ja	2018/09/15 23:02:05	2439	4.500	NMT	Yes	rewarding model
5	EHR	en-ja	2018/09/08 12:42:03	2245	-0.500	NMT	No	SMT reranked NMT
6	TMU	en-ja	2018/09/16 12:44:23	2469	-12.000	NMT	No	Ensemble of 6 Baseline-NMT
7	Osaka-U	en-ja	2018/09/16 13:06:45	2470	-82.250	SMT	No	preordering with neural network

Notice:

HUMAN (WAT2018) is the result of the Pairwise Crowdsourcing Evaluation on WAT2018.
HUMAN (WAT2018) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2017)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	NTT	en-ja	2017/08/01 15:42:13	1729	75.750	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
2	NICT-2	en-ja	2017/07/26 14:03:00	1479	74.750	NMT	No	NMT 6 Ensembles * Bi-directional Reranking
3	NTT	en-ja	2017/08/01 07:12:25	1684	72.250	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
4	NAIST-NICT	en-ja	2017/07/27 21:50:15	1507	70.000	NMT	No	Single Model: SPM16k/16k, BiLSTM Encoder 51222, UniLSTM Decoder 512*2, Adjusted Search
5	Kyoto-U	en-ja	2017/08/01 15:49:51	1731	69.750	NMT	No	Ensemble of 4 BPE Averaged
6	ORGANIZER	en-ja	2017/08/02 01:04:49	1737	69.750	NMT	No	Google's "Attention Is All You Need"
7	UT-IIS	en-ja	2017/08/01 12:03:22	1710	68.000	NMT	No	NMT 8 Ensembles with beam search, Sentence Piece, Embedding layer initialization
8	NAIST-NICT	en-ja	2017/07/27 21:49:04	1506	63.500	NMT	No	Single Model: SPM16k/16k, BiLSTM Encoder 51222, UniLSTM Decoder 512*2, One-best Search
9	NICT-2	en-ja	2017/07/26 13:52:35	1475	62.000	NMT	No	NMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
10	TMU	en-ja	2017/08/01 11:56:31	1709	56.500	NMT	No	baseline system with beam20
11	TMU	en-ja	2017/08/01 11:42:03	1704	50.750	NMT	No	our baseline system in 2017

Notice:

HUMAN (WAT2017) is the result of the Pairwise Crowdsourcing Evaluation on WAT2017.
HUMAN (WAT2017) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2016)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	ORGANIZER	en-ja	2016/11/16 10:50:13	1334	74.000	NMT	Yes	Online A (2016/11/14)
2	Kyoto-U	en-ja	2016/08/18 03:45:51	1172	55.250	NMT	No	BPE tgt/src: 52k 2-layer lstm self-ensemble of 3
3	ORGANIZER	en-ja	2016/07/26 11:31:47	1041	49.750	Other	Yes	Online A (2016)
4	NICT-2	en-ja	2016/08/05 17:47:05	1097	41.250	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
5	bjtu_nlp	en-ja	2016/08/16 11:21:07	1143	39.500	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
6	EHR	en-ja	2016/08/15 11:21:51	1140	39.000	SMT	No	PBSMT with preordering (DL=6)
7	UT-AKY	en-ja	2016/08/20 12:34:43	1228	36.250	NMT	No	tree-to-seq NMT model (word-based decoder)
8	TOKYOMT	en-ja	2016/08/19 23:18:46	1217	30.500	NMT	No	Combination of NMT and T2S
9	TOKYOMT	en-ja	2016/08/12 11:21:43	1131	29.750	NMT	No	char 1 , ens 2 , version 1
10	UT-AKY	en-ja	2016/08/20 09:25:15	1224	21.750	NMT	No	tree-to-seq NMT model (character-based decoder)
11	JAPIO	en-ja	2016/08/17 12:50:50	1165	4.250	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus + rule-based posteditor

Notice:

HUMAN (WAT2016) is the result of the Pairwise Crowdsourcing Evaluation on WAT2016.
HUMAN (WAT2016) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2015)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	NAIST	en-ja	2015/08/25 12:39:23	761	62.250	SMT	No	Travatar System with NeuralMT Reranking
2	W	en-ja	2015/08/26 16:00:46	786	53.750	SMT	No	NMT, LSTM Search, 5 ensembles, beam size 20, UNK replacing, System Combination with NMT score (Pick top-1k results from NMT)
3	naver	en-ja	2015/08/31 14:18:18	837	53.250	SMT	No	SMT t2s + Spell correction + NMT reranking
4	Kyoto-U	en-ja	2015/08/30 19:59:18	832	51.000	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking
5	NAIST	en-ja	2015/08/25 12:47:49	763	49.750	SMT	No	Travatar System Baseline
6	naver	en-ja	2015/08/31 14:12:03	836	48.500	SMT	No	NMT only
7	W	en-ja	2015/08/28 14:30:56	813	43.500	Other	No	NMT, LSTM Search, Beam Size 20, Ensemble of 2 models, UNK replacing
8	Kyoto-U	en-ja	2015/08/27 22:56:13	805	40.500	EBMT	No	KyotoEBMT system without reranking
9	TOSHIBA	en-ja	2015/07/28 16:24:37	524	40.250	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
10	ORGANIZER	en-ja	2015/08/25 18:54:29	774	34.250	Other	Yes	Online A (2015)
11	EHR	en-ja	2015/08/22 12:28:15	742	32.500	SMT	No	Phrase based SMT with preordering.
12	ORGANIZER	en-ja	2015/09/10 13:29:35	875	30.000	SMT	No	Tree-to-String SMT (2015)
13	ORGANIZER	en-ja	2015/09/10 14:26:28	883	9.750	Other	Yes	RBMT B (2015)
14	Sense	en-ja	2015/08/18 22:04:09	715	-31.000	SMT	Yes	Passive JSTx3
15	Sense	en-ja	2015/08/18 21:52:01	700	-36.250	SMT	No	Baseline-dictmt

Notice:

HUMAN (WAT2015) is the result of the Pairwise Crowdsourcing Evaluation on WAT2015.
HUMAN (WAT2015) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.
The detail of the evaluation can be found in the pdf document (PDF file).

HUMAN (WAT2014)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	NAIST	en-ja	2014/07/31 11:38:37	118	56.250	SMT	No	Travatar-based Forest-to-String SMT System
2	NAIST	en-ja	2014/08/01 17:37:23	126	51.500	SMT	No	Travatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
3	W	en-ja	2014/08/16 00:57:16	132	43.250	SMT	No	Weblio Pre-reordering SMT System Baseline
4	ORGANIZER	en-ja	2014/07/18 11:02:25	34	42.500	Other	Yes	Online A (2014)
5	Kyoto-U	en-ja	2014/09/01 21:06:32	267	38.000	EBMT	No	Our new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
6	W	en-ja	2014/08/26 16:17:15	202	36.000	SMT	No	Weblio Pre-reordering SMT System (with forest inputs)
7	ORGANIZER	en-ja	2014/07/11 20:03:21	12	34.250	SMT	No	Tree-to-String SMT (2014)
8	Kyoto-U	en-ja	2014/08/31 10:33:23	253	33.750	EBMT	No	Our new baseline system after several modifications.
9	ORGANIZER	en-ja	2014/09/16 13:36:35	367	31.500	SMT	No	Hierarchical Phrase-based SMT (2014)
10	SAS_MT	en-ja	2014/09/01 10:39:27	264	27.500	SMT	No	Syntactic reordering Hierarchical SMT （using part of data）
11	Sense	en-ja	2014/08/25 01:07:27	184	3.750	SMT	No	Baseline SMT
12	ORGANIZER	en-ja	2014/07/21 11:38:12	66	0.750	Other	Yes	RBMT B (2014)

Notice:

HUMAN (WAT2014) is the result of the Pairwise Crowdsourcing Evaluation on WAT2014.
HUMAN (WAT2014) was evaluated by 3 different workers and the final decision is made by the voting of the judgements.
The detail of the evaluation can be found in the pdf document (PDF file).

EVALUATION RESULTS USAGE POLICY

When you use the WAT evaluation results for any purpose such as:
- writing technical papers,
- making presentations about your system,
- advertising your MT system to the customers,
you can use the information about translation directions, scores (including both automatic and human evaluations) and ranks of your system among others. You can also use the scores of the other systems, but you MUST anonymize the other system's names. In addition, you can show the links (URLs) to the WAT evaluation result pages.

NICT (National Institute of Information and Communications Technology)
Kyoto University
Last Modified: 2018-08-02

WAT The Workshop on Asian Translation Evaluation Results

BLEU

RIBES

AMFM

HUMAN (WAT2022)

HUMAN (WAT2021)

HUMAN (WAT2020)

HUMAN (WAT2019)

HUMAN (WAT2018)

HUMAN (WAT2017)

HUMAN (WAT2016)

HUMAN (WAT2015)

HUMAN (WAT2014)

EVALUATION RESULTS USAGE POLICY

WAT

The Workshop on Asian Translation

Evaluation Results