WAT

The Workshop on Asian Translation

Evaluation Results

BLEU

#	Team	Task	Date/Time	DataID	BLEU										Method	Other Resources	System Description
#	Team	Task	Date/Time	DataID	juman	kytea	mecab	moses- tokenizer	stanford- segmenter- ctb	stanford- segmenter- pku	indic- tokenizer	unuse	myseg	kmseg	Method	Other Resources	System Description
1	srcb	en-ja	2019/07/25 11:43:06	2918	43.60	45.49	44.29	-	-	-	-	-	-	-	NMT	No	Transformer (Big) with relative position, sentence-wise smooth
2	KNU_Hyundai	en-ja	2019/07/27 09:19:31	3172	44.08	45.88	44.78	-	-	-	-	-	-	-	NMT	No	Transformer Base, relative position, BT, r2l reranking, ensemble of 3 models
3	NICT-2	en-ja	2019/07/27 10:56:02	3181	43.79	45.82	44.83	-	-	-	-	-	-	-	NMT	No	Transformer, sigle model w/ long warm-up and self-training
4	NICT-2	en-ja	2019/07/27 10:57:26	3182	44.61	46.59	45.66	-	-	-	-	-	-	-	NMT	No	Transformer, ensemble of 4 models w/ long warm-up and self-training
5	srcb	en-ja	2019/07/27 15:28:30	3206	45.30	47.12	45.93	-	-	-	-	-	-	-	NMT	No	Transformer(Big) with relative position, sentence-wise smooth, deep transformer, back translation, ensemble of 7 models, re-rank.
6	srcb	en-ja	2019/07/27 15:50:19	3212	45.71	47.55	46.29	-	-	-	-	-	-	-	NMT	No	Transformer(Big) with relative position, sentence-wise smooth, deep transformer, back translation, ensemble of 10 models, rerank.
7	NTT	en-ja	2019/07/28 15:44:12	3236	45.83	47.63	46.57	-	-	-	-	-	-	-	NMT	No	ASPEC first 1.5M + Synthetic 1.5M, 6 ensemble
8	NTT	en-ja	2019/07/28 15:55:18	3239	45.30	47.13	46.04	-	-	-	-	-	-	-	NMT	Yes	ParaCrawl + (ASPEC first 1.5M + Synthetic 1.5M) * 2 oversampling, fine-tune ASPEC, SINGLE MODEL
9	AISTAI	en-ja	2019/07/29 07:33:56	3251	42.64	44.17	43.34	-	-	-	-	-	-	-	NMT	No	Transformer (big). 1.5M sentences, train_steps=131000 only. Averaged the last 10 ckpts.
10	AISTAI	en-ja	2019/08/30 20:23:24	3357	43.76	45.70	44.52	-	-	-	-	-	-	-	NMT	No	Transformer, 1.5M sentences, relative position, ensemble of 3 models, by OpenNMT-py.
11	AISTAI	en-ja	2019/09/05 08:18:45	3373	42.92	44.90	43.67	-	-	-	-	-	-	-	NMT	No	Transformer (big), 1.5M sentences, train_steps=300000, Averaged the last 20 ckpts, by Tensor2Tensor.
12	ORGANIZER	en-ja	2014/07/11 19:49:03	5	27.48	29.80	28.27	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Phrase-based SMT
13	ORGANIZER	en-ja	2014/07/11 20:03:21	12	31.05	33.44	32.10	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Tree-to-String SMT (2014)
14	ORGANIZER	en-ja	2014/07/18 11:02:25	34	19.66	21.63	20.17	-	-	-	0.00	0.00	0.00	0.00	Other	Yes	Online A (2014)
15	ORGANIZER	en-ja	2014/07/21 11:38:12	66	13.18	14.85	13.48	-	-	-	0.00	0.00	0.00	0.00	Other	Yes	RBMT B (2014)
16	ORGANIZER	en-ja	2014/07/21 11:40:50	68	12.86	14.43	13.16	-	-	-	0.00	0.00	0.00	0.00	Other	Yes	RBMT A
17	ORGANIZER	en-ja	2014/07/22 13:30:13	91	17.04	18.67	17.36	-	-	-	0.00	0.00	0.00	0.00	Other	Yes	Online B (2014)
18	ORGANIZER	en-ja	2014/07/23 14:50:44	95	12.19	13.32	12.14	-	-	-	0.00	0.00	0.00	0.00	Other	Yes	RBMT C
19	NAIST	en-ja	2014/07/31 11:38:37	118	35.03	37.16	35.81	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Travatar-based Forest-to-String SMT System
20	NAIST	en-ja	2014/08/01 17:37:23	126	34.84	37.15	35.67	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Travatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
21	W	en-ja	2014/08/16 00:57:16	132	32.53	34.87	33.26	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Weblio Pre-reordering SMT System Baseline
22	Kyoto-U	en-ja	2014/08/19 10:16:02	134	28.93	31.61	29.59	-	-	-	0.00	0.00	0.00	0.00	EBMT	No	Our baseline system using 3M parallel sentences.
23	Sense	en-ja	2014/08/25 01:07:27	184	27.92	30.18	28.66	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Baseline SMT
24	Kyoto-U	en-ja	2014/08/25 14:16:15	186	30.25	32.78	30.84	-	-	-	0.00	0.00	0.00	0.00	EBMT	No	Using n-best parses and RNNLM.
25	W	en-ja	2014/08/26 16:17:15	202	32.69	35.04	33.40	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Weblio Pre-reordering SMT System (with forest inputs)
26	Kyoto-U	en-ja	2014/08/31 10:33:23	253	29.76	32.46	30.46	-	-	-	0.00	0.00	0.00	0.00	EBMT	No	Our new baseline system after several modifications.
27	SAS_MT	en-ja	2014/09/01 10:39:27	264	30.47	33.00	31.47	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Syntactic reordering Hierarchical SMT （using part of data）
28	Kyoto-U	en-ja	2014/09/01 21:06:32	267	31.09	33.55	31.73	-	-	-	0.00	0.00	0.00	0.00	EBMT	No	Our new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
29	ORGANIZER	en-ja	2014/09/16 13:36:35	367	30.19	32.56	30.94	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Hierarchical Phrase-based SMT (2014)
30	TOSHIBA	en-ja	2015/07/28 16:24:37	524	32.06	34.17	32.76	-	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
31	Sense	en-ja	2015/07/28 22:26:55	531	25.22	27.22	25.90	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Baseline-2015
32	TOSHIBA	en-ja	2015/07/30 11:22:11	540	31.82	33.94	32.54	-	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	System combination SMT and RBMT(SPE) with RNNLM language model
33	naver	en-ja	2015/08/04 16:48:32	581	32.63	34.46	33.34	-	-	-	0.00	0.00	0.00	0.00	SMT	No	SMT t2s
34	Sense	en-ja	2015/08/18 21:52:01	700	24.13	26.24	24.96	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Baseline-dictmt
35	Sense	en-ja	2015/08/18 22:04:09	715	24.43	26.58	25.36	-	-	-	0.00	0.00	0.00	0.00	SMT	Yes	Passive JSTx3
36	EHR	en-ja	2015/08/22 12:28:15	742	29.78	32.36	30.71	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Phrase based SMT with preordering.
37	NAIST	en-ja	2015/08/25 12:39:23	761	35.83	38.17	36.61	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Travatar System with NeuralMT Reranking
38	NAIST	en-ja	2015/08/25 12:47:49	763	34.38	36.58	35.16	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Travatar System Baseline
39	naver	en-ja	2015/08/25 16:20:30	770	32.76	34.53	33.47	-	-	-	0.00	0.00	0.00	0.00	SMT	No	SMT t2s + Spell correction
40	ORGANIZER	en-ja	2015/08/25 18:54:29	774	18.22	19.77	18.46	-	-	-	0.00	0.00	0.00	0.00	Other	Yes	Online A (2015)
41	W	en-ja	2015/08/26 16:00:46	786	33.23	36.21	34.05	-	-	-	0.00	0.00	0.00	0.00	SMT	No	NMT, LSTM Search, 5 ensembles, beam size 20, UNK replacing, System Combination with NMT score (Pick top-1k results from NMT)
42	Kyoto-U	en-ja	2015/08/27 22:56:13	805	30.69	33.25	31.71	-	-	-	0.00	0.00	0.00	0.00	EBMT	No	KyotoEBMT system without reranking
43	W	en-ja	2015/08/28 14:30:56	813	30.72	34.19	31.57	-	-	-	0.00	0.00	0.00	0.00	Other	No	NMT, LSTM Search, Beam Size 20, Ensemble of 2 models, UNK replacing
44	Sense	en-ja	2015/08/28 19:22:24	821	27.91	30.03	28.78	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Baseline-2015 (train1 only)
45	Kyoto-U	en-ja	2015/08/30 19:59:18	832	33.06	35.57	33.99	-	-	-	0.00	0.00	0.00	0.00	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking
46	naver	en-ja	2015/08/31 14:12:03	836	33.14	35.75	33.93	-	-	-	0.00	0.00	0.00	0.00	SMT	No	NMT only
47	naver	en-ja	2015/08/31 14:18:18	837	34.60	36.14	35.30	-	-	-	0.00	0.00	0.00	0.00	SMT	No	SMT t2s + Spell correction + NMT reranking
48	EHR	en-ja	2015/09/09 12:21:34	873	30.88	33.20	31.74	-	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	System combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
49	EHR	en-ja	2015/09/09 12:22:58	874	30.46	32.56	31.28	-	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	RBMT with user dictionary plus SPE.
50	ORGANIZER	en-ja	2015/09/10 13:29:35	875	31.05	33.44	32.10	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Tree-to-String SMT (2015)
51	ORGANIZER	en-ja	2015/09/10 14:26:28	883	13.18	14.85	13.48	-	-	-	0.00	0.00	0.00	0.00	Other	Yes	RBMT B (2015)
52	ORGANIZER	en-ja	2015/09/10 19:02:38	889	17.80	19.52	18.11	-	-	-	0.00	0.00	0.00	0.00	Other	Yes	Online B (2015)
53	ORGANIZER	en-ja	2016/07/26 11:31:47	1041	18.28	19.81	18.51	-	-	-	-	0.00	0.00	0.00	Other	Yes	Online A (2016)
54	NICT-2	en-ja	2016/08/05 17:47:05	1097	34.67	36.86	35.37	-	-	-	-	0.00	0.00	0.00	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
55	TOKYOMT	en-ja	2016/08/12 11:21:43	1131	30.21	33.38	31.24	-	-	-	-	0.00	0.00	0.00	NMT	No	char 1 , ens 2 , version 1
56	EHR	en-ja	2016/08/15 11:21:51	1140	31.32	33.58	32.28	-	-	-	-	0.00	0.00	0.00	SMT	No	PBSMT with preordering (DL=6)
57	bjtu_nlp	en-ja	2016/08/16 11:21:07	1143	31.18	33.47	31.80	-	-	-	-	0.00	0.00	0.00	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
58	JAPIO	en-ja	2016/08/17 12:50:50	1165	20.52	22.56	21.05	-	-	-	-	0.00	0.00	0.00	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus + rule-based posteditor
59	Kyoto-U	en-ja	2016/08/18 03:45:51	1172	36.19	38.20	36.78	-	-	-	-	0.00	0.00	0.00	NMT	No	BPE tgt/src: 52k 2-layer lstm self-ensemble of 3
60	Kyoto-U	en-ja	2016/08/19 10:18:09	1201	31.03	33.40	31.88	-	-	-	-	0.00	0.00	0.00	EBMT	No	KyotoEBMT 2016 w/o reranking
61	TOKYOMT	en-ja	2016/08/19 23:18:46	1217	32.03	34.77	32.98	-	-	-	-	0.00	0.00	0.00	NMT	No	Combination of NMT and T2S
62	UT-AKY	en-ja	2016/08/20 09:25:15	1224	30.14	33.20	31.09	-	-	-	-	0.00	0.00	0.00	NMT	No	tree-to-seq NMT model (character-based decoder)
63	UT-AKY	en-ja	2016/08/20 12:34:43	1228	33.57	36.95	34.65	-	-	-	-	0.00	0.00	0.00	NMT	No	tree-to-seq NMT model (word-based decoder)
64	ORGANIZER	en-ja	2016/11/16 10:50:13	1334	26.19	28.22	26.68	-	-	-	-	0.00	0.00	0.00	NMT	Yes	Online A (2016/11/14)
65	UT-KAY	en-ja	2017/06/27 16:58:42	1369	38.32	41.12	39.32	-	-	-	-	0.00	0.00	0.00	NMT	No	Hashimoto and Tsuruoka (2017), https://arxiv.org/abs/1702.02265
66	NICT-2	en-ja	2017/07/26 13:52:35	1475	36.85	38.94	37.87	-	-	-	-	0.00	0.00	0.00	NMT	No	NMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
67	NICT-2	en-ja	2017/07/26 14:03:00	1479	40.17	42.25	41.17	-	-	-	-	0.00	0.00	0.00	NMT	No	NMT 6 Ensembles * Bi-directional Reranking
68	NAIST-NICT	en-ja	2017/07/27 21:49:04	1506	36.47	38.54	37.30	-	-	-	-	0.00	0.00	0.00	NMT	No	Single Model: SPM16k/16k, BiLSTM Encoder 51222, UniLSTM Decoder 512*2, One-best Search
69	NAIST-NICT	en-ja	2017/07/27 21:50:15	1507	38.25	40.29	39.05	-	-	-	-	0.00	0.00	0.00	NMT	No	Single Model: SPM16k/16k, BiLSTM Encoder 51222, UniLSTM Decoder 512*2, Adjusted Search
70	NTT	en-ja	2017/07/30 16:16:25	1608	37.90	40.48	38.61	-	-	-	-	0.00	0.00	0.00	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
71	NTT	en-ja	2017/08/01 02:22:56	1673	38.87	41.42	39.71	-	-	-	-	0.00	0.00	0.00	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
72	NTT	en-ja	2017/08/01 07:12:25	1684	39.80	42.27	40.47	-	-	-	-	0.00	0.00	0.00	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
73	TMU	en-ja	2017/08/01 11:42:03	1704	32.65	35.05	33.72	-	-	-	-	0.00	0.00	0.00	NMT	No	our baseline system in 2017
74	TMU	en-ja	2017/08/01 11:56:31	1709	34.05	36.69	35.32	-	-	-	-	0.00	0.00	0.00	NMT	No	baseline system with beam20
75	UT-IIS	en-ja	2017/08/01 12:03:22	1710	36.26	38.93	37.06	-	-	-	-	0.00	0.00	0.00	NMT	No	NMT 8 Ensembles with beam search, Sentence Piece, Embedding layer initialization
76	NTT	en-ja	2017/08/01 15:42:13	1729	40.32	42.80	40.95	-	-	-	-	0.00	0.00	0.00	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
77	Kyoto-U	en-ja	2017/08/01 15:49:51	1731	38.72	40.65	39.37	-	-	-	-	0.00	0.00	0.00	NMT	No	Ensemble of 4 BPE Averaged
78	ORGANIZER	en-ja	2017/08/02 01:04:49	1737	40.79	42.55	41.50	-	-	-	-	0.00	0.00	0.00	NMT	No	Google's "Attention Is All You Need"
79	TMU	en-ja	2017/08/02 10:20:14	1741	34.58	36.95	35.63	-	-	-	-	0.00	0.00	0.00	NMT	No	the ensemble system of different dropout rate.
80	TMU	en-ja	2017/08/04 11:06:51	1747	35.73	38.34	37.00	-	-	-	-	0.00	0.00	0.00	NMT	No	beam_size: 10, ensemble of different dropout rates.
81	Kyoto-U	en-ja	2017/09/04 22:31:18	1759	41.53	43.26	42.13	-	-	-	-	0.00	0.00	0.00	SMT	No	Syscomb of AIAYN and KNMT
82	ORGANIZER	en-ja	2018/08/14 10:58:49	1900	36.37	38.48	37.15	-	-	-	-	-	0.00	0.00	NMT	No	NMT with Attention
83	NICT-5	en-ja	2018/08/22 18:43:00	2048	41.91	43.50	42.60	-	-	-	-	-	0.00	0.00	NMT	No	Tensor2Tensor's transformer implementation. Used recurrently stacked layers model logic. Layer recurrence logic: 1-2-3-1-2-3-1-2-3-1-2-3. Codename: Megarecurrence.
84	NICT-5	en-ja	2018/08/22 18:48:51	2050	40.02	41.74	40.77	-	-	-	-	-	0.00	0.00	NMT	No	RSNMT 6 layer
85	Sense	en-ja	2018/08/24 11:38:23	2092	35.29	37.21	36.02	-	-	-	-	-	0.00	0.00	NMT	No
86	NICT-5	en-ja	2018/09/03 16:49:57	2219	42.87	44.42	43.49	-	-	-	-	-	0.00	0.00	NMT	No	Big Bidirectional Transformer. 1.5M sentences only.
87	EHR	en-ja	2018/09/08 12:42:03	2245	37.97	40.00	38.66	-	-	-	-	-	0.00	0.00	NMT	No	SMT reranked NMT
88	Osaka-U	en-ja	2018/09/15 23:02:05	2439	38.01	40.00	39.10	-	-	-	-	-	0.00	0.00	NMT	Yes	rewarding model
89	TMU	en-ja	2018/09/16 12:10:49	2467	34.02	36.69	35.17	-	-	-	-	-	0.00	0.00	NMT	No	Baseline-NMT ( Single )
90	TMU	en-ja	2018/09/16 12:11:58	2468	34.00	36.65	35.09	-	-	-	-	-	0.00	0.00	NMT	No	GAN-NMT ( Single )
91	TMU	en-ja	2018/09/16 12:44:23	2469	35.08	37.69	36.14	-	-	-	-	-	0.00	0.00	NMT	No	Ensemble of 6 Baseline-NMT
92	Osaka-U	en-ja	2018/09/16 13:06:45	2470	23.24	25.50	24.26	-	-	-	-	-	0.00	0.00	SMT	No	preordering with neural network
93	srcb	en-ja	2018/09/16 15:26:37	2479	42.49	44.11	43.20	-	-	-	-	-	0.00	0.00	NMT	No	Transformer with relative position, ensemble of 3 models.
94	srcb	en-ja	2018/09/16 15:52:17	2480	43.43	44.94	44.05	-	-	-	-	-	0.00	0.00	NMT	No	Transformer with relative position, ensemble of 4 models, rerank.
95	TMU	en-ja	2018/09/16 16:56:49	2482	33.86	36.58	34.89	-	-	-	-	-	0.00	0.00	NMT	No	Reconstructor-NMT ( Single )
96	TMU	en-ja	2018/09/16 17:25:36	2484	34.38	37.06	35.44	-	-	-	-	-	0.00	0.00	NMT	No	Ensemble of 6 NMT ( 2 Baseline + 2 Reconstructor + 2 GAN )

Notice:

This table is sorted by the leftmost segmenters. You can change the segmenter used to sort by clicking each segmenter link.

RIBES

#	Team	Task	Date/Time	DataID	RIBES										Method	Other Resources	System Description
#	Team	Task	Date/Time	DataID	juman	kytea	mecab	moses- tokenizer	stanford- segmenter- ctb	stanford- segmenter- pku	indic- tokenizer	unuse	myseg	kmseg	Method	Other Resources	System Description
1	srcb	en-ja	2019/07/25 11:43:06	2918	0.848929	0.852948	0.855857	-	-	-	-	-	-	-	NMT	No	Transformer (Big) with relative position, sentence-wise smooth
2	KNU_Hyundai	en-ja	2019/07/27 09:19:31	3172	0.857060	0.859885	0.863400	-	-	-	-	-	-	-	NMT	No	Transformer Base, relative position, BT, r2l reranking, ensemble of 3 models
3	NICT-2	en-ja	2019/07/27 10:56:02	3181	0.850458	0.853909	0.857275	-	-	-	-	-	-	-	NMT	No	Transformer, sigle model w/ long warm-up and self-training
4	NICT-2	en-ja	2019/07/27 10:57:26	3182	0.852970	0.856755	0.859059	-	-	-	-	-	-	-	NMT	No	Transformer, ensemble of 4 models w/ long warm-up and self-training
5	srcb	en-ja	2019/07/27 15:28:30	3206	0.856245	0.859025	0.863177	-	-	-	-	-	-	-	NMT	No	Transformer(Big) with relative position, sentence-wise smooth, deep transformer, back translation, ensemble of 7 models, re-rank.
6	srcb	en-ja	2019/07/27 15:50:19	3212	0.857506	0.860642	0.862819	-	-	-	-	-	-	-	NMT	No	Transformer(Big) with relative position, sentence-wise smooth, deep transformer, back translation, ensemble of 10 models, rerank.
7	NTT	en-ja	2019/07/28 15:44:12	3236	0.861994	0.865640	0.868175	-	-	-	-	-	-	-	NMT	No	ASPEC first 1.5M + Synthetic 1.5M, 6 ensemble
8	NTT	en-ja	2019/07/28 15:55:18	3239	0.859673	0.863532	0.866183	-	-	-	-	-	-	-	NMT	Yes	ParaCrawl + (ASPEC first 1.5M + Synthetic 1.5M) * 2 oversampling, fine-tune ASPEC, SINGLE MODEL
9	AISTAI	en-ja	2019/07/29 07:33:56	3251	0.849129	0.851400	0.856177	-	-	-	-	-	-	-	NMT	No	Transformer (big). 1.5M sentences, train_steps=131000 only. Averaged the last 10 ckpts.
10	AISTAI	en-ja	2019/08/30 20:23:24	3357	0.856082	0.859166	0.862434	-	-	-	-	-	-	-	NMT	No	Transformer, 1.5M sentences, relative position, ensemble of 3 models, by OpenNMT-py.
11	AISTAI	en-ja	2019/09/05 08:18:45	3373	0.847863	0.851673	0.854716	-	-	-	-	-	-	-	NMT	No	Transformer (big), 1.5M sentences, train_steps=300000, Averaged the last 20 ckpts, by Tensor2Tensor.
12	ORGANIZER	en-ja	2014/07/11 19:49:03	5	0.683735	0.691926	0.695390	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Phrase-based SMT
13	ORGANIZER	en-ja	2014/07/11 20:03:21	12	0.748883	0.758031	0.760516	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Tree-to-String SMT (2014)
14	ORGANIZER	en-ja	2014/07/18 11:02:25	34	0.718019	0.723486	0.725848	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online A (2014)
15	ORGANIZER	en-ja	2014/07/21 11:38:12	66	0.671958	0.680748	0.682683	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT B (2014)
16	ORGANIZER	en-ja	2014/07/21 11:40:50	68	0.670167	0.676464	0.678934	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT A
17	ORGANIZER	en-ja	2014/07/22 13:30:13	91	0.687797	0.693390	0.698126	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online B (2014)
18	ORGANIZER	en-ja	2014/07/23 14:50:44	95	0.668372	0.672645	0.676018	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT C
19	NAIST	en-ja	2014/07/31 11:38:37	118	0.796079	0.801520	0.806581	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar-based Forest-to-String SMT System
20	NAIST	en-ja	2014/08/01 17:37:23	126	0.801742	0.807010	0.811081	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
21	W	en-ja	2014/08/16 00:57:16	132	0.782066	0.786902	0.792616	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Weblio Pre-reordering SMT System Baseline
22	Kyoto-U	en-ja	2014/08/19 10:16:02	134	0.743969	0.755744	0.756545	-	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our baseline system using 3M parallel sentences.
23	Sense	en-ja	2014/08/25 01:07:27	184	0.690464	0.700583	0.703049	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline SMT
24	Kyoto-U	en-ja	2014/08/25 14:16:15	186	0.755629	0.765251	0.766495	-	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	Using n-best parses and RNNLM.
25	W	en-ja	2014/08/26 16:17:15	202	0.785015	0.790066	0.795027	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Weblio Pre-reordering SMT System (with forest inputs)
26	Kyoto-U	en-ja	2014/08/31 10:33:23	253	0.752058	0.764049	0.766435	-	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our new baseline system after several modifications.
27	SAS_MT	en-ja	2014/09/01 10:39:27	264	0.759415	0.770948	0.771605	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Syntactic reordering Hierarchical SMT （using part of data）
28	Kyoto-U	en-ja	2014/09/01 21:06:32	267	0.759608	0.770908	0.771545	-	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
29	ORGANIZER	en-ja	2014/09/16 13:36:35	367	0.734705	0.746978	0.747722	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Hierarchical Phrase-based SMT (2014)
30	TOSHIBA	en-ja	2015/07/28 16:24:37	524	0.770989	0.778570	0.780467	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
31	Sense	en-ja	2015/07/28 22:26:55	531	0.646288	0.653814	0.659505	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-2015
32	TOSHIBA	en-ja	2015/07/30 11:22:11	540	0.769865	0.776841	0.779574	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	System combination SMT and RBMT(SPE) with RNNLM language model
33	naver	en-ja	2015/08/04 16:48:32	581	0.783310	0.788165	0.793723	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	SMT t2s
34	Sense	en-ja	2015/08/18 21:52:01	700	0.637378	0.642789	0.647831	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-dictmt
35	Sense	en-ja	2015/08/18 22:04:09	715	0.635933	0.641517	0.646682	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Passive JSTx3
36	EHR	en-ja	2015/08/22 12:28:15	742	0.753576	0.766044	0.768105	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Phrase based SMT with preordering.
37	NAIST	en-ja	2015/08/25 12:39:23	761	0.811479	0.813827	0.820337	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System with NeuralMT Reranking
38	NAIST	en-ja	2015/08/25 12:47:49	763	0.792447	0.796489	0.802228	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System Baseline
39	naver	en-ja	2015/08/25 16:20:30	770	0.784174	0.788811	0.794452	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	SMT t2s + Spell correction
40	ORGANIZER	en-ja	2015/08/25 18:54:29	774	0.705882	0.713960	0.718150	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online A (2015)
41	W	en-ja	2015/08/26 16:00:46	786	0.804722	0.809065	0.814337	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	NMT, LSTM Search, 5 ensembles, beam size 20, UNK replacing, System Combination with NMT score (Pick top-1k results from NMT)
42	Kyoto-U	en-ja	2015/08/27 22:56:13	805	0.767778	0.776672	0.778358	-	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT system without reranking
43	W	en-ja	2015/08/28 14:30:56	813	0.796863	0.802666	0.807186	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	No	NMT, LSTM Search, Beam Size 20, Ensemble of 2 models, UNK replacing
44	Sense	en-ja	2015/08/28 19:22:24	821	0.689313	0.697817	0.700994	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-2015 (train1 only)
45	Kyoto-U	en-ja	2015/08/30 19:59:18	832	0.789514	0.797182	0.799979	-	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking
46	naver	en-ja	2015/08/31 14:12:03	836	0.807280	0.811487	0.817343	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	NMT only
47	naver	en-ja	2015/08/31 14:18:18	837	0.799966	0.803154	0.808787	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	SMT t2s + Spell correction + NMT reranking
48	EHR	en-ja	2015/09/09 12:21:34	873	0.765700	0.774819	0.778269	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	System combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
49	EHR	en-ja	2015/09/09 12:22:58	874	0.768563	0.776526	0.778113	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with user dictionary plus SPE.
50	ORGANIZER	en-ja	2015/09/10 13:29:35	875	0.748883	0.758031	0.760516	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Tree-to-String SMT (2015)
51	ORGANIZER	en-ja	2015/09/10 14:26:28	883	0.671958	0.680748	0.682683	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT B (2015)
52	ORGANIZER	en-ja	2015/09/10 19:02:38	889	0.693359	0.701966	0.703859	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online B (2015)
53	ORGANIZER	en-ja	2016/07/26 11:31:47	1041	0.706639	0.715222	0.718559	-	-	-	-	0.000000	0.000000	0.000000	Other	Yes	Online A (2016)
54	NICT-2	en-ja	2016/08/05 17:47:05	1097	0.784335	0.790993	0.793409	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
55	TOKYOMT	en-ja	2016/08/12 11:21:43	1131	0.809691	0.817258	0.819951	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	char 1 , ens 2 , version 1
56	EHR	en-ja	2016/08/15 11:21:51	1140	0.759914	0.771427	0.775023	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	PBSMT with preordering (DL=6)
57	bjtu_nlp	en-ja	2016/08/16 11:21:07	1143	0.780510	0.787497	0.791088	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
58	JAPIO	en-ja	2016/08/17 12:50:50	1165	0.723467	0.728584	0.731474	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus + rule-based posteditor
59	Kyoto-U	en-ja	2016/08/18 03:45:51	1172	0.819836	0.823878	0.828956	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	BPE tgt/src: 52k 2-layer lstm self-ensemble of 3
60	Kyoto-U	en-ja	2016/08/19 10:18:09	1201	0.771240	0.779774	0.781960	-	-	-	-	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT 2016 w/o reranking
61	TOKYOMT	en-ja	2016/08/19 23:18:46	1217	0.808189	0.814452	0.818130	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Combination of NMT and T2S
62	UT-AKY	en-ja	2016/08/20 09:25:15	1224	0.806025	0.814490	0.815836	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	tree-to-seq NMT model (character-based decoder)
63	UT-AKY	en-ja	2016/08/20 12:34:43	1228	0.816984	0.824456	0.827647	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	tree-to-seq NMT model (word-based decoder)
64	ORGANIZER	en-ja	2016/11/16 10:50:13	1334	0.776787	0.780217	0.782674	-	-	-	-	0.000000	0.000000	0.000000	NMT	Yes	Online A (2016/11/14)
65	UT-KAY	en-ja	2017/06/27 16:58:42	1369	0.829600	0.833920	0.838260	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Hashimoto and Tsuruoka (2017), https://arxiv.org/abs/1702.02265
66	NICT-2	en-ja	2017/07/26 13:52:35	1475	0.826791	0.834448	0.835255	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
67	NICT-2	en-ja	2017/07/26 14:03:00	1479	0.842206	0.848170	0.849929	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT 6 Ensembles * Bi-directional Reranking
68	NAIST-NICT	en-ja	2017/07/27 21:49:04	1506	0.821989	0.827225	0.830116	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: SPM16k/16k, BiLSTM Encoder 51222, UniLSTM Decoder 512*2, One-best Search
69	NAIST-NICT	en-ja	2017/07/27 21:50:15	1507	0.834492	0.839321	0.842337	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: SPM16k/16k, BiLSTM Encoder 51222, UniLSTM Decoder 512*2, Adjusted Search
70	NTT	en-ja	2017/07/30 16:16:25	1608	0.828193	0.832006	0.836381	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
71	NTT	en-ja	2017/08/01 02:22:56	1673	0.833541	0.838245	0.841739	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
72	NTT	en-ja	2017/08/01 07:12:25	1684	0.835806	0.839981	0.844326	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
73	TMU	en-ja	2017/08/01 11:42:03	1704	0.802262	0.809649	0.811057	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	our baseline system in 2017
74	TMU	en-ja	2017/08/01 11:56:31	1709	0.812926	0.818443	0.821563	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	baseline system with beam20
75	UT-IIS	en-ja	2017/08/01 12:03:22	1710	0.827891	0.832054	0.836394	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT 8 Ensembles with beam search, Sentence Piece, Embedding layer initialization
76	NTT	en-ja	2017/08/01 15:42:13	1729	0.838594	0.841769	0.846486	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
77	Kyoto-U	en-ja	2017/08/01 15:49:51	1731	0.832472	0.835870	0.839646	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble of 4 BPE Averaged
78	ORGANIZER	en-ja	2017/08/02 01:04:49	1737	0.844896	0.847559	0.851471	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Google's "Attention Is All You Need"
79	TMU	en-ja	2017/08/02 10:20:14	1741	0.817280	0.824332	0.825683	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	the ensemble system of different dropout rate.
80	TMU	en-ja	2017/08/04 11:06:51	1747	0.824801	0.829580	0.832569	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	beam_size: 10, ensemble of different dropout rates.
81	Kyoto-U	en-ja	2017/09/04 22:31:18	1759	0.841000	0.843725	0.847819	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Syscomb of AIAYN and KNMT
82	ORGANIZER	en-ja	2018/08/14 10:58:49	1900	0.824985	0.831183	0.833207	-	-	-	-	-	0.000000	0.000000	NMT	No	NMT with Attention
83	NICT-5	en-ja	2018/08/22 18:43:00	2048	0.840776	0.845042	0.849326	-	-	-	-	-	0.000000	0.000000	NMT	No	Tensor2Tensor's transformer implementation. Used recurrently stacked layers model logic. Layer recurrence logic: 1-2-3-1-2-3-1-2-3-1-2-3. Codename: Megarecurrence.
84	NICT-5	en-ja	2018/08/22 18:48:51	2050	0.841011	0.844360	0.848283	-	-	-	-	-	0.000000	0.000000	NMT	No	RSNMT 6 layer
85	Sense	en-ja	2018/08/24 11:38:23	2092	0.796214	0.800727	0.803453	-	-	-	-	-	0.000000	0.000000	NMT	No
86	NICT-5	en-ja	2018/09/03 16:49:57	2219	0.847134	0.849399	0.853634	-	-	-	-	-	0.000000	0.000000	NMT	No	Big Bidirectional Transformer. 1.5M sentences only.
87	EHR	en-ja	2018/09/08 12:42:03	2245	0.828746	0.833333	0.837806	-	-	-	-	-	0.000000	0.000000	NMT	No	SMT reranked NMT
88	Osaka-U	en-ja	2018/09/15 23:02:05	2439	0.825061	0.829328	0.833200	-	-	-	-	-	0.000000	0.000000	NMT	Yes	rewarding model
89	TMU	en-ja	2018/09/16 12:10:49	2467	0.817948	0.824239	0.827386	-	-	-	-	-	0.000000	0.000000	NMT	No	Baseline-NMT ( Single )
90	TMU	en-ja	2018/09/16 12:11:58	2468	0.819148	0.825367	0.827650	-	-	-	-	-	0.000000	0.000000	NMT	No	GAN-NMT ( Single )
91	TMU	en-ja	2018/09/16 12:44:23	2469	0.823653	0.829156	0.831219	-	-	-	-	-	0.000000	0.000000	NMT	No	Ensemble of 6 Baseline-NMT
92	Osaka-U	en-ja	2018/09/16 13:06:45	2470	0.716889	0.726469	0.729323	-	-	-	-	-	0.000000	0.000000	SMT	No	preordering with neural network
93	srcb	en-ja	2018/09/16 15:26:37	2479	0.850318	0.852209	0.857017	-	-	-	-	-	0.000000	0.000000	NMT	No	Transformer with relative position, ensemble of 3 models.
94	srcb	en-ja	2018/09/16 15:52:17	2480	0.847678	0.850180	0.855073	-	-	-	-	-	0.000000	0.000000	NMT	No	Transformer with relative position, ensemble of 4 models, rerank.
95	TMU	en-ja	2018/09/16 16:56:49	2482	0.817153	0.824066	0.826013	-	-	-	-	-	0.000000	0.000000	NMT	No	Reconstructor-NMT ( Single )
96	TMU	en-ja	2018/09/16 17:25:36	2484	0.820961	0.826779	0.829178	-	-	-	-	-	0.000000	0.000000	NMT	No	Ensemble of 6 NMT ( 2 Baseline + 2 Reconstructor + 2 GAN )

Notice:

This table is sorted by the leftmost segmenters. You can change the segmenter used to sort by clicking each segmenter link.

AMFM

#	Team	Task	Date/Time	DataID	AMFM										Method	Other Resources	System Description
#	Team	Task	Date/Time	DataID	unuse	unuse	unuse	unuse	unuse	unuse	unuse	unuse	unuse	unuse	Method	Other Resources	System Description
1	srcb	en-ja	2019/07/25 11:43:06	2918	0.759510	0.759510	0.759510	-	-	-	-	-	-	-	NMT	No	Transformer (Big) with relative position, sentence-wise smooth
2	KNU_Hyundai	en-ja	2019/07/27 09:19:31	3172	0.760050	0.760050	0.760050	-	-	-	-	-	-	-	NMT	No	Transformer Base, relative position, BT, r2l reranking, ensemble of 3 models
3	NICT-2	en-ja	2019/07/27 10:56:02	3181	0.759260	0.759260	0.759260	-	-	-	-	-	-	-	NMT	No	Transformer, sigle model w/ long warm-up and self-training
4	NICT-2	en-ja	2019/07/27 10:57:26	3182	0.761340	0.761340	0.761340	-	-	-	-	-	-	-	NMT	No	Transformer, ensemble of 4 models w/ long warm-up and self-training
5	srcb	en-ja	2019/07/27 15:28:30	3206	0.765330	0.765330	0.765330	-	-	-	-	-	-	-	NMT	No	Transformer(Big) with relative position, sentence-wise smooth, deep transformer, back translation, ensemble of 7 models, re-rank.
6	srcb	en-ja	2019/07/27 15:50:19	3212	0.770440	0.770440	0.770440	-	-	-	-	-	-	-	NMT	No	Transformer(Big) with relative position, sentence-wise smooth, deep transformer, back translation, ensemble of 10 models, rerank.
7	NTT	en-ja	2019/07/28 15:44:12	3236	0.774950	0.774950	0.774950	-	-	-	-	-	-	-	NMT	No	ASPEC first 1.5M + Synthetic 1.5M, 6 ensemble
8	NTT	en-ja	2019/07/28 15:55:18	3239	0.772590	0.772590	0.772590	-	-	-	-	-	-	-	NMT	Yes	ParaCrawl + (ASPEC first 1.5M + Synthetic 1.5M) * 2 oversampling, fine-tune ASPEC, SINGLE MODEL
9	AISTAI	en-ja	2019/07/29 07:33:56	3251	0.757590	0.757590	0.757590	-	-	-	-	-	-	-	NMT	No	Transformer (big). 1.5M sentences, train_steps=131000 only. Averaged the last 10 ckpts.
10	AISTAI	en-ja	2019/08/30 20:23:24	3357	0.760410	0.760410	0.760410	-	-	-	-	-	-	-	NMT	No	Transformer, 1.5M sentences, relative position, ensemble of 3 models, by OpenNMT-py.
11	AISTAI	en-ja	2019/09/05 08:18:45	3373	0.759940	0.759940	0.759940	-	-	-	-	-	-	-	NMT	No	Transformer (big), 1.5M sentences, train_steps=300000, Averaged the last 20 ckpts, by Tensor2Tensor.
12	ORGANIZER	en-ja	2014/07/11 19:49:03	5	0.736380	0.736380	0.736380	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Phrase-based SMT
13	ORGANIZER	en-ja	2014/07/11 20:03:21	12	0.744370	0.744370	0.744370	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Tree-to-String SMT (2014)
14	ORGANIZER	en-ja	2014/07/18 11:02:25	34	0.695420	0.695420	0.695420	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online A (2014)
15	ORGANIZER	en-ja	2014/07/21 11:38:12	66	0.622930	0.622930	0.622930	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT B (2014)
16	ORGANIZER	en-ja	2014/07/21 11:40:50	68	0.626940	0.626940	0.626940	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT A
17	ORGANIZER	en-ja	2014/07/22 13:30:13	91	0.643070	0.643070	0.643070	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online B (2014)
18	ORGANIZER	en-ja	2014/07/23 14:50:44	95	0.594380	0.594380	0.594380	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT C
19	NAIST	en-ja	2014/07/31 11:38:37	118	0.752850	0.752850	0.752850	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar-based Forest-to-String SMT System
20	NAIST	en-ja	2014/08/01 17:37:23	126	0.758740	0.758740	0.758740	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
21	W	en-ja	2014/08/16 00:57:16	132	0.746570	0.746570	0.746570	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Weblio Pre-reordering SMT System Baseline
22	Kyoto-U	en-ja	2014/08/19 10:16:02	134	0.737890	0.737890	0.737890	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our baseline system using 3M parallel sentences.
23	Sense	en-ja	2014/08/25 01:07:27	184	0.733360	0.733360	0.733360	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline SMT
24	Kyoto-U	en-ja	2014/08/25 14:16:15	186	0.732380	0.732380	0.732380	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	Using n-best parses and RNNLM.
25	W	en-ja	2014/08/26 16:17:15	202	0.747650	0.747650	0.747650	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Weblio Pre-reordering SMT System (with forest inputs)
26	Kyoto-U	en-ja	2014/08/31 10:33:23	253	0.741710	0.741710	0.741710	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our new baseline system after several modifications.
27	SAS_MT	en-ja	2014/09/01 10:39:27	264	0.740580	0.740580	0.740580	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Syntactic reordering Hierarchical SMT （using part of data）
28	Kyoto-U	en-ja	2014/09/01 21:06:32	267	0.738680	0.738680	0.738680	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	Our new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
29	ORGANIZER	en-ja	2014/09/16 13:36:35	367	0.743900	0.743900	0.743900	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Hierarchical Phrase-based SMT (2014)
30	TOSHIBA	en-ja	2015/07/28 16:24:37	524	0.742100	0.742100	0.742100	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
31	Sense	en-ja	2015/07/28 22:26:55	531	0.731620	0.731620	0.731620	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-2015
32	TOSHIBA	en-ja	2015/07/30 11:22:11	540	0.744660	0.744660	0.744660	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	System combination SMT and RBMT(SPE) with RNNLM language model
33	naver	en-ja	2015/08/04 16:48:32	581	0.737590	0.737590	0.737590	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	SMT t2s
34	Sense	en-ja	2015/08/18 21:52:01	700	0.729200	0.729200	0.729200	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-dictmt
35	Sense	en-ja	2015/08/18 22:04:09	715	0.731400	0.731400	0.731400	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Passive JSTx3
36	EHR	en-ja	2015/08/22 12:28:15	742	0.620500	0.620500	0.620500	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Phrase based SMT with preordering.
37	NAIST	en-ja	2015/08/25 12:39:23	761	0.748870	0.748870	0.748870	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System with NeuralMT Reranking
38	NAIST	en-ja	2015/08/25 12:47:49	763	0.752260	0.752260	0.752260	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Travatar System Baseline
39	naver	en-ja	2015/08/25 16:20:30	770	0.740210	0.740210	0.740210	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	SMT t2s + Spell correction
40	ORGANIZER	en-ja	2015/08/25 18:54:29	774	0.677200	0.677200	0.677200	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online A (2015)
41	W	en-ja	2015/08/26 16:00:46	786	0.725370	0.725370	0.725370	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	NMT, LSTM Search, 5 ensembles, beam size 20, UNK replacing, System Combination with NMT score (Pick top-1k results from NMT)
42	Kyoto-U	en-ja	2015/08/27 22:56:13	805	0.741050	0.741050	0.741050	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT system without reranking
43	W	en-ja	2015/08/28 14:30:56	813	0.705340	0.705340	0.705340	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	No	NMT, LSTM Search, Beam Size 20, Ensemble of 2 models, UNK replacing
44	Sense	en-ja	2015/08/28 19:22:24	821	0.737560	0.737560	0.737560	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Baseline-2015 (train1 only)
45	Kyoto-U	en-ja	2015/08/30 19:59:18	832	0.743710	0.743710	0.743710	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking
46	naver	en-ja	2015/08/31 14:12:03	836	0.725860	0.725860	0.725860	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	NMT only
47	naver	en-ja	2015/08/31 14:18:18	837	0.744630	0.744630	0.744630	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	SMT t2s + Spell correction + NMT reranking
48	EHR	en-ja	2015/09/09 12:21:34	873	0.630410	0.630410	0.630410	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	System combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
49	EHR	en-ja	2015/09/09 12:22:58	874	0.604090	0.604090	0.604090	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with user dictionary plus SPE.
50	ORGANIZER	en-ja	2015/09/10 13:29:35	875	0.744370	0.744370	0.744370	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Tree-to-String SMT (2015)
51	ORGANIZER	en-ja	2015/09/10 14:26:28	883	0.622930	0.622930	0.622930	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	RBMT B (2015)
52	ORGANIZER	en-ja	2015/09/10 19:02:38	889	0.646160	0.646160	0.646160	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online B (2015)
53	ORGANIZER	en-ja	2016/07/26 11:31:47	1041	0.677020	0.677020	0.677020	-	-	-	-	0.000000	0.000000	0.000000	Other	Yes	Online A (2016)
54	NICT-2	en-ja	2016/08/05 17:47:05	1097	0.753080	0.753080	0.753080	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
55	TOKYOMT	en-ja	2016/08/12 11:21:43	1131	0.705210	0.705210	0.705210	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	char 1 , ens 2 , version 1
56	EHR	en-ja	2016/08/15 11:21:51	1140	0.746720	0.746720	0.746720	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	PBSMT with preordering (DL=6)
57	bjtu_nlp	en-ja	2016/08/16 11:21:07	1143	0.704340	0.704340	0.704340	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
58	JAPIO	en-ja	2016/08/17 12:50:50	1165	0.660790	0.660790	0.660790	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus + rule-based posteditor
59	Kyoto-U	en-ja	2016/08/18 03:45:51	1172	0.738700	0.738700	0.738700	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	BPE tgt/src: 52k 2-layer lstm self-ensemble of 3
60	Kyoto-U	en-ja	2016/08/19 10:18:09	1201	0.747510	0.747510	0.747510	-	-	-	-	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT 2016 w/o reranking
61	TOKYOMT	en-ja	2016/08/19 23:18:46	1217	0.720810	0.720810	0.720810	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Combination of NMT and T2S
62	UT-AKY	en-ja	2016/08/20 09:25:15	1224	0.708140	0.708140	0.708140	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	tree-to-seq NMT model (character-based decoder)
63	UT-AKY	en-ja	2016/08/20 12:34:43	1228	0.731440	0.731440	0.731440	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	tree-to-seq NMT model (word-based decoder)
64	ORGANIZER	en-ja	2016/11/16 10:50:13	1334	0.727040	0.727040	0.727040	-	-	-	-	0.000000	0.000000	0.000000	NMT	Yes	Online A (2016/11/14)
65	UT-KAY	en-ja	2017/06/27 16:58:42	1369	0.753090	0.753090	0.753090	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Hashimoto and Tsuruoka (2017), https://arxiv.org/abs/1702.02265
66	NICT-2	en-ja	2017/07/26 13:52:35	1475	0.759570	0.759570	0.759570	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
67	NICT-2	en-ja	2017/07/26 14:03:00	1479	0.765580	0.765580	0.765580	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT 6 Ensembles * Bi-directional Reranking
68	NAIST-NICT	en-ja	2017/07/27 21:49:04	1506	0.763310	0.763310	0.763310	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: SPM16k/16k, BiLSTM Encoder 51222, UniLSTM Decoder 512*2, One-best Search
69	NAIST-NICT	en-ja	2017/07/27 21:50:15	1507	0.770480	0.770480	0.770480	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: SPM16k/16k, BiLSTM Encoder 51222, UniLSTM Decoder 512*2, Adjusted Search
70	NTT	en-ja	2017/07/30 16:16:25	1608	0.756480	0.756480	0.756480	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
71	NTT	en-ja	2017/08/01 02:22:56	1673	0.763770	0.763770	0.763770	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Single Model: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
72	NTT	en-ja	2017/08/01 07:12:25	1684	0.757740	0.757740	0.757740	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
73	TMU	en-ja	2017/08/01 11:42:03	1704	0.740620	0.740620	0.740620	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	our baseline system in 2017
74	TMU	en-ja	2017/08/01 11:56:31	1709	0.744890	0.744890	0.744890	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	baseline system with beam20
75	UT-IIS	en-ja	2017/08/01 12:03:22	1710	0.746910	0.746910	0.746910	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT 8 Ensembles with beam search, Sentence Piece, Embedding layer initialization
76	NTT	en-ja	2017/08/01 15:42:13	1729	0.762170	0.762170	0.762170	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
77	Kyoto-U	en-ja	2017/08/01 15:49:51	1731	0.754220	0.754220	0.754220	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Ensemble of 4 BPE Averaged
78	ORGANIZER	en-ja	2017/08/02 01:04:49	1737	0.768630	0.768630	0.768630	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Google's "Attention Is All You Need"
79	TMU	en-ja	2017/08/02 10:20:14	1741	0.751660	0.751660	0.751660	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	the ensemble system of different dropout rate.
80	TMU	en-ja	2017/08/04 11:06:51	1747	0.749410	0.749410	0.749410	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	beam_size: 10, ensemble of different dropout rates.
81	Kyoto-U	en-ja	2017/09/04 22:31:18	1759	0.771020	0.771020	0.771020	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Syscomb of AIAYN and KNMT
82	ORGANIZER	en-ja	2018/08/14 10:58:49	1900	0.759910	0.759910	0.759910	-	-	-	-	-	0.000000	0.000000	NMT	No	NMT with Attention
83	NICT-5	en-ja	2018/08/22 18:43:00	2048	0.771400	0.771400	0.771400	-	-	-	-	-	0.000000	0.000000	NMT	No	Tensor2Tensor's transformer implementation. Used recurrently stacked layers model logic. Layer recurrence logic: 1-2-3-1-2-3-1-2-3-1-2-3. Codename: Megarecurrence.
84	NICT-5	en-ja	2018/08/22 18:48:51	2050	0.770560	0.770560	0.770560	-	-	-	-	-	0.000000	0.000000	NMT	No	RSNMT 6 layer
85	Sense	en-ja	2018/08/24 11:38:23	2092	0.741090	0.741090	0.741090	-	-	-	-	-	0.000000	0.000000	NMT	No
86	NICT-5	en-ja	2018/09/03 16:49:57	2219	0.779560	0.779560	0.779560	-	-	-	-	-	0.000000	0.000000	NMT	No	Big Bidirectional Transformer. 1.5M sentences only.
87	EHR	en-ja	2018/09/08 12:42:03	2245	0.758750	0.758750	0.758750	-	-	-	-	-	0.000000	0.000000	NMT	No	SMT reranked NMT
88	Osaka-U	en-ja	2018/09/15 23:02:05	2439	0.763140	0.763140	0.763140	-	-	-	-	-	0.000000	0.000000	NMT	Yes	rewarding model
89	TMU	en-ja	2018/09/16 12:10:49	2467	0.749190	0.749190	0.749190	-	-	-	-	-	0.000000	0.000000	NMT	No	Baseline-NMT ( Single )
90	TMU	en-ja	2018/09/16 12:11:58	2468	0.750350	0.750350	0.750350	-	-	-	-	-	0.000000	0.000000	NMT	No	GAN-NMT ( Single )
91	TMU	en-ja	2018/09/16 12:44:23	2469	0.753040	0.753040	0.753040	-	-	-	-	-	0.000000	0.000000	NMT	No	Ensemble of 6 Baseline-NMT
92	Osaka-U	en-ja	2018/09/16 13:06:45	2470	0.705050	0.705050	0.705050	-	-	-	-	-	0.000000	0.000000	SMT	No	preordering with neural network
93	srcb	en-ja	2018/09/16 15:26:37	2479	0.781000	0.781000	0.781000	-	-	-	-	-	0.000000	0.000000	NMT	No	Transformer with relative position, ensemble of 3 models.
94	srcb	en-ja	2018/09/16 15:52:17	2480	0.779820	0.779820	0.779820	-	-	-	-	-	0.000000	0.000000	NMT	No	Transformer with relative position, ensemble of 4 models, rerank.
95	TMU	en-ja	2018/09/16 16:56:49	2482	0.752100	0.752100	0.752100	-	-	-	-	-	0.000000	0.000000	NMT	No	Reconstructor-NMT ( Single )
96	TMU	en-ja	2018/09/16 17:25:36	2484	0.752420	0.752420	0.752420	-	-	-	-	-	0.000000	0.000000	NMT	No	Ensemble of 6 NMT ( 2 Baseline + 2 Reconstructor + 2 GAN )

Notice:

This table is sorted by the leftmost segmenters. You can change the segmenter used to sort by clicking each segmenter link.
Adequacy-Fluency Metrics (AMFM) is a two-dimensional automatic evaluation metric for machine translation, designed to operate at the sentence level. It is based on adequacy and fluency, to decouple semantic and syntactic components of the translation process to provide a balanced view on translation quality.
AMFM is calculated without tokenizers.
The detail of AMFM is shown on the following paper: "Adequacy–Fluency Metrics: Evaluating MT in the Continuous Space Model Framework" [pdf]. Invited Talk in WAT2015 also helps understanding [slide].

HUMAN (WAT2022)

Notice:

HUMAN (WAT2022) is the result of the Pairwise Crowdsourcing Evaluation on WAT2022.
HUMAN (WAT2022) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2021)

Notice:

HUMAN (WAT2021) is the result of the Pairwise Crowdsourcing Evaluation on WAT2021.
HUMAN (WAT2021) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2020)

Notice:

HUMAN (WAT2020) is the result of the Pairwise Crowdsourcing Evaluation on WAT2020.
HUMAN (WAT2020) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2019)

Notice:

HUMAN (WAT2019) is the result of the Pairwise Crowdsourcing Evaluation on WAT2019.
HUMAN (WAT2019) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2018)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	NICT-5	en-ja	2018/09/03 16:49:57	2219	28.500	NMT	No	Big Bidirectional Transformer. 1.5M sentences only.
2	srcb	en-ja	2018/09/16 15:26:37	2479	25.000	NMT	No	Transformer with relative position, ensemble of 3 models.
3	NICT-5	en-ja	2018/08/22 18:43:00	2048	20.250	NMT	No	Tensor2Tensor's transformer implementation. Used recurrently stacked layers model logic. Layer recurrence logic: 1-2-3-1-2-3-1-2-3-1-2-3. Codename: Megarecurrence.
4	Osaka-U	en-ja	2018/09/15 23:02:05	2439	4.500	NMT	Yes	rewarding model
5	EHR	en-ja	2018/09/08 12:42:03	2245	-0.500	NMT	No	SMT reranked NMT
6	TMU	en-ja	2018/09/16 12:44:23	2469	-12.000	NMT	No	Ensemble of 6 Baseline-NMT
7	Osaka-U	en-ja	2018/09/16 13:06:45	2470	-82.250	SMT	No	preordering with neural network

Notice:

HUMAN (WAT2018) is the result of the Pairwise Crowdsourcing Evaluation on WAT2018.
HUMAN (WAT2018) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2017)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	NTT	en-ja	2017/08/01 15:42:13	1729	75.750	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking, add 1M pseudo training data (generated from provided training data)
2	NICT-2	en-ja	2017/07/26 14:03:00	1479	74.750	NMT	No	NMT 6 Ensembles * Bi-directional Reranking
3	NTT	en-ja	2017/08/01 07:12:25	1684	72.250	NMT	No	Ensemble 8 Models: joint BPE 16k, BiLSTM Encoder 51222, LtoR LSTM Decoder 512*2, Beam Search 20 w/ length-based reranking
4	NAIST-NICT	en-ja	2017/07/27 21:50:15	1507	70.000	NMT	No	Single Model: SPM16k/16k, BiLSTM Encoder 51222, UniLSTM Decoder 512*2, Adjusted Search
5	Kyoto-U	en-ja	2017/08/01 15:49:51	1731	69.750	NMT	No	Ensemble of 4 BPE Averaged
6	ORGANIZER	en-ja	2017/08/02 01:04:49	1737	69.750	NMT	No	Google's "Attention Is All You Need"
7	UT-IIS	en-ja	2017/08/01 12:03:22	1710	68.000	NMT	No	NMT 8 Ensembles with beam search, Sentence Piece, Embedding layer initialization
8	NAIST-NICT	en-ja	2017/07/27 21:49:04	1506	63.500	NMT	No	Single Model: SPM16k/16k, BiLSTM Encoder 51222, UniLSTM Decoder 512*2, One-best Search
9	NICT-2	en-ja	2017/07/26 13:52:35	1475	62.000	NMT	No	NMT Single Model: BPE50k, Bi-LSTM(500*2) Encoder, LSTM(1000) Left-to-Right Decoder
10	TMU	en-ja	2017/08/01 11:56:31	1709	56.500	NMT	No	baseline system with beam20
11	TMU	en-ja	2017/08/01 11:42:03	1704	50.750	NMT	No	our baseline system in 2017

Notice:

HUMAN (WAT2017) is the result of the Pairwise Crowdsourcing Evaluation on WAT2017.
HUMAN (WAT2017) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2016)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	ORGANIZER	en-ja	2016/11/16 10:50:13	1334	74.000	NMT	Yes	Online A (2016/11/14)
2	Kyoto-U	en-ja	2016/08/18 03:45:51	1172	55.250	NMT	No	BPE tgt/src: 52k 2-layer lstm self-ensemble of 3
3	ORGANIZER	en-ja	2016/07/26 11:31:47	1041	49.750	Other	Yes	Online A (2016)
4	NICT-2	en-ja	2016/08/05 17:47:05	1097	41.250	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
5	bjtu_nlp	en-ja	2016/08/16 11:21:07	1143	39.500	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
6	EHR	en-ja	2016/08/15 11:21:51	1140	39.000	SMT	No	PBSMT with preordering (DL=6)
7	UT-AKY	en-ja	2016/08/20 12:34:43	1228	36.250	NMT	No	tree-to-seq NMT model (word-based decoder)
8	TOKYOMT	en-ja	2016/08/19 23:18:46	1217	30.500	NMT	No	Combination of NMT and T2S
9	TOKYOMT	en-ja	2016/08/12 11:21:43	1131	29.750	NMT	No	char 1 , ens 2 , version 1
10	UT-AKY	en-ja	2016/08/20 09:25:15	1224	21.750	NMT	No	tree-to-seq NMT model (character-based decoder)
11	JAPIO	en-ja	2016/08/17 12:50:50	1165	4.250	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus + rule-based posteditor

Notice:

HUMAN (WAT2016) is the result of the Pairwise Crowdsourcing Evaluation on WAT2016.
HUMAN (WAT2016) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2015)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	NAIST	en-ja	2015/08/25 12:39:23	761	62.250	SMT	No	Travatar System with NeuralMT Reranking
2	W	en-ja	2015/08/26 16:00:46	786	53.750	SMT	No	NMT, LSTM Search, 5 ensembles, beam size 20, UNK replacing, System Combination with NMT score (Pick top-1k results from NMT)
3	naver	en-ja	2015/08/31 14:18:18	837	53.250	SMT	No	SMT t2s + Spell correction + NMT reranking
4	Kyoto-U	en-ja	2015/08/30 19:59:18	832	51.000	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking
5	NAIST	en-ja	2015/08/25 12:47:49	763	49.750	SMT	No	Travatar System Baseline
6	naver	en-ja	2015/08/31 14:12:03	836	48.500	SMT	No	NMT only
7	W	en-ja	2015/08/28 14:30:56	813	43.500	Other	No	NMT, LSTM Search, Beam Size 20, Ensemble of 2 models, UNK replacing
8	Kyoto-U	en-ja	2015/08/27 22:56:13	805	40.500	EBMT	No	KyotoEBMT system without reranking
9	TOSHIBA	en-ja	2015/07/28 16:24:37	524	40.250	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
10	ORGANIZER	en-ja	2015/08/25 18:54:29	774	34.250	Other	Yes	Online A (2015)
11	EHR	en-ja	2015/08/22 12:28:15	742	32.500	SMT	No	Phrase based SMT with preordering.
12	ORGANIZER	en-ja	2015/09/10 13:29:35	875	30.000	SMT	No	Tree-to-String SMT (2015)
13	ORGANIZER	en-ja	2015/09/10 14:26:28	883	9.750	Other	Yes	RBMT B (2015)
14	Sense	en-ja	2015/08/18 22:04:09	715	-31.000	SMT	Yes	Passive JSTx3
15	Sense	en-ja	2015/08/18 21:52:01	700	-36.250	SMT	No	Baseline-dictmt

Notice:

HUMAN (WAT2015) is the result of the Pairwise Crowdsourcing Evaluation on WAT2015.
HUMAN (WAT2015) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.
The detail of the evaluation can be found in the pdf document (PDF file).

HUMAN (WAT2014)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	NAIST	en-ja	2014/07/31 11:38:37	118	56.250	SMT	No	Travatar-based Forest-to-String SMT System
2	NAIST	en-ja	2014/08/01 17:37:23	126	51.500	SMT	No	Travatar-based Forest-to-String SMT System (Tuned BLEU+RIBES)
3	W	en-ja	2014/08/16 00:57:16	132	43.250	SMT	No	Weblio Pre-reordering SMT System Baseline
4	ORGANIZER	en-ja	2014/07/18 11:02:25	34	42.500	Other	Yes	Online A (2014)
5	Kyoto-U	en-ja	2014/09/01 21:06:32	267	38.000	EBMT	No	Our new baseline system after several modifications + 20-best parses, KN7, RNNLM reranking
6	W	en-ja	2014/08/26 16:17:15	202	36.000	SMT	No	Weblio Pre-reordering SMT System (with forest inputs)
7	ORGANIZER	en-ja	2014/07/11 20:03:21	12	34.250	SMT	No	Tree-to-String SMT (2014)
8	Kyoto-U	en-ja	2014/08/31 10:33:23	253	33.750	EBMT	No	Our new baseline system after several modifications.
9	ORGANIZER	en-ja	2014/09/16 13:36:35	367	31.500	SMT	No	Hierarchical Phrase-based SMT (2014)
10	SAS_MT	en-ja	2014/09/01 10:39:27	264	27.500	SMT	No	Syntactic reordering Hierarchical SMT （using part of data）
11	Sense	en-ja	2014/08/25 01:07:27	184	3.750	SMT	No	Baseline SMT
12	ORGANIZER	en-ja	2014/07/21 11:38:12	66	0.750	Other	Yes	RBMT B (2014)

Notice:

HUMAN (WAT2014) is the result of the Pairwise Crowdsourcing Evaluation on WAT2014.
HUMAN (WAT2014) was evaluated by 3 different workers and the final decision is made by the voting of the judgements.
The detail of the evaluation can be found in the pdf document (PDF file).

EVALUATION RESULTS USAGE POLICY

When you use the WAT evaluation results for any purpose such as:
- writing technical papers,
- making presentations about your system,
- advertising your MT system to the customers,
you can use the information about translation directions, scores (including both automatic and human evaluations) and ranks of your system among others. You can also use the scores of the other systems, but you MUST anonymize the other system's names. In addition, you can show the links (URLs) to the WAT evaluation result pages.

NICT (National Institute of Information and Communications Technology)
Kyoto University
Last Modified: 2018-08-02

WAT The Workshop on Asian Translation Evaluation Results

BLEU

RIBES

AMFM

HUMAN (WAT2022)

HUMAN (WAT2021)

HUMAN (WAT2020)

HUMAN (WAT2019)

HUMAN (WAT2018)

HUMAN (WAT2017)

HUMAN (WAT2016)

HUMAN (WAT2015)

HUMAN (WAT2014)

EVALUATION RESULTS USAGE POLICY

WAT

The Workshop on Asian Translation

Evaluation Results