WAT

The Workshop on Asian Translation

Evaluation Results

BLEU

#	Team	Task	Date/Time	DataID	BLEU										Method	Other Resources	System Description
#	Team	Task	Date/Time	DataID	juman	kytea	mecab	moses- tokenizer	stanford- segmenter- ctb	stanford- segmenter- pku	indic- tokenizer	unuse	myseg	kmseg	Method	Other Resources	System Description
1	JAPIO	JPCzh-ja	2016/08/17 11:48:56	1161	58.66	59.19	58.63	-	-	-	-	0.00	0.00	0.00	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus including some sentences in testset
2	KNU_Hyundai	JPCzh-ja	2019/07/27 08:29:23	3153	53.59	53.91	53.47	-	-	-	-	-	-	-	NMT	Yes	Transformer(base) + Used ASPEC corpus with relative position, bt, multi source, r2l rerank, 5-model ensemble
3	Bering Lab	JPCzh-ja	2021/05/04 06:53:17	6171	52.99	53.57	53.03	-	-	-	-	-	-	-	NMT	Yes	Transformer Ensemble with additional crawled parallel corpus
4	sakura	JPCzh-ja	2024/08/08 19:35:56	7264	52.60	53.40	52.80	-	-	-	-	-	-	-	NMT	No	LLM: Rakuten/RakutenAI-7B-chat Fine-Tuned with JPC Corpus in six direction (En-Ja, Ja-En, Ko-Ja, Ja-Ko, Zh-Ja, Ja-Zh) - Best
5	sakura	JPCzh-ja	2024/08/09 00:24:08	7296	52.40	53.20	52.50	-	-	-	-	-	-	-	NMT	No	LLM: Rakuten/RakutenAI-7B-chat Fine-Tuned with JPC Corpus in six direction (En-Ja, Ja-En, Ko-Ja, Ja-Ko, Zh-Ja, Ja-Zh)
6	sarah	JPCzh-ja	2019/07/26 11:32:28	2976	50.90	52.04	51.09	-	-	-	-	-	-	-	NMT	No	Transformer, ensemble of 4 models
7	JAPIO	JPCzh-ja	2017/07/25 12:22:07	1447	50.52	51.25	50.57	-	-	-	-	0.00	0.00	0.00	SMT	Yes	Phrase-based SMT with Preordering + Japio corpus
8	JAPIO	JPCzh-ja	2017/07/26 14:21:18	1484	50.06	50.51	50.00	-	-	-	-	0.00	0.00	0.00	NMT	Yes	Combination of 3 NMT systems (OpenNMT(dbrnn) + JPC/Japio corpora + NMT/rule-based posteditor)
9	ryan	JPCzh-ja	2019/07/25 22:12:26	2954	49.98	50.67	50.12	-	-	-	-	-	-	-	NMT	No	Base Transformer
10	JAPIO	JPCzh-ja	2017/07/26 14:09:22	1482	49.51	50.00	49.48	-	-	-	-	0.00	0.00	0.00	NMT	Yes	OpenNMT(dbrnn) + JPC/Japio corpora + NMT/rule-based posteditor
11	tpt_wat	JPCzh-ja	2021/04/27 01:42:09	5690	49.42	50.46	49.55	-	-	-	-	-	-	-	NMT	No	Base Transformer model with shared vocab 8k size
12	goku20	JPCzh-ja	2020/09/21 11:54:22	4087	49.07	50.04	49.34	-	-	-	-	-	-	-	NMT	No	mBART pre-training transformer, single model
13	goku20	JPCzh-ja	2020/09/22 00:04:26	4105	48.98	49.92	49.21	-	-	-	-	-	-	-	NMT	No	mBART pre-training transformer, ensemble of 3 models
14	USTC	JPCzh-ja	2018/08/31 17:24:35	2206	48.37	49.78	48.57	-	-	-	-	-	0.00	0.00	NMT	No	tensor2tensor, 4 model average, r2l rerank
15	EHR	JPCzh-ja	2018/08/31 18:51:15	2210	48.10	48.51	47.96	-	-	-	-	-	0.00	0.00	NMT	No	SMT reranked NMT
16	EHR	JPCzh-ja	2017/07/19 19:28:31	1408	47.08	47.44	46.83	-	-	-	-	0.00	0.00	0.00	NMT	No	SMT reranked NMT (word based, by Moses and OpenNMT)
17	EHR	JPCzh-ja	2017/07/19 20:41:27	1414	46.52	47.17	46.35	-	-	-	-	0.00	0.00	0.00	NMT	No	SMT reranked NMT (character based, by Moses and OpenNMT)
18	ORGANIZER	JPCzh-ja	2018/08/15 18:29:31	1963	46.32	46.73	46.11	-	-	-	-	-	0.00	0.00	NMT	No	NMT with Attention
19	EHR	JPCzh-ja	2017/07/19 20:45:00	1415	46.03	46.42	45.95	-	-	-	-	0.00	0.00	0.00	NMT	No	Simple NMT (word based, by OpenNMT)
20	EHR	JPCzh-ja	2017/07/19 19:35:03	1409	45.27	45.87	45.24	-	-	-	-	0.00	0.00	0.00	NMT	No	Simple NMT (character based, by OpenNMT)
21	JAPIO	JPCzh-ja	2017/07/25 18:26:52	1458	45.07	45.79	45.10	-	-	-	-	0.00	0.00	0.00	NMT	No	OpenNMT(dbrnn)
22	NTT	JPCzh-ja	2016/08/19 08:53:34	1199	44.99	45.84	45.02	-	-	-	-	0.00	0.00	0.00	NMT	No	Baseline NMT with attention over bidirectional LSTMs (by Harvard NMT)
23	JAPIO	JPCzh-ja	2016/08/19 08:26:57	1192	44.32	45.12	44.09	-	-	-	-	0.00	0.00	0.00	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus
24	JAPIO	JPCzh-ja	2016/08/18 14:15:46	1180	43.87	44.47	43.66	-	-	-	-	0.00	0.00	0.00	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus
25	NTT	JPCzh-ja	2016/08/19 08:55:20	1200	43.47	44.27	43.53	-	-	-	-	0.00	0.00	0.00	NMT	No	NMT with pre-ordering and attention over bidirectional LSTMs (pre-ordering module is the same as the PBMT submission)
26	ORGANIZER	JPCzh-ja	2016/11/16 11:19:58	1341	42.66	43.76	42.95	-	-	-	-	0.00	0.00	0.00	NMT	Yes	Online A (2016/11/14)
27	NICT-2	JPCzh-ja	2016/08/05 18:06:47	1100	41.87	42.39	42.13	-	-	-	-	0.00	0.00	0.00	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
28	TOSHIBA	JPCzh-ja	2015/07/23 14:43:30	504	41.82	41.90	41.60	-	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	Combination of phrase-based SMT and SPE systems.
29	Kyoto-U	JPCzh-ja	2015/09/02 09:25:04	864	41.35	41.92	41.16	-	-	-	0.00	0.00	0.00	0.00	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking (only character-base model)
30	TOSHIBA	JPCzh-ja	2015/07/28 16:30:41	526	41.12	40.87	40.59	-	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
31	NICT-2	JPCzh-ja	2016/08/04 17:34:38	1079	41.09	41.27	41.24	-	-	-	-	0.00	0.00	0.00	SMT	No	Phrase-based SMT with Preordering + Domain Adaptation
32	EHR	JPCzh-ja	2015/08/17 14:05:20	671	41.06	42.24	41.15	-	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	System combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
33	EHR	JPCzh-ja	2016/07/18 15:33:03	1009	41.05	41.05	40.52	-	-	-	-	0.00	0.00	0.00	SMT and RBMT	Yes	Combination of word-based PBSMT, character-based PBSMT and RBMT+PBSPE with DL=6.
34	EHR	JPCzh-ja	2016/07/18 15:25:53	1007	40.95	41.20	40.51	-	-	-	-	0.00	0.00	0.00	SMT	Yes	Combination of word-based PBSMT and character-based PBSMT with DL=6.
35	NTT	JPCzh-ja	2016/08/19 08:28:00	1193	40.75	41.05	40.68	-	-	-	-	0.00	0.00	0.00	SMT	No	PBMT with pre-ordering on dependency structures
36	EHR	JPCzh-ja	2015/08/30 15:22:25	830	40.70	41.49	40.79	-	-	-	0.00	0.00	0.00	0.00	SMT	Yes	Phrase based SMT with preordering
37	NTT	JPCzh-ja	2015/08/21 08:07:18	736	40.60	41.10	40.63	-	-	-	0.00	0.00	0.00	0.00	SMT	No	A pre-ordering-based PBMT with patent-tuned dependency parsing and phrase table smoothing.
38	EHR	JPCzh-ja	2015/08/30 12:42:52	828	40.35	40.16	39.92	-	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	RBMT with user dictionary plus SPE
39	NTT	JPCzh-ja	2015/08/28 09:53:24	811	39.77	40.08	39.88	-	-	-	0.00	0.00	0.00	0.00	SMT	No	A pre-ordering-based PBMT with patent-tuned dependency parsing, learning-based pre-ordering, and phrase table smoothing.
40	ORGANIZER	JPCzh-ja	2015/05/14 18:00:16	432	39.39	39.90	39.39	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Tree-to-String SMT (2015)
41	bjtu_nlp	JPCzh-ja	2016/08/09 18:44:56	1128	39.34	39.72	39.30	-	-	-	-	0.00	0.00	0.00	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
42	JAPIO	JPCzh-ja	2016/10/27 13:01:42	1329	39.29	40.67	39.51	-	-	-	-	0.00	0.00	0.00	SMT	No	Phrase-based SMT with Preordering
43	ORGANIZER	JPCzh-ja	2015/05/14 17:55:51	430	39.22	39.52	39.14	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Hierarchical Phrase-based SMT
44	ORGANIZER	JPCzh-ja	2016/07/15 11:22:35	998	39.07	39.45	38.95	-	-	-	-	0.00	0.00	0.00	SMT	No	Tree-to-String SMT (2016)
45	NTT	JPCzh-ja	2016/08/19 08:26:18	1191	39.03	39.17	38.99	-	-	-	-	0.00	0.00	0.00	SMT	No	Baseline PBMT (Moses)
46	Sense	JPCzh-ja	2016/08/29 01:06:19	1281	38.90	38.58	38.65	-	-	-	-	0.00	0.00	0.00	SMT	No	Clustercat-C10-PBMT
47	u-tkb	JPCzh-ja	2017/07/26 12:44:18	1468	38.79	40.47	38.99	-	-	-	-	0.00	0.00	0.00	NMT	No	NMT with SMT phrase translation (phrase extraction with branching entropy; attention over bidirectional LSTMs; by Harvard NMT)
48	Sense	JPCzh-ja	2016/08/29 09:55:33	1284	38.75	38.32	38.38	-	-	-	-	0.00	0.00	0.00	SMT	No	Baseline-C10-PBMT
49	Sense	JPCzh-ja	2016/08/29 23:08:29	1292	38.71	38.35	38.38	-	-	-	-	0.00	0.00	0.00	SMT	No	Baseline-C50-PBMT
50	Sense	JPCzh-ja	2016/08/30 07:37:39	1294	38.71	38.51	38.54	-	-	-	-	0.00	0.00	0.00	SMT	No	Clustercat-C50-PBMT
51	ORGANIZER	JPCzh-ja	2015/05/14 17:58:14	431	38.34	38.51	38.22	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Phrase-based SMT
52	Kyoto-U	JPCzh-ja	2015/08/26 13:10:44	781	37.87	38.62	37.71	-	-	-	0.00	0.00	0.00	0.00	EBMT	No	Baseline w/o reranking
53	WASUIPS	JPCzh-ja	2015/09/01 14:16:16	853	33.48	34.55	33.55	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Combining sampling-based alignment and bilingual hierarchical sub-sentential alignment methods.
54	WASUIPS	JPCzh-ja	2016/10/12 21:06:36	1326	31.00	31.63	30.86	-	-	-	-	0.00	0.00	0.00	SMT	No	Our improved system: train=100,000 tune=1,000 test=2,000 BLEU (our PC): 33.61. Using bilingual term extraction and re-tokenization for Chinese–Japanese.
55	WASUIPS	JPCzh-ja	2016/10/12 21:04:52	1325	29.38	30.82	29.66	-	-	-	-	0.00	0.00	0.00	SMT	No	Our baseline system: train=100,000 tune=1,000 test=2,000 BLEU (our PC): 32.29.
56	TOSHIBA	JPCzh-ja	2015/08/17 11:53:34	667	28.06	27.44	27.56	-	-	-	0.00	0.00	0.00	0.00	RBMT	Yes	RBMT
57	ORGANIZER	JPCzh-ja	2016/07/26 11:18:45	1040	26.99	27.91	27.02	-	-	-	-	0.00	0.00	0.00	Other	Yes	Online A (2016)
58	ORGANIZER	JPCzh-ja	2015/08/14 16:52:02	647	26.80	27.81	26.89	-	-	-	0.00	0.00	0.00	0.00	Other	Yes	Online A (2015)
59	EHR	JPCzh-ja	2018/05/04 14:17:39	1803	15.77	16.12	15.53	-	-	-	-	0.00	0.00	0.00	RBMT	Yes	RBMT system for WAT2015's submission
60	ORGANIZER	JPCzh-ja	2015/08/14 16:55:19	648	12.33	12.72	12.44	-	-	-	0.00	0.00	0.00	0.00	Other	Yes	Online B (2015)
61	ORGANIZER	JPCzh-ja	2015/08/25 11:42:02	759	10.49	10.72	10.35	-	-	-	0.00	0.00	0.00	0.00	RBMT	No	RBMT A (2015)
62	ORGANIZER	JPCzh-ja	2015/08/25 11:53:50	760	7.94	8.07	7.73	-	-	-	0.00	0.00	0.00	0.00	RBMT	No	RBMT B

Notice:

This table is sorted by the leftmost segmenters. You can change the segmenter used to sort by clicking each segmenter link.

RIBES

#	Team	Task	Date/Time	DataID	RIBES										Method	Other Resources	System Description
#	Team	Task	Date/Time	DataID	juman	kytea	mecab	moses- tokenizer	stanford- segmenter- ctb	stanford- segmenter- pku	indic- tokenizer	unuse	myseg	kmseg	Method	Other Resources	System Description
1	Bering Lab	JPCzh-ja	2021/05/04 06:53:17	6171	0.883744	0.881710	0.883204	-	-	-	-	-	-	-	NMT	Yes	Transformer Ensemble with additional crawled parallel corpus
2	KNU_Hyundai	JPCzh-ja	2019/07/27 08:29:23	3153	0.880251	0.879452	0.879471	-	-	-	-	-	-	-	NMT	Yes	Transformer(base) + Used ASPEC corpus with relative position, bt, multi source, r2l rerank, 5-model ensemble
3	sakura	JPCzh-ja	2024/08/09 00:24:08	7296	0.878851	0.876452	0.878847	-	-	-	-	-	-	-	NMT	No	LLM: Rakuten/RakutenAI-7B-chat Fine-Tuned with JPC Corpus in six direction (En-Ja, Ja-En, Ko-Ja, Ja-Ko, Zh-Ja, Ja-Zh)
4	sakura	JPCzh-ja	2024/08/08 19:35:56	7264	0.877180	0.875374	0.876600	-	-	-	-	-	-	-	NMT	No	LLM: Rakuten/RakutenAI-7B-chat Fine-Tuned with JPC Corpus in six direction (En-Ja, Ja-En, Ko-Ja, Ja-Ko, Zh-Ja, Ja-Zh) - Best
5	JAPIO	JPCzh-ja	2017/07/26 14:21:18	1484	0.875398	0.873390	0.874822	-	-	-	-	0.000000	0.000000	0.000000	NMT	Yes	Combination of 3 NMT systems (OpenNMT(dbrnn) + JPC/Japio corpora + NMT/rule-based posteditor)
6	JAPIO	JPCzh-ja	2017/07/26 14:09:22	1482	0.872625	0.870537	0.872038	-	-	-	-	0.000000	0.000000	0.000000	NMT	Yes	OpenNMT(dbrnn) + JPC/Japio corpora + NMT/rule-based posteditor
7	ryan	JPCzh-ja	2019/07/25 22:12:26	2954	0.869890	0.867919	0.868936	-	-	-	-	-	-	-	NMT	No	Base Transformer
8	sarah	JPCzh-ja	2019/07/26 11:32:28	2976	0.869159	0.867133	0.868496	-	-	-	-	-	-	-	NMT	No	Transformer, ensemble of 4 models
9	tpt_wat	JPCzh-ja	2021/04/27 01:42:09	5690	0.869034	0.867678	0.868716	-	-	-	-	-	-	-	NMT	No	Base Transformer model with shared vocab 8k size
10	JAPIO	JPCzh-ja	2016/08/17 11:48:56	1161	0.868027	0.864893	0.866692	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus including some sentences in testset
11	USTC	JPCzh-ja	2018/08/31 17:24:35	2206	0.866232	0.864284	0.865423	-	-	-	-	-	0.000000	0.000000	NMT	No	tensor2tensor, 4 model average, r2l rerank
12	goku20	JPCzh-ja	2020/09/21 11:54:22	4087	0.865852	0.863774	0.865703	-	-	-	-	-	-	-	NMT	No	mBART pre-training transformer, single model
13	goku20	JPCzh-ja	2020/09/22 00:04:26	4105	0.865111	0.863206	0.864862	-	-	-	-	-	-	-	NMT	No	mBART pre-training transformer, ensemble of 3 models
14	JAPIO	JPCzh-ja	2017/07/25 18:26:52	1458	0.859883	0.857056	0.859411	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	OpenNMT(dbrnn)
15	EHR	JPCzh-ja	2017/07/19 20:41:27	1414	0.859619	0.856784	0.858353	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	SMT reranked NMT (character based, by Moses and OpenNMT)
16	EHR	JPCzh-ja	2017/07/19 19:28:31	1408	0.859070	0.856376	0.858888	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	SMT reranked NMT (word based, by Moses and OpenNMT)
17	EHR	JPCzh-ja	2017/07/19 20:45:00	1415	0.858591	0.855917	0.858511	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Simple NMT (word based, by OpenNMT)
18	EHR	JPCzh-ja	2018/08/31 18:51:15	2210	0.858259	0.855649	0.858142	-	-	-	-	-	0.000000	0.000000	NMT	No	SMT reranked NMT
19	ORGANIZER	JPCzh-ja	2018/08/15 18:29:31	1963	0.857318	0.855085	0.856442	-	-	-	-	-	0.000000	0.000000	NMT	No	NMT with Attention
20	EHR	JPCzh-ja	2017/07/19 19:35:03	1409	0.854447	0.852615	0.853226	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Simple NMT (character based, by OpenNMT)
21	NTT	JPCzh-ja	2016/08/19 08:53:34	1199	0.853004	0.851859	0.852430	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Baseline NMT with attention over bidirectional LSTMs (by Harvard NMT)
22	JAPIO	JPCzh-ja	2017/07/25 12:22:07	1447	0.847793	0.843774	0.846081	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + Japio corpus
23	ORGANIZER	JPCzh-ja	2016/11/16 11:19:58	1341	0.845858	0.844918	0.845794	-	-	-	-	0.000000	0.000000	0.000000	NMT	Yes	Online A (2016/11/14)
24	NTT	JPCzh-ja	2016/08/19 08:55:20	1200	0.845271	0.843105	0.844968	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT with pre-ordering and attention over bidirectional LSTMs (pre-ordering module is the same as the PBMT submission)
25	bjtu_nlp	JPCzh-ja	2016/08/09 18:44:56	1128	0.835314	0.830505	0.833216	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
26	JAPIO	JPCzh-ja	2016/08/19 08:26:57	1192	0.834959	0.830164	0.832955	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus
27	JAPIO	JPCzh-ja	2016/08/18 14:15:46	1180	0.833586	0.829360	0.831534	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus
28	u-tkb	JPCzh-ja	2017/07/26 12:44:18	1468	0.832144	0.833610	0.831209	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT with SMT phrase translation (phrase extraction with branching entropy; attention over bidirectional LSTMs; by Harvard NMT)
29	NICT-2	JPCzh-ja	2016/08/05 18:06:47	1100	0.829640	0.826744	0.828107	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
30	Kyoto-U	JPCzh-ja	2015/09/02 09:25:04	864	0.828543	0.824199	0.827230	-	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking (only character-base model)
31	EHR	JPCzh-ja	2016/07/18 15:25:53	1007	0.828040	0.824502	0.826864	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Combination of word-based PBSMT and character-based PBSMT with DL=6.
32	EHR	JPCzh-ja	2016/07/18 15:33:03	1009	0.827048	0.821940	0.824852	-	-	-	-	0.000000	0.000000	0.000000	SMT and RBMT	Yes	Combination of word-based PBSMT, character-based PBSMT and RBMT+PBSPE with DL=6.
33	NICT-2	JPCzh-ja	2016/08/04 17:34:38	1079	0.827009	0.822664	0.825323	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Phrase-based SMT with Preordering + Domain Adaptation
34	EHR	JPCzh-ja	2015/08/17 14:05:20	671	0.826987	0.821983	0.825056	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	System combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
35	NTT	JPCzh-ja	2016/08/19 08:28:00	1193	0.825985	0.822125	0.824840	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	PBMT with pre-ordering on dependency structures
36	EHR	JPCzh-ja	2015/08/30 15:22:25	830	0.824264	0.821055	0.823192	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Phrase based SMT with preordering
37	NTT	JPCzh-ja	2015/08/21 08:07:18	736	0.823436	0.820252	0.822026	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	A pre-ordering-based PBMT with patent-tuned dependency parsing and phrase table smoothing.
38	TOSHIBA	JPCzh-ja	2015/07/28 16:30:41	526	0.822268	0.814249	0.818981	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
39	TOSHIBA	JPCzh-ja	2015/07/23 14:43:30	504	0.820568	0.813536	0.817614	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	Combination of phrase-based SMT and SPE systems.
40	JAPIO	JPCzh-ja	2016/10/27 13:01:42	1329	0.820339	0.817352	0.819850	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Phrase-based SMT with Preordering
41	EHR	JPCzh-ja	2015/08/30 12:42:52	828	0.819516	0.812982	0.816743	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with user dictionary plus SPE
42	NTT	JPCzh-ja	2015/08/28 09:53:24	811	0.816288	0.811911	0.815543	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	A pre-ordering-based PBMT with patent-tuned dependency parsing, learning-based pre-ordering, and phrase table smoothing.
43	ORGANIZER	JPCzh-ja	2015/05/14 18:00:16	432	0.814919	0.811350	0.813595	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Tree-to-String SMT (2015)
44	ORGANIZER	JPCzh-ja	2016/07/15 11:22:35	998	0.813135	0.809893	0.811644	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Tree-to-String SMT (2016)
45	ORGANIZER	JPCzh-ja	2015/05/14 17:55:51	430	0.806058	0.802059	0.804523	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Hierarchical Phrase-based SMT
46	NTT	JPCzh-ja	2016/08/19 08:26:18	1191	0.805702	0.797991	0.802998	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Baseline PBMT (Moses)
47	Sense	JPCzh-ja	2016/08/29 09:55:33	1284	0.804673	0.795449	0.801496	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Baseline-C10-PBMT
48	Sense	JPCzh-ja	2016/08/30 07:37:39	1294	0.804301	0.796349	0.801596	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Clustercat-C50-PBMT
49	Sense	JPCzh-ja	2016/08/29 01:06:19	1281	0.803155	0.794679	0.800689	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Clustercat-C10-PBMT
50	Sense	JPCzh-ja	2016/08/29 23:08:29	1292	0.802673	0.794127	0.799531	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Baseline-C50-PBMT
51	Kyoto-U	JPCzh-ja	2015/08/26 13:10:44	781	0.799730	0.797700	0.798979	-	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	Baseline w/o reranking
52	ORGANIZER	JPCzh-ja	2015/05/14 17:58:14	431	0.782019	0.778921	0.781456	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Phrase-based SMT
53	WASUIPS	JPCzh-ja	2015/09/01 14:16:16	853	0.773985	0.771099	0.772202	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Combining sampling-based alignment and bilingual hierarchical sub-sentential alignment methods.
54	TOSHIBA	JPCzh-ja	2015/08/17 11:53:34	667	0.772054	0.758756	0.767076	-	-	-	0.000000	0.000000	0.000000	0.000000	RBMT	Yes	RBMT
55	WASUIPS	JPCzh-ja	2016/10/12 21:06:36	1326	0.754626	0.751804	0.753376	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Our improved system: train=100,000 tune=1,000 test=2,000 BLEU (our PC): 33.61. Using bilingual term extraction and re-tokenization for Chinese–Japanese.
56	WASUIPS	JPCzh-ja	2016/10/12 21:04:52	1325	0.751847	0.748474	0.750678	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Our baseline system: train=100,000 tune=1,000 test=2,000 BLEU (our PC): 32.29.
57	EHR	JPCzh-ja	2018/05/04 14:17:39	1803	0.721715	0.710085	0.716303	-	-	-	-	0.000000	0.000000	0.000000	RBMT	Yes	RBMT system for WAT2015's submission
58	ORGANIZER	JPCzh-ja	2015/08/14 16:52:02	647	0.712242	0.707264	0.711273	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online A (2015)
59	ORGANIZER	JPCzh-ja	2016/07/26 11:18:45	1040	0.707739	0.702718	0.706707	-	-	-	-	0.000000	0.000000	0.000000	Other	Yes	Online A (2016)
60	ORGANIZER	JPCzh-ja	2015/08/25 11:42:02	759	0.674060	0.664098	0.667349	-	-	-	0.000000	0.000000	0.000000	0.000000	RBMT	No	RBMT A (2015)
61	ORGANIZER	JPCzh-ja	2015/08/14 16:55:19	648	0.648996	0.641255	0.648742	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online B (2015)
62	ORGANIZER	JPCzh-ja	2015/08/25 11:53:50	760	0.596200	0.581837	0.586941	-	-	-	0.000000	0.000000	0.000000	0.000000	RBMT	No	RBMT B

Notice:

This table is sorted by the leftmost segmenters. You can change the segmenter used to sort by clicking each segmenter link.

AMFM

#	Team	Task	Date/Time	DataID	AMFM										Method	Other Resources	System Description
#	Team	Task	Date/Time	DataID	unuse	unuse	unuse	unuse	unuse	unuse	unuse	unuse	unuse	unuse	Method	Other Resources	System Description
1	Bering Lab	JPCzh-ja	2021/05/04 06:53:17	6171	0.896405	0.896405	0.896405	-	-	-	-	-	-	-	NMT	Yes	Transformer Ensemble with additional crawled parallel corpus
2	tpt_wat	JPCzh-ja	2021/04/27 01:42:09	5690	0.886918	0.886918	0.886918	-	-	-	-	-	-	-	NMT	No	Base Transformer model with shared vocab 8k size
3	JAPIO	JPCzh-ja	2016/08/17 11:48:56	1161	0.808090	0.808090	0.808090	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus including some sentences in testset
4	JAPIO	JPCzh-ja	2017/07/26 14:21:18	1484	0.779420	0.779420	0.779420	-	-	-	-	0.000000	0.000000	0.000000	NMT	Yes	Combination of 3 NMT systems (OpenNMT(dbrnn) + JPC/Japio corpora + NMT/rule-based posteditor)
5	JAPIO	JPCzh-ja	2017/07/26 14:09:22	1482	0.777460	0.777460	0.777460	-	-	-	-	0.000000	0.000000	0.000000	NMT	Yes	OpenNMT(dbrnn) + JPC/Japio corpora + NMT/rule-based posteditor
6	JAPIO	JPCzh-ja	2017/07/25 12:22:07	1447	0.774660	0.774660	0.774660	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + Japio corpus
7	USTC	JPCzh-ja	2018/08/31 17:24:35	2206	0.771310	0.771310	0.771310	-	-	-	-	-	0.000000	0.000000	NMT	No	tensor2tensor, 4 model average, r2l rerank
8	EHR	JPCzh-ja	2018/08/31 18:51:15	2210	0.764670	0.764670	0.764670	-	-	-	-	-	0.000000	0.000000	NMT	No	SMT reranked NMT
9	ORGANIZER	JPCzh-ja	2018/08/15 18:29:31	1963	0.761820	0.761820	0.761820	-	-	-	-	-	0.000000	0.000000	NMT	No	NMT with Attention
10	EHR	JPCzh-ja	2017/07/19 20:41:27	1414	0.761370	0.761370	0.761370	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	SMT reranked NMT (character based, by Moses and OpenNMT)
11	EHR	JPCzh-ja	2017/07/19 19:35:03	1409	0.757130	0.757130	0.757130	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Simple NMT (character based, by OpenNMT)
12	EHR	JPCzh-ja	2017/07/19 19:28:31	1408	0.756350	0.756350	0.756350	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	SMT reranked NMT (word based, by Moses and OpenNMT)
13	EHR	JPCzh-ja	2017/07/19 20:45:00	1415	0.755900	0.755900	0.755900	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Simple NMT (word based, by OpenNMT)
14	JAPIO	JPCzh-ja	2017/07/25 18:26:52	1458	0.754970	0.754970	0.754970	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	OpenNMT(dbrnn)
15	NTT	JPCzh-ja	2016/08/19 08:53:34	1199	0.752200	0.752200	0.752200	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Baseline NMT with attention over bidirectional LSTMs (by Harvard NMT)
16	JAPIO	JPCzh-ja	2016/08/19 08:26:57	1192	0.751200	0.751200	0.751200	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus
17	NTT	JPCzh-ja	2016/08/19 08:55:20	1200	0.749270	0.749270	0.749270	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT with pre-ordering and attention over bidirectional LSTMs (pre-ordering module is the same as the PBMT submission)
18	JAPIO	JPCzh-ja	2016/08/18 14:15:46	1180	0.748330	0.748330	0.748330	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus
19	ORGANIZER	JPCzh-ja	2016/11/16 11:19:58	1341	0.747240	0.747240	0.747240	-	-	-	-	0.000000	0.000000	0.000000	NMT	Yes	Online A (2016/11/14)
20	EHR	JPCzh-ja	2016/07/18 15:25:53	1007	0.745080	0.745080	0.745080	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Combination of word-based PBSMT and character-based PBSMT with DL=6.
21	Kyoto-U	JPCzh-ja	2015/09/02 09:25:04	864	0.744190	0.744190	0.744190	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking (only character-base model)
22	TOSHIBA	JPCzh-ja	2015/07/28 16:30:41	526	0.741990	0.741990	0.741990	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
23	TOSHIBA	JPCzh-ja	2015/07/23 14:43:30	504	0.740180	0.740180	0.740180	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	Combination of phrase-based SMT and SPE systems.
24	NICT-2	JPCzh-ja	2016/08/05 18:06:47	1100	0.739890	0.739890	0.739890	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
25	EHR	JPCzh-ja	2016/07/18 15:33:03	1009	0.735010	0.735010	0.735010	-	-	-	-	0.000000	0.000000	0.000000	SMT and RBMT	Yes	Combination of word-based PBSMT, character-based PBSMT and RBMT+PBSPE with DL=6.
26	JAPIO	JPCzh-ja	2016/10/27 13:01:42	1329	0.733300	0.733300	0.733300	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Phrase-based SMT with Preordering
27	NICT-2	JPCzh-ja	2016/08/04 17:34:38	1079	0.733020	0.733020	0.733020	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Phrase-based SMT with Preordering + Domain Adaptation
28	NTT	JPCzh-ja	2015/08/21 08:07:18	736	0.732450	0.732450	0.732450	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	A pre-ordering-based PBMT with patent-tuned dependency parsing and phrase table smoothing.
29	Kyoto-U	JPCzh-ja	2015/08/26 13:10:44	781	0.731420	0.731420	0.731420	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	Baseline w/o reranking
30	NTT	JPCzh-ja	2016/08/19 08:28:00	1193	0.730190	0.730190	0.730190	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	PBMT with pre-ordering on dependency structures
31	u-tkb	JPCzh-ja	2017/07/26 12:44:18	1468	0.729580	0.729580	0.729580	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT with SMT phrase translation (phrase extraction with branching entropy; attention over bidirectional LSTMs; by Harvard NMT)
32	ORGANIZER	JPCzh-ja	2015/05/14 17:55:51	430	0.729370	0.729370	0.729370	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Hierarchical Phrase-based SMT
33	ORGANIZER	JPCzh-ja	2016/07/15 11:22:35	998	0.728520	0.728520	0.728520	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Tree-to-String SMT (2016)
34	ORGANIZER	JPCzh-ja	2015/05/14 18:00:16	432	0.725920	0.725920	0.725920	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Tree-to-String SMT (2015)
35	NTT	JPCzh-ja	2015/08/28 09:53:24	811	0.723200	0.723200	0.723200	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	A pre-ordering-based PBMT with patent-tuned dependency parsing, learning-based pre-ordering, and phrase table smoothing.
36	ORGANIZER	JPCzh-ja	2015/05/14 17:58:14	431	0.723110	0.723110	0.723110	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Phrase-based SMT
37	bjtu_nlp	JPCzh-ja	2016/08/09 18:44:56	1128	0.721460	0.721460	0.721460	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
38	EHR	JPCzh-ja	2015/08/17 14:05:20	671	0.721400	0.721400	0.721400	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	System combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
39	NTT	JPCzh-ja	2016/08/19 08:26:18	1191	0.720260	0.720260	0.720260	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Baseline PBMT (Moses)
40	Sense	JPCzh-ja	2016/08/29 23:08:29	1292	0.719330	0.719330	0.719330	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Baseline-C50-PBMT
41	Sense	JPCzh-ja	2016/08/29 01:06:19	1281	0.718590	0.718590	0.718590	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Clustercat-C10-PBMT
42	Sense	JPCzh-ja	2016/08/29 09:55:33	1284	0.715450	0.715450	0.715450	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Baseline-C10-PBMT
43	Sense	JPCzh-ja	2016/08/30 07:37:39	1294	0.715290	0.715290	0.715290	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Clustercat-C50-PBMT
44	WASUIPS	JPCzh-ja	2015/09/01 14:16:16	853	0.709700	0.709700	0.709700	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Combining sampling-based alignment and bilingual hierarchical sub-sentential alignment methods.
45	EHR	JPCzh-ja	2015/08/30 15:22:25	830	0.706550	0.706550	0.706550	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Phrase based SMT with preordering
46	EHR	JPCzh-ja	2015/08/30 12:42:52	828	0.701880	0.701880	0.701880	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with user dictionary plus SPE
47	ORGANIZER	JPCzh-ja	2015/08/14 16:52:02	647	0.693840	0.693840	0.693840	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online A (2015)
48	ORGANIZER	JPCzh-ja	2016/07/26 11:18:45	1040	0.693720	0.693720	0.693720	-	-	-	-	0.000000	0.000000	0.000000	Other	Yes	Online A (2016)
49	WASUIPS	JPCzh-ja	2016/10/12 21:06:36	1326	0.686030	0.686030	0.686030	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Our improved system: train=100,000 tune=1,000 test=2,000 BLEU (our PC): 33.61. Using bilingual term extraction and re-tokenization for Chinese–Japanese.
50	WASUIPS	JPCzh-ja	2016/10/12 21:04:52	1325	0.679110	0.679110	0.679110	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Our baseline system: train=100,000 tune=1,000 test=2,000 BLEU (our PC): 32.29.
51	TOSHIBA	JPCzh-ja	2015/08/17 11:53:34	667	0.668780	0.668780	0.668780	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	RBMT	Yes	RBMT
52	EHR	JPCzh-ja	2018/05/04 14:17:39	1803	0.623300	0.623300	0.623300	-	-	-	-	0.000000	0.000000	0.000000	RBMT	Yes	RBMT system for WAT2015's submission
53	ORGANIZER	JPCzh-ja	2015/08/14 16:55:19	648	0.588380	0.588380	0.588380	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online B (2015)
54	ORGANIZER	JPCzh-ja	2015/08/25 11:42:02	759	0.557130	0.557130	0.557130	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	RBMT	No	RBMT A (2015)
55	ORGANIZER	JPCzh-ja	2015/08/25 11:53:50	760	0.502100	0.502100	0.502100	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	RBMT	No	RBMT B
56	ryan	JPCzh-ja	2019/07/25 22:12:26	2954	0.000000	0.000000	0.000000	-	-	-	-	-	-	-	NMT	No	Base Transformer
57	sarah	JPCzh-ja	2019/07/26 11:32:28	2976	0.000000	0.000000	0.000000	-	-	-	-	-	-	-	NMT	No	Transformer, ensemble of 4 models
58	KNU_Hyundai	JPCzh-ja	2019/07/27 08:29:23	3153	0.000000	0.000000	0.000000	-	-	-	-	-	-	-	NMT	Yes	Transformer(base) + Used ASPEC corpus with relative position, bt, multi source, r2l rerank, 5-model ensemble
59	goku20	JPCzh-ja	2020/09/21 11:54:22	4087	0.000000	0.000000	0.000000	-	-	-	-	-	-	-	NMT	No	mBART pre-training transformer, single model
60	goku20	JPCzh-ja	2020/09/22 00:04:26	4105	0.000000	0.000000	0.000000	-	-	-	-	-	-	-	NMT	No	mBART pre-training transformer, ensemble of 3 models
61	sakura	JPCzh-ja	2024/08/08 19:35:56	7264	0.000000	0.000000	0.000000	-	-	-	-	-	-	-	NMT	No	LLM: Rakuten/RakutenAI-7B-chat Fine-Tuned with JPC Corpus in six direction (En-Ja, Ja-En, Ko-Ja, Ja-Ko, Zh-Ja, Ja-Zh) - Best
62	sakura	JPCzh-ja	2024/08/09 00:24:08	7296	0.000000	0.000000	0.000000	-	-	-	-	-	-	-	NMT	No	LLM: Rakuten/RakutenAI-7B-chat Fine-Tuned with JPC Corpus in six direction (En-Ja, Ja-En, Ko-Ja, Ja-Ko, Zh-Ja, Ja-Zh)

Notice:

This table is sorted by the leftmost segmenters. You can change the segmenter used to sort by clicking each segmenter link.
Adequacy-Fluency Metrics (AMFM) is a two-dimensional automatic evaluation metric for machine translation, designed to operate at the sentence level. It is based on adequacy and fluency, to decouple semantic and syntactic components of the translation process to provide a balanced view on translation quality.
AMFM is calculated without tokenizers.
The detail of AMFM is shown on the following paper: "Adequacy–Fluency Metrics: Evaluating MT in the Continuous Space Model Framework" [pdf]. Invited Talk in WAT2015 also helps understanding [slide].

HUMAN (WAT2022)

Notice:

HUMAN (WAT2022) is the result of the Pairwise Crowdsourcing Evaluation on WAT2022.
HUMAN (WAT2022) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2021)

Notice:

HUMAN (WAT2021) is the result of the Pairwise Crowdsourcing Evaluation on WAT2021.
HUMAN (WAT2021) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2020)

Notice:

HUMAN (WAT2020) is the result of the Pairwise Crowdsourcing Evaluation on WAT2020.
HUMAN (WAT2020) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2019)

Notice:

HUMAN (WAT2019) is the result of the Pairwise Crowdsourcing Evaluation on WAT2019.
HUMAN (WAT2019) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2018)

Notice:

HUMAN (WAT2018) is the result of the Pairwise Crowdsourcing Evaluation on WAT2018.
HUMAN (WAT2018) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2017)

Notice:

HUMAN (WAT2017) is the result of the Pairwise Crowdsourcing Evaluation on WAT2017.
HUMAN (WAT2017) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2016)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	ORGANIZER	JPCzh-ja	2016/11/16 11:19:58	1341	54.250	NMT	Yes	Online A (2016/11/14)
2	NTT	JPCzh-ja	2016/08/19 08:55:20	1200	46.500	NMT	No	NMT with pre-ordering and attention over bidirectional LSTMs (pre-ordering module is the same as the PBMT submission)
3	JAPIO	JPCzh-ja	2016/08/19 08:26:57	1192	46.250	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus
4	JAPIO	JPCzh-ja	2016/08/18 14:15:46	1180	43.500	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus
5	NICT-2	JPCzh-ja	2016/08/05 18:06:47	1100	43.250	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
6	NTT	JPCzh-ja	2016/08/19 08:28:00	1193	39.250	SMT	No	PBMT with pre-ordering on dependency structures
7	EHR	JPCzh-ja	2016/07/18 15:25:53	1007	39.000	SMT	Yes	Combination of word-based PBSMT and character-based PBSMT with DL=6.
8	NICT-2	JPCzh-ja	2016/08/04 17:34:38	1079	36.750	SMT	No	Phrase-based SMT with Preordering + Domain Adaptation
9	EHR	JPCzh-ja	2016/07/18 15:33:03	1009	35.500	SMT and RBMT	Yes	Combination of word-based PBSMT, character-based PBSMT and RBMT+PBSPE with DL=6.
10	bjtu_nlp	JPCzh-ja	2016/08/09 18:44:56	1128	32.250	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
11	ORGANIZER	JPCzh-ja	2016/07/26 11:18:45	1040	-19.750	Other	Yes	Online A (2016)

Notice:

HUMAN (WAT2016) is the result of the Pairwise Crowdsourcing Evaluation on WAT2016.
HUMAN (WAT2016) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2015)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	Kyoto-U	JPCzh-ja	2015/09/02 09:25:04	864	27.500	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking (only character-base model)
2	TOSHIBA	JPCzh-ja	2015/07/28 16:30:41	526	24.250	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
3	EHR	JPCzh-ja	2015/08/17 14:05:20	671	22.000	SMT and RBMT	Yes	System combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
4	ORGANIZER	JPCzh-ja	2015/05/14 18:00:16	432	20.750	SMT	No	Tree-to-String SMT (2015)
5	NTT	JPCzh-ja	2015/08/21 08:07:18	736	16.250	SMT	No	A pre-ordering-based PBMT with patent-tuned dependency parsing and phrase table smoothing.
6	TOSHIBA	JPCzh-ja	2015/07/23 14:43:30	504	14.500	SMT and RBMT	Yes	Combination of phrase-based SMT and SPE systems.
7	Kyoto-U	JPCzh-ja	2015/08/26 13:10:44	781	14.500	EBMT	No	Baseline w/o reranking
8	EHR	JPCzh-ja	2015/08/30 12:42:52	828	8.250	SMT and RBMT	Yes	RBMT with user dictionary plus SPE
9	NTT	JPCzh-ja	2015/08/28 09:53:24	811	8.000	SMT	No	A pre-ordering-based PBMT with patent-tuned dependency parsing, learning-based pre-ordering, and phrase table smoothing.
10	ORGANIZER	JPCzh-ja	2015/08/14 16:52:02	647	-7.000	Other	Yes	Online A (2015)
11	WASUIPS	JPCzh-ja	2015/09/01 14:16:16	853	-12.000	SMT	No	Combining sampling-based alignment and bilingual hierarchical sub-sentential alignment methods.
12	ORGANIZER	JPCzh-ja	2015/08/25 11:42:02	759	-39.250	RBMT	No	RBMT A (2015)

Notice:

HUMAN (WAT2015) is the result of the Pairwise Crowdsourcing Evaluation on WAT2015.
HUMAN (WAT2015) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.
The detail of the evaluation can be found in the pdf document (PDF file).

HUMAN (WAT2014)

Notice:

HUMAN (WAT2014) is the result of the Pairwise Crowdsourcing Evaluation on WAT2014.
HUMAN (WAT2014) was evaluated by 3 different workers and the final decision is made by the voting of the judgements.
The detail of the evaluation can be found in the pdf document (PDF file).

EVALUATION RESULTS USAGE POLICY

When you use the WAT evaluation results for any purpose such as:
- writing technical papers,
- making presentations about your system,
- advertising your MT system to the customers,
you can use the information about translation directions, scores (including both automatic and human evaluations) and ranks of your system among others. You can also use the scores of the other systems, but you MUST anonymize the other system's names. In addition, you can show the links (URLs) to the WAT evaluation result pages.

NICT (National Institute of Information and Communications Technology)
Kyoto University
Last Modified: 2018-08-02

WAT The Workshop on Asian Translation Evaluation Results

BLEU

RIBES

AMFM

HUMAN (WAT2022)

HUMAN (WAT2021)

HUMAN (WAT2020)

HUMAN (WAT2019)

HUMAN (WAT2018)

HUMAN (WAT2017)

HUMAN (WAT2016)

HUMAN (WAT2015)

HUMAN (WAT2014)

EVALUATION RESULTS USAGE POLICY

WAT

The Workshop on Asian Translation

Evaluation Results