WAT

The Workshop on Asian Translation

Evaluation Results

BLEU

#	Team	Task	Date/Time	DataID	BLEU										Method	Other Resources	System Description
#	Team	Task	Date/Time	DataID	juman	kytea	mecab	moses- tokenizer	stanford- segmenter- ctb	stanford- segmenter- pku	indic- tokenizer	unuse	myseg	kmseg	Method	Other Resources	System Description
1	ORGANIZER	JPCzh-ja	2015/05/14 17:55:51	430	39.22	39.52	39.14	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Hierarchical Phrase-based SMT
2	ORGANIZER	JPCzh-ja	2015/05/14 17:58:14	431	38.34	38.51	38.22	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Phrase-based SMT
3	ORGANIZER	JPCzh-ja	2015/05/14 18:00:16	432	39.39	39.90	39.39	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Tree-to-String SMT (2015)
4	TOSHIBA	JPCzh-ja	2015/07/23 14:43:30	504	41.82	41.90	41.60	-	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	Combination of phrase-based SMT and SPE systems.
5	TOSHIBA	JPCzh-ja	2015/07/28 16:30:41	526	41.12	40.87	40.59	-	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
6	ORGANIZER	JPCzh-ja	2015/08/14 16:52:02	647	26.80	27.81	26.89	-	-	-	0.00	0.00	0.00	0.00	Other	Yes	Online A (2015)
7	ORGANIZER	JPCzh-ja	2015/08/14 16:55:19	648	12.33	12.72	12.44	-	-	-	0.00	0.00	0.00	0.00	Other	Yes	Online B (2015)
8	TOSHIBA	JPCzh-ja	2015/08/17 11:53:34	667	28.06	27.44	27.56	-	-	-	0.00	0.00	0.00	0.00	RBMT	Yes	RBMT
9	EHR	JPCzh-ja	2015/08/17 14:05:20	671	41.06	42.24	41.15	-	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	System combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
10	NTT	JPCzh-ja	2015/08/21 08:07:18	736	40.60	41.10	40.63	-	-	-	0.00	0.00	0.00	0.00	SMT	No	A pre-ordering-based PBMT with patent-tuned dependency parsing and phrase table smoothing.
11	ORGANIZER	JPCzh-ja	2015/08/25 11:42:02	759	10.49	10.72	10.35	-	-	-	0.00	0.00	0.00	0.00	RBMT	No	RBMT A (2015)
12	ORGANIZER	JPCzh-ja	2015/08/25 11:53:50	760	7.94	8.07	7.73	-	-	-	0.00	0.00	0.00	0.00	RBMT	No	RBMT B
13	Kyoto-U	JPCzh-ja	2015/08/26 13:10:44	781	37.87	38.62	37.71	-	-	-	0.00	0.00	0.00	0.00	EBMT	No	Baseline w/o reranking
14	NTT	JPCzh-ja	2015/08/28 09:53:24	811	39.77	40.08	39.88	-	-	-	0.00	0.00	0.00	0.00	SMT	No	A pre-ordering-based PBMT with patent-tuned dependency parsing, learning-based pre-ordering, and phrase table smoothing.
15	EHR	JPCzh-ja	2015/08/30 12:42:52	828	40.35	40.16	39.92	-	-	-	0.00	0.00	0.00	0.00	SMT and RBMT	Yes	RBMT with user dictionary plus SPE
16	EHR	JPCzh-ja	2015/08/30 15:22:25	830	40.70	41.49	40.79	-	-	-	0.00	0.00	0.00	0.00	SMT	Yes	Phrase based SMT with preordering
17	WASUIPS	JPCzh-ja	2015/09/01 14:16:16	853	33.48	34.55	33.55	-	-	-	0.00	0.00	0.00	0.00	SMT	No	Combining sampling-based alignment and bilingual hierarchical sub-sentential alignment methods.
18	Kyoto-U	JPCzh-ja	2015/09/02 09:25:04	864	41.35	41.92	41.16	-	-	-	0.00	0.00	0.00	0.00	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking (only character-base model)
19	ORGANIZER	JPCzh-ja	2016/07/15 11:22:35	998	39.07	39.45	38.95	-	-	-	-	0.00	0.00	0.00	SMT	No	Tree-to-String SMT (2016)
20	EHR	JPCzh-ja	2016/07/18 15:25:53	1007	40.95	41.20	40.51	-	-	-	-	0.00	0.00	0.00	SMT	Yes	Combination of word-based PBSMT and character-based PBSMT with DL=6.
21	EHR	JPCzh-ja	2016/07/18 15:33:03	1009	41.05	41.05	40.52	-	-	-	-	0.00	0.00	0.00	SMT and RBMT	Yes	Combination of word-based PBSMT, character-based PBSMT and RBMT+PBSPE with DL=6.
22	ORGANIZER	JPCzh-ja	2016/07/26 11:18:45	1040	26.99	27.91	27.02	-	-	-	-	0.00	0.00	0.00	Other	Yes	Online A (2016)
23	NICT-2	JPCzh-ja	2016/08/04 17:34:38	1079	41.09	41.27	41.24	-	-	-	-	0.00	0.00	0.00	SMT	No	Phrase-based SMT with Preordering + Domain Adaptation
24	NICT-2	JPCzh-ja	2016/08/05 18:06:47	1100	41.87	42.39	42.13	-	-	-	-	0.00	0.00	0.00	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
25	bjtu_nlp	JPCzh-ja	2016/08/09 18:44:56	1128	39.34	39.72	39.30	-	-	-	-	0.00	0.00	0.00	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
26	JAPIO	JPCzh-ja	2016/08/17 11:48:56	1161	58.66	59.19	58.63	-	-	-	-	0.00	0.00	0.00	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus including some sentences in testset
27	JAPIO	JPCzh-ja	2016/08/18 14:15:46	1180	43.87	44.47	43.66	-	-	-	-	0.00	0.00	0.00	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus
28	NTT	JPCzh-ja	2016/08/19 08:26:18	1191	39.03	39.17	38.99	-	-	-	-	0.00	0.00	0.00	SMT	No	Baseline PBMT (Moses)
29	JAPIO	JPCzh-ja	2016/08/19 08:26:57	1192	44.32	45.12	44.09	-	-	-	-	0.00	0.00	0.00	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus
30	NTT	JPCzh-ja	2016/08/19 08:28:00	1193	40.75	41.05	40.68	-	-	-	-	0.00	0.00	0.00	SMT	No	PBMT with pre-ordering on dependency structures
31	NTT	JPCzh-ja	2016/08/19 08:53:34	1199	44.99	45.84	45.02	-	-	-	-	0.00	0.00	0.00	NMT	No	Baseline NMT with attention over bidirectional LSTMs (by Harvard NMT)
32	NTT	JPCzh-ja	2016/08/19 08:55:20	1200	43.47	44.27	43.53	-	-	-	-	0.00	0.00	0.00	NMT	No	NMT with pre-ordering and attention over bidirectional LSTMs (pre-ordering module is the same as the PBMT submission)
33	Sense	JPCzh-ja	2016/08/29 01:06:19	1281	38.90	38.58	38.65	-	-	-	-	0.00	0.00	0.00	SMT	No	Clustercat-C10-PBMT
34	Sense	JPCzh-ja	2016/08/29 09:55:33	1284	38.75	38.32	38.38	-	-	-	-	0.00	0.00	0.00	SMT	No	Baseline-C10-PBMT
35	Sense	JPCzh-ja	2016/08/29 23:08:29	1292	38.71	38.35	38.38	-	-	-	-	0.00	0.00	0.00	SMT	No	Baseline-C50-PBMT
36	Sense	JPCzh-ja	2016/08/30 07:37:39	1294	38.71	38.51	38.54	-	-	-	-	0.00	0.00	0.00	SMT	No	Clustercat-C50-PBMT
37	WASUIPS	JPCzh-ja	2016/10/12 21:04:52	1325	29.38	30.82	29.66	-	-	-	-	0.00	0.00	0.00	SMT	No	Our baseline system: train=100,000 tune=1,000 test=2,000 BLEU (our PC): 32.29.
38	WASUIPS	JPCzh-ja	2016/10/12 21:06:36	1326	31.00	31.63	30.86	-	-	-	-	0.00	0.00	0.00	SMT	No	Our improved system: train=100,000 tune=1,000 test=2,000 BLEU (our PC): 33.61. Using bilingual term extraction and re-tokenization for Chinese–Japanese.
39	JAPIO	JPCzh-ja	2016/10/27 13:01:42	1329	39.29	40.67	39.51	-	-	-	-	0.00	0.00	0.00	SMT	No	Phrase-based SMT with Preordering
40	ORGANIZER	JPCzh-ja	2016/11/16 11:19:58	1341	42.66	43.76	42.95	-	-	-	-	0.00	0.00	0.00	NMT	Yes	Online A (2016/11/14)
41	EHR	JPCzh-ja	2017/07/19 19:28:31	1408	47.08	47.44	46.83	-	-	-	-	0.00	0.00	0.00	NMT	No	SMT reranked NMT (word based, by Moses and OpenNMT)
42	EHR	JPCzh-ja	2017/07/19 19:35:03	1409	45.27	45.87	45.24	-	-	-	-	0.00	0.00	0.00	NMT	No	Simple NMT (character based, by OpenNMT)
43	EHR	JPCzh-ja	2017/07/19 20:41:27	1414	46.52	47.17	46.35	-	-	-	-	0.00	0.00	0.00	NMT	No	SMT reranked NMT (character based, by Moses and OpenNMT)
44	EHR	JPCzh-ja	2017/07/19 20:45:00	1415	46.03	46.42	45.95	-	-	-	-	0.00	0.00	0.00	NMT	No	Simple NMT (word based, by OpenNMT)
45	JAPIO	JPCzh-ja	2017/07/25 12:22:07	1447	50.52	51.25	50.57	-	-	-	-	0.00	0.00	0.00	SMT	Yes	Phrase-based SMT with Preordering + Japio corpus
46	JAPIO	JPCzh-ja	2017/07/25 18:26:52	1458	45.07	45.79	45.10	-	-	-	-	0.00	0.00	0.00	NMT	No	OpenNMT(dbrnn)
47	u-tkb	JPCzh-ja	2017/07/26 12:44:18	1468	38.79	40.47	38.99	-	-	-	-	0.00	0.00	0.00	NMT	No	NMT with SMT phrase translation (phrase extraction with branching entropy; attention over bidirectional LSTMs; by Harvard NMT)
48	JAPIO	JPCzh-ja	2017/07/26 14:09:22	1482	49.51	50.00	49.48	-	-	-	-	0.00	0.00	0.00	NMT	Yes	OpenNMT(dbrnn) + JPC/Japio corpora + NMT/rule-based posteditor
49	JAPIO	JPCzh-ja	2017/07/26 14:21:18	1484	50.06	50.51	50.00	-	-	-	-	0.00	0.00	0.00	NMT	Yes	Combination of 3 NMT systems (OpenNMT(dbrnn) + JPC/Japio corpora + NMT/rule-based posteditor)
50	EHR	JPCzh-ja	2018/05/04 14:17:39	1803	15.77	16.12	15.53	-	-	-	-	0.00	0.00	0.00	RBMT	Yes	RBMT system for WAT2015's submission
51	ORGANIZER	JPCzh-ja	2018/08/15 18:29:31	1963	46.32	46.73	46.11	-	-	-	-	-	0.00	0.00	NMT	No	NMT with Attention
52	USTC	JPCzh-ja	2018/08/31 17:24:35	2206	48.37	49.78	48.57	-	-	-	-	-	0.00	0.00	NMT	No	tensor2tensor, 4 model average, r2l rerank
53	EHR	JPCzh-ja	2018/08/31 18:51:15	2210	48.10	48.51	47.96	-	-	-	-	-	0.00	0.00	NMT	No	SMT reranked NMT
54	ryan	JPCzh-ja	2019/07/25 22:12:26	2954	49.98	50.67	50.12	-	-	-	-	-	-	-	NMT	No	Base Transformer
55	sarah	JPCzh-ja	2019/07/26 11:32:28	2976	50.90	52.04	51.09	-	-	-	-	-	-	-	NMT	No	Transformer, ensemble of 4 models
56	KNU_Hyundai	JPCzh-ja	2019/07/27 08:29:23	3153	53.59	53.91	53.47	-	-	-	-	-	-	-	NMT	Yes	Transformer(base) + Used ASPEC corpus with relative position, bt, multi source, r2l rerank, 5-model ensemble
57	goku20	JPCzh-ja	2020/09/21 11:54:22	4087	49.07	50.04	49.34	-	-	-	-	-	-	-	NMT	No	mBART pre-training transformer, single model
58	goku20	JPCzh-ja	2020/09/22 00:04:26	4105	48.98	49.92	49.21	-	-	-	-	-	-	-	NMT	No	mBART pre-training transformer, ensemble of 3 models
59	tpt_wat	JPCzh-ja	2021/04/27 01:42:09	5690	49.42	50.46	49.55	-	-	-	-	-	-	-	NMT	No	Base Transformer model with shared vocab 8k size
60	Bering Lab	JPCzh-ja	2021/05/04 06:53:17	6171	52.99	53.57	53.03	-	-	-	-	-	-	-	NMT	Yes	Transformer Ensemble with additional crawled parallel corpus
61	sakura	JPCzh-ja	2024/08/08 19:35:56	7264	52.60	53.40	52.80	-	-	-	-	-	-	-	NMT	No	LLM: Rakuten/RakutenAI-7B-chat Fine-Tuned with JPC Corpus in six direction (En-Ja, Ja-En, Ko-Ja, Ja-Ko, Zh-Ja, Ja-Zh) - Best
62	sakura	JPCzh-ja	2024/08/09 00:24:08	7296	52.40	53.20	52.50	-	-	-	-	-	-	-	NMT	No	LLM: Rakuten/RakutenAI-7B-chat Fine-Tuned with JPC Corpus in six direction (En-Ja, Ja-En, Ko-Ja, Ja-Ko, Zh-Ja, Ja-Zh)

Notice:

This table is sorted by the leftmost segmenters. You can change the segmenter used to sort by clicking each segmenter link.

RIBES

#	Team	Task	Date/Time	DataID	RIBES										Method	Other Resources	System Description
#	Team	Task	Date/Time	DataID	juman	kytea	mecab	moses- tokenizer	stanford- segmenter- ctb	stanford- segmenter- pku	indic- tokenizer	unuse	myseg	kmseg	Method	Other Resources	System Description
1	ORGANIZER	JPCzh-ja	2015/05/14 17:55:51	430	0.806058	0.802059	0.804523	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Hierarchical Phrase-based SMT
2	ORGANIZER	JPCzh-ja	2015/05/14 17:58:14	431	0.782019	0.778921	0.781456	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Phrase-based SMT
3	ORGANIZER	JPCzh-ja	2015/05/14 18:00:16	432	0.814919	0.811350	0.813595	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Tree-to-String SMT (2015)
4	TOSHIBA	JPCzh-ja	2015/07/23 14:43:30	504	0.820568	0.813536	0.817614	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	Combination of phrase-based SMT and SPE systems.
5	TOSHIBA	JPCzh-ja	2015/07/28 16:30:41	526	0.822268	0.814249	0.818981	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
6	ORGANIZER	JPCzh-ja	2015/08/14 16:52:02	647	0.712242	0.707264	0.711273	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online A (2015)
7	ORGANIZER	JPCzh-ja	2015/08/14 16:55:19	648	0.648996	0.641255	0.648742	-	-	-	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online B (2015)
8	TOSHIBA	JPCzh-ja	2015/08/17 11:53:34	667	0.772054	0.758756	0.767076	-	-	-	0.000000	0.000000	0.000000	0.000000	RBMT	Yes	RBMT
9	EHR	JPCzh-ja	2015/08/17 14:05:20	671	0.826987	0.821983	0.825056	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	System combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
10	NTT	JPCzh-ja	2015/08/21 08:07:18	736	0.823436	0.820252	0.822026	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	A pre-ordering-based PBMT with patent-tuned dependency parsing and phrase table smoothing.
11	ORGANIZER	JPCzh-ja	2015/08/25 11:42:02	759	0.674060	0.664098	0.667349	-	-	-	0.000000	0.000000	0.000000	0.000000	RBMT	No	RBMT A (2015)
12	ORGANIZER	JPCzh-ja	2015/08/25 11:53:50	760	0.596200	0.581837	0.586941	-	-	-	0.000000	0.000000	0.000000	0.000000	RBMT	No	RBMT B
13	Kyoto-U	JPCzh-ja	2015/08/26 13:10:44	781	0.799730	0.797700	0.798979	-	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	Baseline w/o reranking
14	NTT	JPCzh-ja	2015/08/28 09:53:24	811	0.816288	0.811911	0.815543	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	A pre-ordering-based PBMT with patent-tuned dependency parsing, learning-based pre-ordering, and phrase table smoothing.
15	EHR	JPCzh-ja	2015/08/30 12:42:52	828	0.819516	0.812982	0.816743	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with user dictionary plus SPE
16	EHR	JPCzh-ja	2015/08/30 15:22:25	830	0.824264	0.821055	0.823192	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Phrase based SMT with preordering
17	WASUIPS	JPCzh-ja	2015/09/01 14:16:16	853	0.773985	0.771099	0.772202	-	-	-	0.000000	0.000000	0.000000	0.000000	SMT	No	Combining sampling-based alignment and bilingual hierarchical sub-sentential alignment methods.
18	Kyoto-U	JPCzh-ja	2015/09/02 09:25:04	864	0.828543	0.824199	0.827230	-	-	-	0.000000	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking (only character-base model)
19	ORGANIZER	JPCzh-ja	2016/07/15 11:22:35	998	0.813135	0.809893	0.811644	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Tree-to-String SMT (2016)
20	EHR	JPCzh-ja	2016/07/18 15:25:53	1007	0.828040	0.824502	0.826864	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Combination of word-based PBSMT and character-based PBSMT with DL=6.
21	EHR	JPCzh-ja	2016/07/18 15:33:03	1009	0.827048	0.821940	0.824852	-	-	-	-	0.000000	0.000000	0.000000	SMT and RBMT	Yes	Combination of word-based PBSMT, character-based PBSMT and RBMT+PBSPE with DL=6.
22	ORGANIZER	JPCzh-ja	2016/07/26 11:18:45	1040	0.707739	0.702718	0.706707	-	-	-	-	0.000000	0.000000	0.000000	Other	Yes	Online A (2016)
23	NICT-2	JPCzh-ja	2016/08/04 17:34:38	1079	0.827009	0.822664	0.825323	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Phrase-based SMT with Preordering + Domain Adaptation
24	NICT-2	JPCzh-ja	2016/08/05 18:06:47	1100	0.829640	0.826744	0.828107	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
25	bjtu_nlp	JPCzh-ja	2016/08/09 18:44:56	1128	0.835314	0.830505	0.833216	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
26	JAPIO	JPCzh-ja	2016/08/17 11:48:56	1161	0.868027	0.864893	0.866692	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus including some sentences in testset
27	JAPIO	JPCzh-ja	2016/08/18 14:15:46	1180	0.833586	0.829360	0.831534	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus
28	NTT	JPCzh-ja	2016/08/19 08:26:18	1191	0.805702	0.797991	0.802998	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Baseline PBMT (Moses)
29	JAPIO	JPCzh-ja	2016/08/19 08:26:57	1192	0.834959	0.830164	0.832955	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus
30	NTT	JPCzh-ja	2016/08/19 08:28:00	1193	0.825985	0.822125	0.824840	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	PBMT with pre-ordering on dependency structures
31	NTT	JPCzh-ja	2016/08/19 08:53:34	1199	0.853004	0.851859	0.852430	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Baseline NMT with attention over bidirectional LSTMs (by Harvard NMT)
32	NTT	JPCzh-ja	2016/08/19 08:55:20	1200	0.845271	0.843105	0.844968	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT with pre-ordering and attention over bidirectional LSTMs (pre-ordering module is the same as the PBMT submission)
33	Sense	JPCzh-ja	2016/08/29 01:06:19	1281	0.803155	0.794679	0.800689	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Clustercat-C10-PBMT
34	Sense	JPCzh-ja	2016/08/29 09:55:33	1284	0.804673	0.795449	0.801496	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Baseline-C10-PBMT
35	Sense	JPCzh-ja	2016/08/29 23:08:29	1292	0.802673	0.794127	0.799531	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Baseline-C50-PBMT
36	Sense	JPCzh-ja	2016/08/30 07:37:39	1294	0.804301	0.796349	0.801596	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Clustercat-C50-PBMT
37	WASUIPS	JPCzh-ja	2016/10/12 21:04:52	1325	0.751847	0.748474	0.750678	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Our baseline system: train=100,000 tune=1,000 test=2,000 BLEU (our PC): 32.29.
38	WASUIPS	JPCzh-ja	2016/10/12 21:06:36	1326	0.754626	0.751804	0.753376	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Our improved system: train=100,000 tune=1,000 test=2,000 BLEU (our PC): 33.61. Using bilingual term extraction and re-tokenization for Chinese–Japanese.
39	JAPIO	JPCzh-ja	2016/10/27 13:01:42	1329	0.820339	0.817352	0.819850	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Phrase-based SMT with Preordering
40	ORGANIZER	JPCzh-ja	2016/11/16 11:19:58	1341	0.845858	0.844918	0.845794	-	-	-	-	0.000000	0.000000	0.000000	NMT	Yes	Online A (2016/11/14)
41	EHR	JPCzh-ja	2017/07/19 19:28:31	1408	0.859070	0.856376	0.858888	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	SMT reranked NMT (word based, by Moses and OpenNMT)
42	EHR	JPCzh-ja	2017/07/19 19:35:03	1409	0.854447	0.852615	0.853226	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Simple NMT (character based, by OpenNMT)
43	EHR	JPCzh-ja	2017/07/19 20:41:27	1414	0.859619	0.856784	0.858353	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	SMT reranked NMT (character based, by Moses and OpenNMT)
44	EHR	JPCzh-ja	2017/07/19 20:45:00	1415	0.858591	0.855917	0.858511	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Simple NMT (word based, by OpenNMT)
45	JAPIO	JPCzh-ja	2017/07/25 12:22:07	1447	0.847793	0.843774	0.846081	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + Japio corpus
46	JAPIO	JPCzh-ja	2017/07/25 18:26:52	1458	0.859883	0.857056	0.859411	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	OpenNMT(dbrnn)
47	u-tkb	JPCzh-ja	2017/07/26 12:44:18	1468	0.832144	0.833610	0.831209	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT with SMT phrase translation (phrase extraction with branching entropy; attention over bidirectional LSTMs; by Harvard NMT)
48	JAPIO	JPCzh-ja	2017/07/26 14:09:22	1482	0.872625	0.870537	0.872038	-	-	-	-	0.000000	0.000000	0.000000	NMT	Yes	OpenNMT(dbrnn) + JPC/Japio corpora + NMT/rule-based posteditor
49	JAPIO	JPCzh-ja	2017/07/26 14:21:18	1484	0.875398	0.873390	0.874822	-	-	-	-	0.000000	0.000000	0.000000	NMT	Yes	Combination of 3 NMT systems (OpenNMT(dbrnn) + JPC/Japio corpora + NMT/rule-based posteditor)
50	EHR	JPCzh-ja	2018/05/04 14:17:39	1803	0.721715	0.710085	0.716303	-	-	-	-	0.000000	0.000000	0.000000	RBMT	Yes	RBMT system for WAT2015's submission
51	ORGANIZER	JPCzh-ja	2018/08/15 18:29:31	1963	0.857318	0.855085	0.856442	-	-	-	-	-	0.000000	0.000000	NMT	No	NMT with Attention
52	USTC	JPCzh-ja	2018/08/31 17:24:35	2206	0.866232	0.864284	0.865423	-	-	-	-	-	0.000000	0.000000	NMT	No	tensor2tensor, 4 model average, r2l rerank
53	EHR	JPCzh-ja	2018/08/31 18:51:15	2210	0.858259	0.855649	0.858142	-	-	-	-	-	0.000000	0.000000	NMT	No	SMT reranked NMT
54	ryan	JPCzh-ja	2019/07/25 22:12:26	2954	0.869890	0.867919	0.868936	-	-	-	-	-	-	-	NMT	No	Base Transformer
55	sarah	JPCzh-ja	2019/07/26 11:32:28	2976	0.869159	0.867133	0.868496	-	-	-	-	-	-	-	NMT	No	Transformer, ensemble of 4 models
56	KNU_Hyundai	JPCzh-ja	2019/07/27 08:29:23	3153	0.880251	0.879452	0.879471	-	-	-	-	-	-	-	NMT	Yes	Transformer(base) + Used ASPEC corpus with relative position, bt, multi source, r2l rerank, 5-model ensemble
57	goku20	JPCzh-ja	2020/09/21 11:54:22	4087	0.865852	0.863774	0.865703	-	-	-	-	-	-	-	NMT	No	mBART pre-training transformer, single model
58	goku20	JPCzh-ja	2020/09/22 00:04:26	4105	0.865111	0.863206	0.864862	-	-	-	-	-	-	-	NMT	No	mBART pre-training transformer, ensemble of 3 models
59	tpt_wat	JPCzh-ja	2021/04/27 01:42:09	5690	0.869034	0.867678	0.868716	-	-	-	-	-	-	-	NMT	No	Base Transformer model with shared vocab 8k size
60	Bering Lab	JPCzh-ja	2021/05/04 06:53:17	6171	0.883744	0.881710	0.883204	-	-	-	-	-	-	-	NMT	Yes	Transformer Ensemble with additional crawled parallel corpus
61	sakura	JPCzh-ja	2024/08/08 19:35:56	7264	0.877180	0.875374	0.876600	-	-	-	-	-	-	-	NMT	No	LLM: Rakuten/RakutenAI-7B-chat Fine-Tuned with JPC Corpus in six direction (En-Ja, Ja-En, Ko-Ja, Ja-Ko, Zh-Ja, Ja-Zh) - Best
62	sakura	JPCzh-ja	2024/08/09 00:24:08	7296	0.878851	0.876452	0.878847	-	-	-	-	-	-	-	NMT	No	LLM: Rakuten/RakutenAI-7B-chat Fine-Tuned with JPC Corpus in six direction (En-Ja, Ja-En, Ko-Ja, Ja-Ko, Zh-Ja, Ja-Zh)

Notice:

This table is sorted by the leftmost segmenters. You can change the segmenter used to sort by clicking each segmenter link.

AMFM

#	Team	Task	Date/Time	DataID	AMFM										Method	Other Resources	System Description
#	Team	Task	Date/Time	DataID	unuse	unuse	unuse	unuse	unuse	unuse	unuse	unuse	unuse	unuse	Method	Other Resources	System Description
1	ORGANIZER	JPCzh-ja	2016/07/15 11:22:35	998	0.728520	0.728520	0.728520	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Tree-to-String SMT (2016)
2	EHR	JPCzh-ja	2016/07/18 15:25:53	1007	0.745080	0.745080	0.745080	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Combination of word-based PBSMT and character-based PBSMT with DL=6.
3	EHR	JPCzh-ja	2016/07/18 15:33:03	1009	0.735010	0.735010	0.735010	-	-	-	-	0.000000	0.000000	0.000000	SMT and RBMT	Yes	Combination of word-based PBSMT, character-based PBSMT and RBMT+PBSPE with DL=6.
4	ORGANIZER	JPCzh-ja	2016/07/26 11:18:45	1040	0.693720	0.693720	0.693720	-	-	-	-	0.000000	0.000000	0.000000	Other	Yes	Online A (2016)
5	NICT-2	JPCzh-ja	2016/08/04 17:34:38	1079	0.733020	0.733020	0.733020	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Phrase-based SMT with Preordering + Domain Adaptation
6	NICT-2	JPCzh-ja	2016/08/05 18:06:47	1100	0.739890	0.739890	0.739890	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
7	bjtu_nlp	JPCzh-ja	2016/08/09 18:44:56	1128	0.721460	0.721460	0.721460	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
8	JAPIO	JPCzh-ja	2016/08/17 11:48:56	1161	0.808090	0.808090	0.808090	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus including some sentences in testset
9	JAPIO	JPCzh-ja	2016/08/18 14:15:46	1180	0.748330	0.748330	0.748330	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus
10	NTT	JPCzh-ja	2016/08/19 08:26:18	1191	0.720260	0.720260	0.720260	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Baseline PBMT (Moses)
11	JAPIO	JPCzh-ja	2016/08/19 08:26:57	1192	0.751200	0.751200	0.751200	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus
12	NTT	JPCzh-ja	2016/08/19 08:28:00	1193	0.730190	0.730190	0.730190	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	PBMT with pre-ordering on dependency structures
13	NTT	JPCzh-ja	2016/08/19 08:53:34	1199	0.752200	0.752200	0.752200	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Baseline NMT with attention over bidirectional LSTMs (by Harvard NMT)
14	NTT	JPCzh-ja	2016/08/19 08:55:20	1200	0.749270	0.749270	0.749270	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT with pre-ordering and attention over bidirectional LSTMs (pre-ordering module is the same as the PBMT submission)
15	Sense	JPCzh-ja	2016/08/29 01:06:19	1281	0.718590	0.718590	0.718590	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Clustercat-C10-PBMT
16	Sense	JPCzh-ja	2016/08/29 09:55:33	1284	0.715450	0.715450	0.715450	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Baseline-C10-PBMT
17	Sense	JPCzh-ja	2016/08/29 23:08:29	1292	0.719330	0.719330	0.719330	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Baseline-C50-PBMT
18	Sense	JPCzh-ja	2016/08/30 07:37:39	1294	0.715290	0.715290	0.715290	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Clustercat-C50-PBMT
19	WASUIPS	JPCzh-ja	2016/10/12 21:04:52	1325	0.679110	0.679110	0.679110	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Our baseline system: train=100,000 tune=1,000 test=2,000 BLEU (our PC): 32.29.
20	WASUIPS	JPCzh-ja	2016/10/12 21:06:36	1326	0.686030	0.686030	0.686030	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Our improved system: train=100,000 tune=1,000 test=2,000 BLEU (our PC): 33.61. Using bilingual term extraction and re-tokenization for Chinese–Japanese.
21	JAPIO	JPCzh-ja	2016/10/27 13:01:42	1329	0.733300	0.733300	0.733300	-	-	-	-	0.000000	0.000000	0.000000	SMT	No	Phrase-based SMT with Preordering
22	ORGANIZER	JPCzh-ja	2016/11/16 11:19:58	1341	0.747240	0.747240	0.747240	-	-	-	-	0.000000	0.000000	0.000000	NMT	Yes	Online A (2016/11/14)
23	EHR	JPCzh-ja	2017/07/19 19:28:31	1408	0.756350	0.756350	0.756350	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	SMT reranked NMT (word based, by Moses and OpenNMT)
24	EHR	JPCzh-ja	2017/07/19 19:35:03	1409	0.757130	0.757130	0.757130	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Simple NMT (character based, by OpenNMT)
25	EHR	JPCzh-ja	2017/07/19 20:41:27	1414	0.761370	0.761370	0.761370	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	SMT reranked NMT (character based, by Moses and OpenNMT)
26	EHR	JPCzh-ja	2017/07/19 20:45:00	1415	0.755900	0.755900	0.755900	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	Simple NMT (word based, by OpenNMT)
27	JAPIO	JPCzh-ja	2017/07/25 12:22:07	1447	0.774660	0.774660	0.774660	-	-	-	-	0.000000	0.000000	0.000000	SMT	Yes	Phrase-based SMT with Preordering + Japio corpus
28	JAPIO	JPCzh-ja	2017/07/25 18:26:52	1458	0.754970	0.754970	0.754970	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	OpenNMT(dbrnn)
29	u-tkb	JPCzh-ja	2017/07/26 12:44:18	1468	0.729580	0.729580	0.729580	-	-	-	-	0.000000	0.000000	0.000000	NMT	No	NMT with SMT phrase translation (phrase extraction with branching entropy; attention over bidirectional LSTMs; by Harvard NMT)
30	JAPIO	JPCzh-ja	2017/07/26 14:09:22	1482	0.777460	0.777460	0.777460	-	-	-	-	0.000000	0.000000	0.000000	NMT	Yes	OpenNMT(dbrnn) + JPC/Japio corpora + NMT/rule-based posteditor
31	JAPIO	JPCzh-ja	2017/07/26 14:21:18	1484	0.779420	0.779420	0.779420	-	-	-	-	0.000000	0.000000	0.000000	NMT	Yes	Combination of 3 NMT systems (OpenNMT(dbrnn) + JPC/Japio corpora + NMT/rule-based posteditor)
32	EHR	JPCzh-ja	2018/05/04 14:17:39	1803	0.623300	0.623300	0.623300	-	-	-	-	0.000000	0.000000	0.000000	RBMT	Yes	RBMT system for WAT2015's submission
33	ORGANIZER	JPCzh-ja	2018/08/15 18:29:31	1963	0.761820	0.761820	0.761820	-	-	-	-	-	0.000000	0.000000	NMT	No	NMT with Attention
34	USTC	JPCzh-ja	2018/08/31 17:24:35	2206	0.771310	0.771310	0.771310	-	-	-	-	-	0.000000	0.000000	NMT	No	tensor2tensor, 4 model average, r2l rerank
35	EHR	JPCzh-ja	2018/08/31 18:51:15	2210	0.764670	0.764670	0.764670	-	-	-	-	-	0.000000	0.000000	NMT	No	SMT reranked NMT
36	ryan	JPCzh-ja	2019/07/25 22:12:26	2954	0.000000	0.000000	0.000000	-	-	-	-	-	-	-	NMT	No	Base Transformer
37	sarah	JPCzh-ja	2019/07/26 11:32:28	2976	0.000000	0.000000	0.000000	-	-	-	-	-	-	-	NMT	No	Transformer, ensemble of 4 models
38	KNU_Hyundai	JPCzh-ja	2019/07/27 08:29:23	3153	0.000000	0.000000	0.000000	-	-	-	-	-	-	-	NMT	Yes	Transformer(base) + Used ASPEC corpus with relative position, bt, multi source, r2l rerank, 5-model ensemble
39	goku20	JPCzh-ja	2020/09/21 11:54:22	4087	0.000000	0.000000	0.000000	-	-	-	-	-	-	-	NMT	No	mBART pre-training transformer, single model
40	goku20	JPCzh-ja	2020/09/22 00:04:26	4105	0.000000	0.000000	0.000000	-	-	-	-	-	-	-	NMT	No	mBART pre-training transformer, ensemble of 3 models
41	tpt_wat	JPCzh-ja	2021/04/27 01:42:09	5690	0.886918	0.886918	0.886918	-	-	-	-	-	-	-	NMT	No	Base Transformer model with shared vocab 8k size
42	Bering Lab	JPCzh-ja	2021/05/04 06:53:17	6171	0.896405	0.896405	0.896405	-	-	-	-	-	-	-	NMT	Yes	Transformer Ensemble with additional crawled parallel corpus
43	sakura	JPCzh-ja	2024/08/08 19:35:56	7264	0.000000	0.000000	0.000000	-	-	-	-	-	-	-	NMT	No	LLM: Rakuten/RakutenAI-7B-chat Fine-Tuned with JPC Corpus in six direction (En-Ja, Ja-En, Ko-Ja, Ja-Ko, Zh-Ja, Ja-Zh) - Best
44	sakura	JPCzh-ja	2024/08/09 00:24:08	7296	0.000000	0.000000	0.000000	-	-	-	-	-	-	-	NMT	No	LLM: Rakuten/RakutenAI-7B-chat Fine-Tuned with JPC Corpus in six direction (En-Ja, Ja-En, Ko-Ja, Ja-Ko, Zh-Ja, Ja-Zh)
45	ORGANIZER	JPCzh-ja	2015/05/14 17:55:51	430	0.729370	0.729370	0.729370	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Hierarchical Phrase-based SMT
46	ORGANIZER	JPCzh-ja	2015/05/14 17:58:14	431	0.723110	0.723110	0.723110	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Phrase-based SMT
47	ORGANIZER	JPCzh-ja	2015/05/14 18:00:16	432	0.725920	0.725920	0.725920	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Tree-to-String SMT (2015)
48	TOSHIBA	JPCzh-ja	2015/07/23 14:43:30	504	0.740180	0.740180	0.740180	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	Combination of phrase-based SMT and SPE systems.
49	TOSHIBA	JPCzh-ja	2015/07/28 16:30:41	526	0.741990	0.741990	0.741990	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
50	ORGANIZER	JPCzh-ja	2015/08/14 16:52:02	647	0.693840	0.693840	0.693840	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online A (2015)
51	ORGANIZER	JPCzh-ja	2015/08/14 16:55:19	648	0.588380	0.588380	0.588380	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	Other	Yes	Online B (2015)
52	TOSHIBA	JPCzh-ja	2015/08/17 11:53:34	667	0.668780	0.668780	0.668780	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	RBMT	Yes	RBMT
53	EHR	JPCzh-ja	2015/08/17 14:05:20	671	0.721400	0.721400	0.721400	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	System combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
54	NTT	JPCzh-ja	2015/08/21 08:07:18	736	0.732450	0.732450	0.732450	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	A pre-ordering-based PBMT with patent-tuned dependency parsing and phrase table smoothing.
55	ORGANIZER	JPCzh-ja	2015/08/25 11:42:02	759	0.557130	0.557130	0.557130	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	RBMT	No	RBMT A (2015)
56	ORGANIZER	JPCzh-ja	2015/08/25 11:53:50	760	0.502100	0.502100	0.502100	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	RBMT	No	RBMT B
57	Kyoto-U	JPCzh-ja	2015/08/26 13:10:44	781	0.731420	0.731420	0.731420	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	Baseline w/o reranking
58	NTT	JPCzh-ja	2015/08/28 09:53:24	811	0.723200	0.723200	0.723200	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	A pre-ordering-based PBMT with patent-tuned dependency parsing, learning-based pre-ordering, and phrase table smoothing.
59	EHR	JPCzh-ja	2015/08/30 12:42:52	828	0.701880	0.701880	0.701880	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT and RBMT	Yes	RBMT with user dictionary plus SPE
60	EHR	JPCzh-ja	2015/08/30 15:22:25	830	0.706550	0.706550	0.706550	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	Yes	Phrase based SMT with preordering
61	WASUIPS	JPCzh-ja	2015/09/01 14:16:16	853	0.709700	0.709700	0.709700	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	SMT	No	Combining sampling-based alignment and bilingual hierarchical sub-sentential alignment methods.
62	Kyoto-U	JPCzh-ja	2015/09/02 09:25:04	864	0.744190	0.744190	0.744190	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	0.000000	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking (only character-base model)

Notice:

This table is sorted by the leftmost segmenters. You can change the segmenter used to sort by clicking each segmenter link.
Adequacy-Fluency Metrics (AMFM) is a two-dimensional automatic evaluation metric for machine translation, designed to operate at the sentence level. It is based on adequacy and fluency, to decouple semantic and syntactic components of the translation process to provide a balanced view on translation quality.
AMFM is calculated without tokenizers.
The detail of AMFM is shown on the following paper: "Adequacy–Fluency Metrics: Evaluating MT in the Continuous Space Model Framework" [pdf]. Invited Talk in WAT2015 also helps understanding [slide].

HUMAN (WAT2022)

Notice:

HUMAN (WAT2022) is the result of the Pairwise Crowdsourcing Evaluation on WAT2022.
HUMAN (WAT2022) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2021)

Notice:

HUMAN (WAT2021) is the result of the Pairwise Crowdsourcing Evaluation on WAT2021.
HUMAN (WAT2021) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2020)

Notice:

HUMAN (WAT2020) is the result of the Pairwise Crowdsourcing Evaluation on WAT2020.
HUMAN (WAT2020) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2019)

Notice:

HUMAN (WAT2019) is the result of the Pairwise Crowdsourcing Evaluation on WAT2019.
HUMAN (WAT2019) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2018)

Notice:

HUMAN (WAT2018) is the result of the Pairwise Crowdsourcing Evaluation on WAT2018.
HUMAN (WAT2018) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2017)

Notice:

HUMAN (WAT2017) is the result of the Pairwise Crowdsourcing Evaluation on WAT2017.
HUMAN (WAT2017) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2016)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	ORGANIZER	JPCzh-ja	2016/11/16 11:19:58	1341	54.250	NMT	Yes	Online A (2016/11/14)
2	NTT	JPCzh-ja	2016/08/19 08:55:20	1200	46.500	NMT	No	NMT with pre-ordering and attention over bidirectional LSTMs (pre-ordering module is the same as the PBMT submission)
3	JAPIO	JPCzh-ja	2016/08/19 08:26:57	1192	46.250	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus
4	JAPIO	JPCzh-ja	2016/08/18 14:15:46	1180	43.500	SMT	Yes	Phrase-based SMT with Preordering + JAPIO corpus
5	NICT-2	JPCzh-ja	2016/08/05 18:06:47	1100	43.250	SMT	Yes	Phrase-based SMT with Preordering + Domain Adaptation (JPC and ASPEC) + Google 5-gram LM
6	NTT	JPCzh-ja	2016/08/19 08:28:00	1193	39.250	SMT	No	PBMT with pre-ordering on dependency structures
7	EHR	JPCzh-ja	2016/07/18 15:25:53	1007	39.000	SMT	Yes	Combination of word-based PBSMT and character-based PBSMT with DL=6.
8	NICT-2	JPCzh-ja	2016/08/04 17:34:38	1079	36.750	SMT	No	Phrase-based SMT with Preordering + Domain Adaptation
9	EHR	JPCzh-ja	2016/07/18 15:33:03	1009	35.500	SMT and RBMT	Yes	Combination of word-based PBSMT, character-based PBSMT and RBMT+PBSPE with DL=6.
10	bjtu_nlp	JPCzh-ja	2016/08/09 18:44:56	1128	32.250	NMT	No	RNN Encoder-Decoder with attention mechanism, single model
11	ORGANIZER	JPCzh-ja	2016/07/26 11:18:45	1040	-19.750	Other	Yes	Online A (2016)

Notice:

HUMAN (WAT2016) is the result of the Pairwise Crowdsourcing Evaluation on WAT2016.
HUMAN (WAT2016) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.

HUMAN (WAT2015)

#	Team	Task	Date/Time	DataID	HUMAN	Method	Other Resources	System Description
1	Kyoto-U	JPCzh-ja	2015/09/02 09:25:04	864	27.500	EBMT	No	KyotoEBMT system with bilingual RNNLM reranking (only character-base model)
2	TOSHIBA	JPCzh-ja	2015/07/28 16:30:41	526	24.250	SMT and RBMT	Yes	RBMT with SPE(Statistical Post Editing) system
3	EHR	JPCzh-ja	2015/08/17 14:05:20	671	22.000	SMT and RBMT	Yes	System combination of RBMT with user dictionary plus SPE and phrase based SMT with preordering. Candidate selection by language model score.
4	ORGANIZER	JPCzh-ja	2015/05/14 18:00:16	432	20.750	SMT	No	Tree-to-String SMT (2015)
5	NTT	JPCzh-ja	2015/08/21 08:07:18	736	16.250	SMT	No	A pre-ordering-based PBMT with patent-tuned dependency parsing and phrase table smoothing.
6	TOSHIBA	JPCzh-ja	2015/07/23 14:43:30	504	14.500	SMT and RBMT	Yes	Combination of phrase-based SMT and SPE systems.
7	Kyoto-U	JPCzh-ja	2015/08/26 13:10:44	781	14.500	EBMT	No	Baseline w/o reranking
8	EHR	JPCzh-ja	2015/08/30 12:42:52	828	8.250	SMT and RBMT	Yes	RBMT with user dictionary plus SPE
9	NTT	JPCzh-ja	2015/08/28 09:53:24	811	8.000	SMT	No	A pre-ordering-based PBMT with patent-tuned dependency parsing, learning-based pre-ordering, and phrase table smoothing.
10	ORGANIZER	JPCzh-ja	2015/08/14 16:52:02	647	-7.000	Other	Yes	Online A (2015)
11	WASUIPS	JPCzh-ja	2015/09/01 14:16:16	853	-12.000	SMT	No	Combining sampling-based alignment and bilingual hierarchical sub-sentential alignment methods.
12	ORGANIZER	JPCzh-ja	2015/08/25 11:42:02	759	-39.250	RBMT	No	RBMT A (2015)

Notice:

HUMAN (WAT2015) is the result of the Pairwise Crowdsourcing Evaluation on WAT2015.
HUMAN (WAT2015) was evaluated by 5 different workers and the final decision is made by the voting of the judgements.
The detail of the evaluation can be found in the pdf document (PDF file).

HUMAN (WAT2014)

Notice:

HUMAN (WAT2014) is the result of the Pairwise Crowdsourcing Evaluation on WAT2014.
HUMAN (WAT2014) was evaluated by 3 different workers and the final decision is made by the voting of the judgements.
The detail of the evaluation can be found in the pdf document (PDF file).

EVALUATION RESULTS USAGE POLICY

When you use the WAT evaluation results for any purpose such as:
- writing technical papers,
- making presentations about your system,
- advertising your MT system to the customers,
you can use the information about translation directions, scores (including both automatic and human evaluations) and ranks of your system among others. You can also use the scores of the other systems, but you MUST anonymize the other system's names. In addition, you can show the links (URLs) to the WAT evaluation result pages.

NICT (National Institute of Information and Communications Technology)
Kyoto University
Last Modified: 2018-08-02

WAT The Workshop on Asian Translation Evaluation Results

BLEU

RIBES

AMFM

HUMAN (WAT2022)

HUMAN (WAT2021)

HUMAN (WAT2020)

HUMAN (WAT2019)

HUMAN (WAT2018)

HUMAN (WAT2017)

HUMAN (WAT2016)

HUMAN (WAT2015)

HUMAN (WAT2014)

EVALUATION RESULTS USAGE POLICY

WAT

The Workshop on Asian Translation

Evaluation Results