NICT_LOGO.JPG KYOTO-U_LOGO.JPG

WAT

The Workshop on Asian Translation
Evaluation Results

[HOME] | [EVALUATION RESULTS] | [AUTOMATIC EVALUATION PROCEDURES] | [EVALUATION RESULTS USAGE POLICY] | [TECHNICAL COLLABORATOR]

EVALUATION RESULTS

+ Document-level translation tasks:
[ASPEC]
Task: ASPECen-jaEvaluation Result
Task: ASPECja-enEvaluation Result
Task: ASPECzh-jaEvaluation Result
Task: ASPECja-zhEvaluation Result
[BSD Corpus (Document-level Business Scene Dialogue Translation)]**
Task: BSDen-jaEvaluation Result
Task: BSDja-enEvaluation Result
[JIJI Corpus]
Task: JIJIen-jaEvaluation Result
Task: JIJIja-enEvaluation Result
Task: JIJICen-jaEvaluation Result
Task: JIJICja-enEvaluation Result
[NICT-SAP (Multilingual Multi-domain evaluation task (IT tasks))]
Task: SOFTWAREen-hiEvaluation Result
Task: SOFTWAREhi-enEvaluation Result
Task: SOFTWAREen-idEvaluation Result
Task: SOFTWAREid-enEvaluation Result
Task: SOFTWAREen-msEvaluation Result
Task: SOFTWAREms-enEvaluation Result
Task: SOFTWAREen-thEvaluation Result
Task: SOFTWAREth-enEvaluation Result
[NICT-SAP (Multilingual Multi-domain evaluation task (Wikinews tasks) (Asian Language Treebank (ALT)))]**
Task: ALT20en-hiEvaluation Result
Task: ALT20hi-enEvaluation Result
Task: ALT20en-idEvaluation Result
Task: ALT20id-enEvaluation Result
Task: ALT20en-msEvaluation Result
Task: ALT20ms-enEvaluation Result
Task: ALT20en-thEvaluation Result
Task: ALT20th-enEvaluation Result
------------------------------
+ Multimodal translation tasks:
[Hindi Visual Genome (IITB Corpus)]
Task: HINDENMMEVTEXT21en-hiEvaluation Result
Task: HINDENMMEVHI21en-hiEvaluation Result
Task: HINDENMMEVMM21en-hiEvaluation Result
Task: HINDENMMCHTEXT21en-hiEvaluation Result
Task: HINDENMMCHHI21en-hiEvaluation Result
Task: HINDENMMCHMM21en-hiEvaluation Result
[Malayalam Visual Genome]**
Task: HINDENMMEVTEXT21en-mlEvaluation Result
Task: HINDENMMEVHI21en-mlEvaluation Result
Task: HINDENMMEVMM21en-mlEvaluation Result
Task: HINDENMMCHTEXT21en-mlEvaluation Result
Task: HINDENMMCHHI21en-mlEvaluation Result
Task: HINDENMMCHMM21en-mlEvaluation Result
[Flickr30kEnt-JP (Multimodal task)]
Task: MMTen-jaEvaluation Result
Task: MMTja-enEvaluation Result
[Ambiguous MS COCO]**
Task: MSCOCOen-jaEvaluation Result
Task: MSCOCOja-enEvaluation Result
[ArEnMulti30K]
TBA
------------------------------
+ Indic tasks:
[MultiIndicMT (Indic Languages Multilingual Parallel Corpus 2021)]**
Task: INDIC21en-bnEvaluation Result
Task: INDIC21bn-enEvaluation Result
Task: INDIC21en-guEvaluation Result
Task: INDIC21gu-enEvaluation Result
Task: INDIC21en-hiEvaluation Result
Task: INDIC21hi-enEvaluation Result
Task: INDIC21en-knEvaluation Result
Task: INDIC21kn-enEvaluation Result
Task: INDIC21en-mlEvaluation Result
Task: INDIC21ml-enEvaluation Result
Task: INDIC21en-mrEvaluation Result
Task: INDIC21mr-enEvaluation Result
Task: INDIC21en-orEvaluation Result
Task: INDIC21or-enEvaluation Result
Task: INDIC21en-paEvaluation Result
Task: INDIC21pa-enEvaluation Result
Task: INDIC21en-taEvaluation Result
Task: INDIC21ta-enEvaluation Result
Task: INDIC21en-teEvaluation Result
Task: INDIC21te-enEvaluation Result
------------------------------
+ ALT+ tasks:
[+UCSY]
Task: ALT2en-myEvaluation Result
Task: ALT2my-enEvaluation Result
------------------------------
+ Patent task:
[JPC2 (JPO Patent Corpus2 (2018 -))]* **
Task: JPCNzh-jaEvaluation Result
Task: JPCNja-zhEvaluation Result
Task: JPCNko-jaEvaluation Result
Task: JPCNja-koEvaluation Result
Task: JPCNen-jaEvaluation Result
Task: JPCNja-enEvaluation Result
Task: JPCN1zh-jaEvaluation Result
Task: JPCN1ja-zhEvaluation Result
Task: JPCN1ko-jaEvaluation Result
Task: JPCN1ja-koEvaluation Result
Task: JPCN1en-jaEvaluation Result
Task: JPCN1ja-enEvaluation Result
Task: JPCN2zh-jaEvaluation Result
Task: JPCN2ja-zhEvaluation Result
Task: JPCN2ko-jaEvaluation Result
Task: JPCN2ja-koEvaluation Result
Task: JPCN2en-jaEvaluation Result
Task: JPCN2ja-enEvaluation Result
Task: JPCN3zh-jaEvaluation Result
Task: JPCN3ja-zhEvaluation Result
Task: JPCN3ko-jaEvaluation Result
Task: JPCN3ja-koEvaluation Result
Task: JPCN3en-jaEvaluation Result
Task: JPCN3ja-enEvaluation Result
Task: JPCEPzh-jaEvaluation Result
------------------------------
+ News Commentary task:
[JaRuNC (News Commentary task)]
Task: NCPDru-jaEvaluation Result
Task: NCPDja-ruEvaluation Result
------------------------------
+ Restricted Translation task:
Task: ASPECRTen-jaEvaluation Result
Task: ASPECRTja-enEvaluation Result

* JPCN1{zh-ja,ja-zh,ko-ja,ja-ko,en-ja,ja-en} are the same tasks as JPC{zh-ja,ja-zh,ko-ja,ja-ko,en-ja,ja-en}, respectively.

** AMFM is not calculated for JPC{N,N2,N3}, MMT, ALT20, BSD, MSCOCO, INDIC21 and MM{EV,CH}{TEXT,HI,MM}21 tasks.

Back to top

EVALUATION RESULTS (2014 - 2020)

[MultiIndicMT (Indic Languages Multilingual Parallel Corpus 2020)]
Task: INDIC20en-bnEvaluation Result
Task: INDIC20bn-enEvaluation Result
Task: INDIC20en-hiEvaluation Result
Task: INDIC20hi-enEvaluation Result
Task: INDIC20en-guEvaluation Result
Task: INDIC20gu-enEvaluation Result
Task: INDIC20en-mlEvaluation Result
Task: INDIC20ml-enEvaluation Result
Task: INDIC20en-mrEvaluation Result
Task: INDIC20mr-enEvaluation Result
Task: INDIC20en-taEvaluation Result
Task: INDIC20ta-enEvaluation Result
Task: INDIC20en-teEvaluation Result
Task: INDIC20te-enEvaluation Result
[IITB Corpus (2020)]
Task: HINDENen-hiEvaluation Result
Task: HINDENhi-enEvaluation Result
Task: HINDENhi-jaEvaluation Result
Task: HINDENja-hiEvaluation Result
Task: UFALta-enEvaluation Result
Task: UFALen-taEvaluation Result
Task: ODIAENen-odEvaluation Result
Task: ODIAENod-enEvaluation Result
[Timely Disclosure tasks: Timely Disclosure Documents Corpus (2020)]
Task: TDDCITMen-jaEvaluation Result
Task: TDDCITMja-enEvaluation Result
Task: TDDCTXTen-jaEvaluation Result
Task: TDDCTXTja-enEvaluation Result
[Asian Language Treebank (ALT) Project 2019]
Task: ALT2en-kmEvaluation Result
Task: ALT2km-enEvaluation Result
Task: ALT2en-myEvaluation Result
Task: ALT2my-enEvaluation Result
[JPO Patent Corpus (2017)]
Task: JPCzh-ja*Evaluation Result
Task: JPCja-zh*Evaluation Result
Task: JPCko-ja*Evaluation Result
Task: JPCja-ko*Evaluation Result
Task: JPCen-ja*Evaluation Result
Task: JPCja-en*Evaluation Result
[BPPT Corpus (2016)]
Task: BPPTen-idEvaluation Result
Task: BPPTid-enEvaluation Result
[RECIPE Corpus]
Task: RECIPEALLen-jaEvaluation Result
Task: RECIPEALLja-enEvaluation Result
Task: RECIPETTLen-jaEvaluation Result (for only title)
Task: RECIPETTLja-enEvaluation Result (for only title)
Task: RECIPESTEen-jaEvaluation Result (for only steps)
Task: RECIPESTEja-enEvaluation Result (for only steps)
Task: RECIPEINGen-jaEvaluation Result (for only ingredients)
Task: RECIPEINGja-enEvaluation Result (for only ingredients)
[Indic Languages Multilingual Parallel Corpus]
Task: INDICen-bnEvaluation Result
Task: INDICbn-enEvaluation Result
Task: INDICen-hiEvaluation Result
Task: INDIChi-enEvaluation Result
Task: INDICen-mlEvaluation Result
Task: INDICml-enEvaluation Result
Task: INDICen-taEvaluation Result
Task: INDICta-enEvaluation Result
Task: INDICen-teEvaluation Result
Task: INDICte-enEvaluation Result
Task: INDICen-urEvaluation Result
Task: INDICur-enEvaluation Result
Task: INDICen-siEvaluation Result
Task: INDICsi-enEvaluation Result
Task: HINDENMMEVTEXTen-hiEvaluation Result
Task: HINDENMMEVHIen-hiEvaluation Result
Task: HINDENMMEVMMen-hiEvaluation Result
Task: HINDENMMCHTEXTen-hiEvaluation Result
Task: HINDENMMCHHIen-hiEvaluation Result
Task: HINDENMMCHMMen-hiEvaluation Result
[Asian Language Treebank (ALT) Project]
Task: ALTen-myEvaluation Result
Task: ALTmy-enEvaluation Result
Back to top

AUTOMATIC EVALUATION PROCEDURES

Tools

Procedures


Back to top

EVALUATION RESULTS USAGE POLICY

When you use the WAT evaluation results for any purpose such as:
- writing technical papers,
- making presentations about your system,
- advertising your MT system to the customers,
you can use the information about translation directions, scores (including both automatic and human evaluations) and ranks of your system among others. You can also use the scores of the other systems, but you MUST anonymize the other system's names. In addition, you can show the links (URLs) to the WAT evaluation result pages.
Back to top

TECHNICAL COLLABORATOR

Back to top

NICT (National Institute of Information and Communications Technology)
Kyoto University
Last Modified: 2018-08-13