WAT 2020

The 7th Workshop on Asian Translation

December 4th, 2020
Suzhou, China (Online)
(Hosted by the AACL-IJCNLP 2020)

[SPONSOR] | [TIMETABLE] | [IMPORTANT DATES] | [POLICY] | [RESEARCH PAPER] | [TRANSLATION TASK] | [EVALUATION] | [BASELINE SYSTEM] | [PAPER SUBMISSION INFORMATION] | [REGISTRATION] | [CONTACT] | [RELATED LINK] | [PREVIOUS WORKSHOPS]

INTRODUCTION

Many Asian countries are rapidly growing these days and the importance of communicating and exchanging the information with these countries has intensified. To satisfy the demand for communication among these countries, machine translation technology is essential.

Machine translation technology has rapidly evolved recently and it is seeing practical use especially between European languages. However, the translation quality of Asian languages is not that high compared to that of European languages, and machine translation technology for these languages has not reached a stage of proliferation yet. This is not only due to the lack of the language resources for Asian languages but also due to the lack of techniques to correctly transfer the meaning of sentences from/to Asian languages. Consequently, a place for gathering and sharing the resources and knowledge about Asian language translation is necessary to enhance machine translation research for Asian languages.

The Workshop on Machine Translation (WMT), the world's largest machine translation workshop, mainly targets on European languages and does not include Asian languages. The International Workshop on Spoken Language Translation (IWSLT) has spoken language translation tasks for some Asian languages using TED talk data, but these is no task for written language.

The Workshop on Asian Translation (WAT) is an open machine translation evaluation campaign focusing on Asian languages. WAT gathers and shares the resources and knowledge of Asian language translation to understand the problems to be solved for the practical use of machine translation technologies among all Asian countries. WAT is unique in that it is an "open innovation platform": the test data is fixed and open, so participants can repeat evaluations on the same data and confirm changes in translation accuracy over time. WAT has no deadline for the automatic translation quality evaluation (continuous evaluation), so participants can submit translation results at any time.

In addition to the shared tasks, the workshop will also feature scientific papers on topics related to the machine translation, especially for Asian languages. Topics of interest include, but are not limited to:

• analysis of the automatic/human evaluation results in the past WAT workshops
  
• word-/phrase-/syntax-/semantics-/rule-based, neural and hybrid machine translation
  
• Asian language processing
  
• incorporating linguistic information into machine translation
  
• decoding algorithms
  
• system combination
  
• error analysis
  
• manual and automatic machine translation evaluation
  
• machine translation applications
  
• quality estimation
  
• domain adaptation
  
• machine translation for low resource languages
  
• language resources
  

SPONSOR

AAMT

SunFlare Co., Ltd.

IMPORTANT DATES

TIMETABLE

POLICY

• There is no limitation on the number of teams from one laboratory to register WAT2020.
• From one laboratory (not one team), at most two submissions are allowed for human evaluation in each sub-task.
• If a team takes part in human evaluation, it is required to submit its system description paper and to attend the workshop.
• A team can submit its system description paper even only taking part in automatic evaluation.
• In the case that a team taking part in human evaluation but does not submit their system description paper, the evaluation results, including automatic and by human, will not be presented in the overview paper provided by the organizers.

RESEARCH PAPER

• Word-/phrase-/syntax-/semantics-/rule-based, neural, and hybrids machine translation
  
• Asian language processing
  
• Incorporating linguistic information into machine translation
  
• Decoding algorithms
  
• System combination
  
• Error analysis
  
• Manual and automatic machine translation evaluation
  
• Machine translation applications
  
• Quality estimation
  
• Domain adaptation
  
• Machine translation for low resource languages
  
• Language resources
  

TRANSLATION TASK

Tasks:

• Document-level Translation tasks: (NEW!)
  
  • ParaNatCom: English --> Japanese Scientific Paper
    
  • BSD Corpus: English <--> Japanese Business Scene Dialogue
    
• Scientific paper tasks: Asian Scientific Paper Excerpt Corpus (ASPEC)
  
  • English <--> Japanese
    
  • Chinese <--> Japanese
    
• Patent tasks: Japan Patent Office Patent Corpus 2.0 (JPC2)
  
  • Chinese <--> Japanese
    
  • Korean <--> Japanese
    
  • English <--> Japanese
    
  • Chinese -> Japanese expression pattern task
    
• Newswire tasks: JIJI Corpus
  
  • Japanese <--> English (description)
    
• News Commentary task:
  
  • Japanese <--> Russian
    
• IT and Wikinews tasks:
  
  • Hindi/Thai/Malay/Indonesian <--> English (NEW!) (Multilingual Multi-domain evaluation task) (Collaboration with NICT-SAP)
    
• Mixed-domain tasks:
  
  • UCSY and ALT corpora: Myanmar <--> English
    
  • ECCC and ALT corpora: Khmer <--> English
    
• Indic tasks:
  • UFAL (EnOdia) corpus: Odia <--> English (NEW!)
    
  • Bengali/Hindi/Malayalam/Tamil/Telugu/Marathi/Gujarati <--> English (NEW!) (Modification of WAT 2018's Indic Multilingual evaluation task !)
• Multimodal:
  English --> Hindi
  English <--> Japanese (NEW!)

Dataset:


• Scientific paper tasks:
  WAT uses ASPEC for the dataset including training, development, development test and test data. Participants of the scientific paper tasks must get a copy of ASPEC by themselves. ASPEC consists of approximately 3 million Japanese-English parallel sentences from paper abstracts (ASPEC-JE) and approximately 0.7 million Japanese-Chinese paper excerpts (ASPEC-JC)
• Patent tasks:
  WAT uses JPO Patent Corpus, which is constructed by Japan Patent Office (JPO). This corpus consists of a Chinese-Japanese, Korean-Japanese, and English-Japanese patent description corpus of 1M parallel sentences with four sections, which are Chemistry (Ch), Electricity (El), Mechanical engineering (Me), and Physics (Ph), based on International Patent Classification (IPC). n
• Newswire tasks:
  WAT uses JIJI Corpus, which is constructed by Jiji Press Ltd. in collaboration with the National Institute of Information and Communications Technology (NICT). This corpus consists of a Japanese-English news corpus of 200K parallel sentences, from Jiji Press news with various categories. Participants of newswire tasks are required to get it on WAT2020 site of JIJI Corpus.
• News Commentary task:
  Like last year, we will evaluate Japanese <--> Russian translation. WAT provides a manually aligned, cleaned and filtered Japanese <--> Russian, Japanese <--> English and English <--> Russian train, development and test corpus to study extremely low resource situations for distant language pairs. Look here for additional details.
• IT and Wikinews Domain Task:
  For the first time, in collaboration with SAP and NICT, WAT will evaluate Hindi/Thai/Malay/Indonesian <--> English translation for two domains: IT domain (Software Documentation) and Wikinews domain (ALT). Look here for additional details. (Task details updated on 15th July 2020).
• Indic Tasks :
  
  • Indic Languages Multilingual task:
    Given the growing sizes of monolingual, parallel training data as well as good quality evaluation data for Indic languages we have decided to resume the 2018 task with the same name. The task covers 7 Indic Languages (Bengali, Hindi, Malayalam, Tamil, Telugu, Gujarati and Marathi) and English. Look here for additional details. (Additional/updated corpora listed on 28th July 2020. Bengali--English test set fixed on 1st August 2020.)
  • Odia <--> English
    For the first time, WAT will use the OdiEnCorp corpus collected by researchers at UFAL. The training data contains around 69,260 lines of parallel corpora and larger monolingual corpora. The data belongs to a variety of domains. For additional task details kindly look here. (Task details updated on 16th July 2020.)
• Multimodal Tasks:
  
  • Hindi --> English
    WAT will once again be organizing a multimodal English --> Hindi translation task where the input will be text and an Image and the output will be a caption (text). The training set contains around 30,000 segments (Visual Genome). For additional details related to the task kindly visit the link provided.
  • Japanese <--> English
    See details of this task.
• Myanmar (Burmese) <--> English
  WAT uses UCSY Corpus and ALT Corpus. The UCSY corpus and a portion of the ALT corpus are use as training data, which are around 220,000 lines of sentences and phrases. The development and test data are from the ALT corpus.
• Khmer <--> English
  WAT uses ECCC Corpus and ALT Corpus. The ECCC corpus and a portion of the ALT corpus are use as training data, which are around 120,000 lines of sentences and phrases. The development and test data are from the ALT corpus.

Baseline system:

Baseline systems site is now open.

Back to top

TRANSLATION TASK EVALUATION

We will evaluate the translation performance of the results submitted through automatic evaluation and human evaluation.



Automatic evaluation:
We will prepare an automatic evaluation server. You will be able to evaluate the translation results at any time using this server.


• Metric:
  BLEU, RIBES, and AM-FM^*
  * A two-dimensional automatic evaluation metric for machine translation, designed to operate at the sentence level.
  * It is based on adequacy and fluency, to decouple semantic and syntactic components of the translation process to provide a balanced view on translation quality.
  
• Format:
  The submission format and the submission method are given at the submission site below.
• Notice:
  When submitting, participants have to agree that the submitted results are attributed to JST and NICT. The results will be used and distributed for research by JST and NICT.
  Thanks to the technical collaborators: Luis Fernando D'Haro, Rafael E. Banchs and Haizhou Li.
  

Human evaluation:
Human evaluation will be carried out with two kinds of method, which are Pairwise Crowdsourcing Evaluation and JPO Adequacy Evaluation.


Submission:
Submission site is now open.
(User Name and Password is necessary to access.)


Evaluation results:
Evaluation results site is now open.


Back to top

PAPER SUBMISSION INFORMATION

Participants who submit results for human evaluation are required to submit description papers of their translation systems and evaluation results. All submissions and feedback are handled electronically as below.

• Format and Template:
  Participants must use the same format as AACL2020 long-paper and follow the same instructions in terms of the format. We do not distinguish between long and short papers.
  We ask you to use the AACL2020 LaTeX style files or Microsoft Word template that are available on the AACL-IJCNLP 2020 conference web site.
• Submission site:
  https://www.softconf.com/aacl-ijcnlp2020/WAT2020/
Back to top

REGISTRATION

The application site for task participants of WAT2020 is now open.


Back to top

ORGANIZERS

• Toshiaki Nakazawa, The University of Tokyo, Japan
• Hideki Nakayama, The University of Tokyo, Japan
• Chenchen Ding, National Institute of Information and Communications Technology (NICT), Japan
• Raj Dabre, National Institute of Information and Communications Technology (NICT), Japan
• Shohei Higashiyama, National Institute of Information and Communications Technology (NICT), Japan
• Anoop Kunchukuttan, Microsoft AI and Research, India
• Win Pa Pa, University of Computer Studies, Yangon (UCSY), Myanmar
• Ondřej Bojar, Charles University, Prague, Czech Republic
• Shantipriya Parida, Idiap Research Institute, Martigny, Switzerland
• Isao Goto, Japan Broadcasting Corporation (NHK), Japan
• Hidaya Mino, Japan Broadcasting Corporation (NHK), Japan
• Hiroshi Manabe, National Institute of Information and Communications Technology (NICT), Japan
• Katsuhito Sudoh, Nara Institute of Science and Technology (NAIST), Japan
• Sadao Kurohashi, Kyoto University, Japan
• Pushpak Bhattacharyya, Indian Institute of Technology Patna (IITP), India
Back to top

PROGRAM COMMITTEE MEMBERS

• Michael Carl, Kent State University, USA
• Chenhui Chu, Kyoto University, Japan
• Fabien Cromières, Free, France
• Hideto Kazawa, Google, Japan
• Anoop Kunchookuttan, Microsoft AI and Research, India
• Kenji Imamura, NICT, Japan
• Liling Tan, Rakuten Institute of Technology, Singapore
• Masao Utiyama, NICT, Japan
• Yang Liu, Tsinghua University, China
• Takashi Ninomiya, Ehime University, Japan
• Jiajun Zhang, Institute of Automation Chinese Academy of Sciences, China
• Gurpreet Singh Lehal, Punjabi University, Patiala, India
Back to top

TECHNICAL COLLABORATOR

• Luis Fernando D'Haro, Universidad Politécnica de Madrid, Spain
• Rafael E. Banchs, Nanyang Technological University, Singapore
• Haizhou Li, National University of Singapore, Singapore
• Chen Zhang, National University of Singapore, Singapore
Back to top

CONTACT

For questions, comments, etc. please email to "wat-organizer -at- googlegroups -dot- com".

Back to top

RELATED LINK

Japan Patent Office
The Association for Natural Language Processing
National Institute of Information and Communications Technology (NICT)
Asia-Pacific Association for Machine Translation (AAMT)

Back to top

PREVIOUS WORKSHOPS

• WAT 2019
  The 6th Workshop on Asian Translation (WAT 2019) was held in December 2019 in Hong Kong, China.
• WAT 2018
  The 5th Workshop on Asian Translation (WAT 2018) was held in December 2018 in Hong Kong, China.
• WAT 2017
  The 4th Workshop on Asian Translation (WAT 2017) was held in November 2017 in Taipei, China.
• WAT 2016
  The 3rd Workshop on Asian Translation (WAT 2016) was held in December 2016 in Osaka, Japan.
• WAT 2015
  The 2nd Workshop on Asian Translation (WAT 2015) was held in October 2015 in Kyoto, Japan.
• WAT 2014
  The 1st Workshop on Asian Translation (WAT 2014) was held in October 2014 in Tokyo, Japan.

CHANGE LOG

2020-07-08: registration page etc. opened
2020-02-01: site opened

Back to top

---

NICT (National Institute of Information and Communications Technology)
Kyoto University
Last Modified: 2020-07-08

* Hailong Cao, Harbin Institute of Technology, China * Michael Carl, Kent State University, USA * Chenhui Chu, Osaka University, Japan * Fabien Cromières, Kyoto University Japan * Hideto Kazawa, Google, Japan * Anoop Kunchookuttan, Microsoft AI and Research, India * Micha Elsner, Ohio State University, USA * Kenji Imamura, NICT, Japan * Liling Tan, Universität des Saarlandes, Germany * Masao Utiyama, NICT, Japan * Gurpreet Singh Lehal, Punjabi University, Patiala, India * Yang Liu, Tsinghua University, China * Fei Liu, University of Central Florida, USA * Takashi Ninomiya, Ehime University, Japan * Pontus Stenetorp, University College London, UK * Jiajun Zhang, Institute of Automation Chinese Academy of Sciences, China * Dongdong Zhang, Microsoft Research Asia, China