The 6th Workshop on Asian Translation
| [PANEL DISCUSSION]
| [IMPORTANT DATES]
| [TRANSLATION TASK]
| [BASELINE SYSTEM]
| [PAPER SUBMISSION INFORMATION]
| [PREVIOUS WORKSHOPS]
November, 3 or 4 2019
Hong Kong, China
Many Asian countries are rapidly growing these days and the importance of communicating and exchanging the information with these countries has intensified.
To satisfy the demand for communication among these countries, machine translation technology is essential.
Machine translation technology has rapidly evolved recently and it is seeing practical use especially between European languages.
However, the translation quality of Asian languages is not that high compared to that of European languages,
and machine translation technology for these languages has not reached a stage of proliferation yet.
This is not only due to the lack of the language resources for Asian languages
but also due to the lack of techniques to correctly transfer the meaning of sentences from/to Asian languages.
Consequently, a place for gathering and sharing the resources and knowledge about Asian language translation is necessary to enhance machine translation research for Asian languages.
The Workshop on Machine Translation (WMT), the world's largest machine translation workshop, mainly targets on European languages and does not include Asian languages.
The International Workshop on Spoken Language Translation (IWSLT) has spoken language translation tasks for some Asian languages using TED talk data,
but these is no task for written language.
The Workshop on Asian Translation (WAT) is an open machine translation evaluation campaign focusing on Asian languages.
WAT gathers and shares the resources and knowledge of Asian language translation to understand the problems to be solved
for the practical use of machine translation technologies among all Asian countries.
WAT is unique in that it is an "open innovation platform":
the test data is fixed and open, so participants can repeat evaluations on the same data and confirm changes in translation accuracy over time.
WAT has no deadline for the automatic translation quality evaluation (continuous evaluation), so participants can submit translation results at any time.
Following the success of the previous WAT workshops (WAT2014 -- WAT2018),
WAT2019 will bring together machine translation researchers and users to try, evaluate, share and discuss brand-new ideas about machine translation.
- Scientific paper tasks: Asian Scientific Paper Excerpt Corpus (ASPEC)
- English <--> Japanese
- Chinese <--> Japanese
- ... TBA
TRANSLATION TASK EVALUATION
We will evaluate the translation performance of the results submitted through
automatic evaluation and human evaluation.
We will prepare an automatic evaluation server.
You will be able to evaluate the translation results at any time using this server.
BLEU, RIBES, and AM-FM*
* A two-dimensional automatic evaluation metric for machine translation, designed to operate at the sentence level.
* It is based on adequacy and fluency,
to decouple semantic and syntactic components of the translation process
to provide a balanced view on translation quality.
The submission format and the submission method are given at the submission site below.
When submitting, participants have to agree that the submitted results
are attributed to JST and NICT.
The results will be used and distributed for research by JST and NICT.
Thanks to the technical collaborators: Luis Fernando D'Haro, Rafael E. Banchs and Haizhou Li.
Human evaluation will be carried out with two kinds of method,
which are Pairwise Crowdsourcing Evaluation and JPO Adequacy Evaluation.
- Pairwise Crowdsourcing Evaluation:
Pairwise Crowdsourcing Evaluation will be carried out using crowdsourcing.
Organizers will sample 400 sentences from the test data for the pairwise
Participants can submit translation results for the human evaluation
a maximum of twice until the pairwise crowdsourcing evaluation is due.
(For automatic evaluation, there is no limitation on submitting times.)
Sentence-by-sentence pair-wise evaluation compared to the baseline system.
The crowdsourcing workers will be asked to judge which translation is better
than the other in view of adequacy and fluency.
To guarantee the quality of the evaluation, each sentence is evaluated by 5
different workers and the final decision is made by the voting of the judgements.
The submission format is the same as that of automatic evaluation.
Participants can select their translation results from the ones submitted
for automatic evaluation.
All systems will be ranked by the percentage of translations judged
to improve upon the baseline system.
- JPO Adequacy Evaluation:
We will also evaluate with the criteria of Content Transmission Level Evaluation which JPO defined
(pages 5 to 8 in the PDF file (Japanese Page)).
We will sample 200 sentences from the pairwise crowdsourcing evaluation data
for the JPO adequacy evaluation.
The JPO adequacy evaluation will be conducted only for translation results of 3 top-scored teams
on the Pairwise Crowdsourcing Evaluation in each subtask's language pair.
PAPER SUBMISSION INFORMATION
- Toshiaki Nakazawa, The University of Tokyo, Japan
- Chenchen Ding, National Institute of Information and Communications Technology (NICT), Japan
- Shohei Higashiyama, National Institute of Information and Communications Technology (NICT), Japan
- Raj Dabre, National Institute of Information and Communications Technology (NICT), Japan
- Anoop Kunchukuttan, Microsoft AI and Research, India
- Win Pa Pa, University of Conputer Study, Yangon (UCSY), Myanmar
- Isao Goto, Japan Broadcasting Corporation (NHK), Japan
- Hidaya Mino, Japan Broadcasting Corporation (NHK), Japan
- Katsuhito Sudoh, Nara Institute of Science and Technology (NAIST), Japan
- Graham Neubig, Carnegie Mellon University (CMU), USA
- Hideto Kazawa, Google, Japan
- Yusuke Oda, Google, Japan
- Jun Harashima, Cookpad Inc., Japan
- Sadao Kurohashi, Kyoto University, Japan
- Ir. Hammam Riza, Agency for the Assessment and Application of Technology (BPPT), Indonesia
- Pushpak Bhattacharyya, Indian Institute of Technology Patna (IITP), India
- Luis Fernando D'Haro, Institute for Infocomm Research, Singapore
- Rafael E. Banchs, Institute for Infocomm Research, Singapore
- Haizhou Li, Institute for Infocomm Research, Singapore
For questions, comments, etc. please email to "wat -at- nlp -dot- ist -dot- i -dot- kyoto -hyphen- u -dot- ac -dot- jp".
Japan Patent Office
The Association for Natural Language Processing (Japanese Page)
National Institute of Information and Communications Technology (NICT)
- WAT 2018
The 5th Workshop on Asian Translation (WAT 2018) was held in December 2018 in Hong Kong, China.
- WAT 2017
The 4th Workshop on Asian Translation (WAT 2017) was held in November 2017 in Taipei, China.
- WAT 2016
The 3rd Workshop on Asian Translation (WAT 2016) was held in December 2016 in Osaka, Japan.
- WAT 2015
The 2nd Workshop on Asian Translation (WAT 2015) was held in October 2015 in Kyoto, Japan.
- WAT 2014
The 1st Workshop on Asian Translation (WAT 2014) was held in October 2014 in Tokyo, Japan.
2019-02-01: site opened
NICT (National Institute of Information and Communications Technology)
Last Modified: 2019-02-01