Task #11: English Lexical Sample Task via English-Chinese Parallel Text
Hwee Tou Ng and
National University of Singapore
We propose an English lexical sample task for word sense
disambiguation (WSD), where the sense-annotated examples are
(semi)-automatically gathered from word-aligned English-Chinese parallel
texts. After assigning appropriate Chinese translations to each sense of an
English word, the English side of the parallel texts can then serve as the
training data, as they are considered to have been disambiguated and
"sense-tagged" by the appropriate Chinese translations.
For more details, please refer to the full description
for this task and the references given.
First, English-Chinese parallel texts are automatically
word-aligned. Then the correct Chinese translations corresponding to the
different WordNet 1.7.1 senses of an English word are manually selected.
Finally, the English half of the parallel texts (the ambiguous English word
and its 3-sentence contexts) are used as the training and test material to
set up an English lexical sample task.
Since more than one English word sense may be translated
by the same Chinese word, two or more English senses s1, s2, ..., sk may be
collapsed into one sense in such cases. This gives rise to a lumped sense
We found from our past work that such an approach of
acquiring training examples can yield sense-tagged data of high quality (at
least as good as the quality of sense-tagged data for nouns collected in
Senseval3 English lexical sample task).
This proposed task is thus similar to the multilingual
lexical sample task in Senseval3, except that the training and test examples
are collected without manually annotating each individual ambiguous word
Datasets and Formats
We have two tracks for this task, each track using a
different corpus. The first corpus is the following English-Chinese
parallel corpus available from the Linguistic Data Consortium (LDC):
LDC2005T10 Chinese English News Magazine Parallel Text
It will be used for the evaluation of 50 English words
(25 nouns and 25 adjectives). Participants taking part in this track will
need to have access to the above LDC corpus in order to access the training
and test material in this track. Institutions that are LDC members can
obtain the corpus by paying US$150. Institutions that are non-LDC members
can obtain the corpus by paying US$2,000.
Since not all interested participants may have access to
the above LDC corpus, the second track of this task will make use of
English-Chinese documents gathered from the URL pairs given by the
STRAND Bilingual Databases. STRAND is a system that acquires
document pairs in parallel translation automatically from the Web. We will
be using this corpus for the evaluation of 40 English words (20 nouns and 20
Participants in this task can choose to participate in
one or both tracks.
The scorer will be the standard Senseval scorer.
This section will contain evaluation software, useful scripts,
complementary materials, baseline systems, etc. but not the datasets
proper. The datasets will be available at the main site for download.
Systems and Results
This section will be completed after the competition.
Chan, Yee Seng & Ng, Hwee Tou (2005). Scaling Up Word Sense Disambiguation via Parallel Texts. Proceedings of the 20th
National Conference on Artificial Intelligence (AAAI 2005). (pp. 1037-1042). Pittsburgh, Pennsylvania, USA.
Ng, Hwee Tou, & Wang, Bin, & Chan, Yee Seng (2003). Exploiting Parallel Texts for Word Sense Disambiguation: An Empirical
Study. Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics (ACL-03).
(pp. 455-462). Sapporo, Japan.
Resnik, Philip & Smith, Noah A (2003). The Web as a Parallel Corpus. Computational Linguistics, Volume 29, Issue 3 (pp. 349-380).