Multilingual modeling of cross-lingual spelling variants |
| |
| Authors: | Krister Lindén |
| |
| Institution: | (1) Department of General Linguistics, Helsinki University, P.O.Box 9, Siltavuorenpenger 20 A, FIN-00014 University of Helsinki, Finland |
| |
| Abstract: | Technical term translations are important for cross-lingual information retrieval. In many languages, new technical terms
have a common origin rendered with different spelling of the underlying sounds, also known as cross-lingual spelling variants
(CLSV).
To find the best CLSV in a text database index, we contribute a formulation of the problem in a probabilistic framework, and
implement this with an instance of the general edit distance using weighted finite-state transducers. Some training data is
required when estimating the costs for the general edit distance. We demonstrate that after some basic training our new multilingual
model is robust and requires little or no adaptation for covering additional languages, as the model takes advantage of language
independent transliteration patterns.
We train the model with medical terms in seven languages and test it with terms from varied domains in six languages. Two
test languages are not in the training data. Against a large text database index, we achieve 64–78 % precision at the point
of 100% recall. This is a relative improvement of 22% on the simple edit distance. |
| |
| Keywords: | Term translations Cross-lingual information retrieval Systematic spelling variants General edit distance |
| 本文献已被 SpringerLink 等数据库收录！ |
|
