Concept unification of terms in different languages via web mining for Information Retrieval |
| |
| Authors: | Qing Li Yuanzhu Peter Chen Sung-Hyon Myaeng Yun Jin Bo-Yeong Kang |
| |
| Institution: | 1. Southwestern University of Finance and Economics, China;2. Memorial University of Newfoundland, St. John’s, Canada;3. Information and Communications University, Daejeon, Republic of Korea;4. Seoul National University, Seoul, Republic of Korea |
| |
| Abstract: | For historical and cultural reasons, English phases, especially proper nouns and new words, frequently appear in Web pages written primarily in East Asian languages such as Chinese, Korean, and Japanese. Although such English terms and their equivalences in these East Asian languages refer to the same concept, they are often erroneously treated as independent index units in traditional Information Retrieval (IR). This paper describes the degree to which the problem arises in IR and proposes a novel technique to solve it. Our method first extracts English terms from native Web documents in an East Asian language, and then unifies the extracted terms and their equivalences in the native language as one index unit. For Cross-Language Information Retrieval (CLIR), one of the major hindrances to achieving retrieval performance at the level of Mono-Lingual Information Retrieval (MLIR) is the translation of terms in search queries which can not be found in a bilingual dictionary. The Web mining approach proposed in this paper for concept unification of terms in different languages can also be applied to solve this well-known challenge in CLIR. Experimental results based on NTCIR and KT-Set test collections show that the high translation precision of our approach greatly improves performance of both Mono-Lingual and Cross-Language Information Retrieval. |
| |
| Keywords: | Information Retrieval Cross-language Machine translation Indexing Out-of-vocabulary (OOV) words |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
