Skip to main content
SpringerLink
Log in

A Hybrid Approach for Word Alignment with Statistical Modeling and Chunker

  • Conference paper
Book cover Computational Linguistics and Intelligent Text Processing (CICLing 2015)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9041))

Abstract

This paper presents a hybrid approach to improve word alignment with Statistical Modeling and Chunker for English-Hindi language pair. We first apply the standard word alignment technique to get an approximate alignment. The source and target language sentences are divided into chunks. The approximate word alignment is then used to align the chunks. The aligned chunks are then used to improve the original word alignment.

The statistical model used here is IBM Model 1. CRF Chunker is used to break the English sentences into chunks. A shallow parser is used to break Hindi sentences into chunks. This paper demonstrates an increment in F-measure by approximately 7% and reduction in Alignment Error Rate (AER) by approximately 7% in comparison to the performance of IBM Model 1 for word alignment. Experiments of this paper are based on TDIL corpus of 1000 sentences.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aswani, N., Gaizauskas, R.: A hybrid approach to align sentences and words in english-hindi parallel corpora. In: Proceedings of the ACL Workshop on Building and Using Parallel Texts, ParaText 2005, pp. 57–64. Association for Computational Linguistics, Stroudsburg (2005), http://dl.acm.org/citation.cfm?id=1654449.1654458

    Chapter  Google Scholar 

  2. Brown, P.F., Pietra, V.J.D., Pietra, S.A.D., Mercer, R.L.: The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics 19(2), 263–311 (1993)

    Google Scholar 

  3. Caseli, H., Ramisch, C., Nunes, M.G.V., Villavicencio, A.: Alignment-based extraction of multiword expressions. Language Resources and Evaluation 44(1-2), 59–77 (2010), http://dx.doi.org/10.1007/s10579-009-9097-9

    Article  Google Scholar 

  4. Deng, Y., Kumar, S., Byrne, W.: Segmentation and alignment of parallel text for statistical machine translation. Natural Language Engineering 13(3), 235–260 (2007)

    Article  Google Scholar 

  5. Gale, W.A., Church, K.W.: Identifying word correspondence in parallel texts. In: Proceedings of the Workshop on Speech and Natural Language, HLT 1991, pp. 152–157. Association for Computational Linguistics, Stroudsburg (1991), http://dx.doi.org/10.3115/112405.112428

    Google Scholar 

  6. Hutchins, W., Somers, H.: An introduction to machine translation. Academic Press (1992), http://books.google.co.in/books?id=0ZhrAAAAIAAJ

  7. Kim, Y.-B., Ehara, T.: A method for partitioning of long japanese sentences with subject resolution in j/e machine translation. In: Proceedings of International Conference on Computer Processing of Oriental Languages, pp. 467–473 (1994)

    Google Scholar 

  8. Koehn, P.: Statistical Machine Translation. Cambridge University Press, New York (2010)

    MATH  Google Scholar 

  9. Kolachina, P., Cancedda, N., Dymetman, M., Venkatapathy, S.: Prediction of learning curves in machine translation. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 22–30. Association for Computational Linguistics, Jeju Island (2012), http://www.aclweb.org/anthology/P12-1003

  10. Lambert, P., de Gispert, A., Banchs, R.E., Mario, J.B.: Guidelines for word alignment evaluation and manual alignment. Language Resources and Evaluation 39(4), 267–285 (2005), http://dblp.uni-trier.de/db/journals/lre/lre39.html#LambertGBM05

    Article  Google Scholar 

  11. Meng, B., Huang, S., Dai, X., Chen, J.: Segmenting long sentence pairs for statistical machine translation. In: International Conference on Asian Language Processing, IALP 2009, Singapore, pp. 53–58 (December 2009)

    Google Scholar 

  12. Och, F.J., Ney, H.: A systematic comparison of various statistical alignment models. Computational Linguistics 29(1), 19–51 (2003), http://dx.doi.org/10.1162/089120103321337421

    Article  MATH  Google Scholar 

  13. Ramanathan, A., Bhattacharyya, P., Visweswariah, K., Ladha, K., Gandhe, A.: Clause-based reordering constraints to improve statistical machine translation. In: Proceedings of 5th International Joint Conference on Natural Language Processing, Chiang Mai, Thailand, pp. 1351–1355 (November 2011)

    Google Scholar 

  14. Smadja, F., McKeown, K.R., Hatzivassiloglou, V.: Translating collocations for bilingual lexicons: a statistical approach. Computational Linguistics 22(1), 1–38 (1996), http://dl.acm.org/citation.cfm?id=234285.234287

    Google Scholar 

  15. Srivastava, J., Sanyal, S.: A hybrid approach for word alignment in english-hindi parallel corpora with scarce resources. In: International Conference on Asian Language Processing (IALP), pp. 185–188 (2012)

    Google Scholar 

  16. Srivastava, J., Sanyal, S.: Segmenting long sentence pairs to improve word alignment in english-hindi parallel corpora. In: Isahara, H., Kanzaki, K. (eds.) JapTAL 2012. LNCS, vol. 7614, pp. 97–107. Springer, Heidelberg (2012), http://dblp.uni-trier.de/db/conf/tal/japtal2012.html#SrivastavaS12

    Chapter  Google Scholar 

  17. Sudoh, K., Duh, K., Tsukada, H., Hirao, T., Nagata, M.: Divide and translate: Improving long distance reordering in statistical machine translation. In: Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and Metrics MATR, pp. 418–427. Association for Computational Linguistics, Uppsala (2010), http://www.aclweb.org/anthology/W10-1762

    Google Scholar 

  18. Sun, L., Jin, Y., Du, L., Sun, Y.: Word alignment of english-chinese bilingual corpus based on chunks. In: Proceedings of the 2000 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora: Held in Conjunction with the 38th Annual Meeting of the Association for Computational Linguistics-Volume 13, pp. 110–116. Association for Computational Linguistics (2000)

    Google Scholar 

  19. Venkataramani, E., Gupta, D.: English-hindi automatic word alignment with scarce resources. In: Dong, M., Zhou, G., Qi, H., Zhang, M. (eds.) International Conference on Asian Language Processing (IALP), pp. 253–256. IEEE Computer Society (2010), http://dblp.uni-trier.de/db/conf/ialp/ialp2010.html#VenkataramaniG10

  20. Watanabe, T., Sumita, E., Okuno, H.G.: Chunk-based statistical translation. In: Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, vol. 1, pp. 303–310. Association for Computational Linguistics (2003)

    Google Scholar 

  21. Xu, J., Zens, R., Ney, H.: Sentence segmentation using ibm word alignment model 1. In: In Proceedings of EAMT 2005 (10th Annual Conference of the European Association for Machine Translation), Budapest, Hungary, pp. 280–287 (May 2005)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Indian Institute of Information Technology, Allahabad, Uttar Pradesh, India

    Jyoti Srivastava & Sudip Sanyal

Authors
  1. Jyoti Srivastava
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sudip Sanyal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jyoti Srivastava .

Editor information

Editors and Affiliations

  1. Centro de Investigación en Computación, Instituto Politécnico Nacional, Mexico DF, Mexico

    Alexander Gelbukh

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Srivastava, J., Sanyal, S. (2015). A Hybrid Approach for Word Alignment with Statistical Modeling and Chunker. In: Gelbukh, A. (eds) Computational Linguistics and Intelligent Text Processing. CICLing 2015. Lecture Notes in Computer Science(), vol 9041. Springer, Cham. https://doi.org/10.1007/978-3-319-18111-0_43

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-18111-0_43

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-18110-3

  • Online ISBN: 978-3-319-18111-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation