Skip to main content
SpringerLink
Log in

Adopting computer-assisted assessment in evaluation of handwritten answer books: An experimental study

  • Published:
Education and Information Technologies Aims and scope Submit manuscript

Abstract

The use of computers in educational assessment is a widely explored territory. Several studies have been performed to show the effectiveness of computer-assisted assessment (CAA) and it has been accepted in various education sectors. However, due to the lack of sufficient infrastructure and other issues, the paper-based examination is still being used for educational assessment in many countries including India. Existing CAA frameworks require the examination to be conducted on a digital platform. So, these do not apply to paper answer books. We propose a two-phase framework for automatic evaluation of handwritten answer books. The first phase converts the answers written on papers to a digital form using a neural network-based handwritten answer recognizer. The second phase evaluates the answer to generate a numerical score. For this evaluation, we use a model answer-based approach where various levels of similarities between the model answer and a student answer are computed. To assess the performance of the developed system, we apply it to the evaluation of class VII Social Science Geography answer books of an Indian school. The experimental result shows that the proposed approach is quite promising.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. https://www.kaggle.com/c/asap-aes/overview

  2. https://en.wikipedia.org/wiki/Cosine_similarity

References

  • Alikaniotis, D., Yannakoudakis, H., & Rei, M. (2016). Automatic text scoring using neural networks. CoRR, abs/1606.04289.

  • Baker, E., & Mayer, R. E. (1999). Computer-based assessment of problem solving. Computers in Human Behavior, 15(3), 269–282.

    Google Scholar 

  • Burrows, S., Gurevych, I., & Stein, B. (2015). The eras and trends of automatic short answer grading. International Journal of Artificial Intelligence in Education, 25(1), 60–117.

    Google Scholar 

  • Burstein, J., Kukich, K., Wolff, S., Lu, C., & Chodorow, M. (1998). Enriching automated scoring using discourse marking. Proceedings of the Workshop on Discourse Relations and Discourse Marking in ACL-1998.

  • Callear, D. H., Jerrams-Smith, J., & Soh, V. (2001). CAA of short non-MCQ answers. In Proceedings of the 5th International CAA conference.

  • Cartwright, G. F., & Derevensky, J. L. (1975). An attitudinal study of computer-assisted testing as a learning method. Psychology in the Schools, 13(3), 317–321.

    Google Scholar 

  • Chapelle, C. A., & Voss, E. (2016). 20 years of technology and language assessment in. Language Learning & Technology, 20(2), 116–128.

    Google Scholar 

  • Clariana, R., & Wallace, P. (2002). Paper–based versus computer–based assessment: Key factors associated with the test mode effect. British Journal of Educational Technology, 33(5), 593–602.

    Google Scholar 

  • Conneau, A., Douwe, K., Holger, S., Loic, B., & Bordes, A. 2017. Supervised learning of universal sentence representations from natural language inference data. Volume~1. arXiv preprint arXiv:1705.02364.

  • Croft, A. C., Danson, M., Dawson, B. R., & Ward, J. P. (2001). Experiences of using computer assisted assessment in engineering mathematics. Computers & Education, 37(1), 53–66.

    Google Scholar 

  • Dumais, S. T. (2005). Latent semantic analysis. Annual Review of Information Science and Technology, 38, 188–230.

    Google Scholar 

  • Dzikovska, M. O., Nielsen, R. D., Brew, C., Leacock, C., Giampiccolo, D., Bentivogli, L., ... & Dang, H. T. (2013). Semeval-2013 task 7: The joint student response analysis and 8th recognizing textual entailment challenge. In proceedings of Seventh International Workshop on Semantic Evaluation (SemEval 2013).

  • Fisteus, J. A., Pardo, A., & García, N. F. (2013). Grading multiple choice exams with low-cost and portable computer-vision techniques. Journal of Science Education and Technology, 22(4), 560–571.

    Google Scholar 

  • Foltz, P. W., Laham, D., & Landauer T. K. (1999). Automated essay scoring: Applications to educational technology. Proceedings of ED-MEDIA Conference on Educational Multimedia, Hypermedia, and Telecommunications.

  • Khoshsima, H., Hosseini, M., & Toroujeni, S. M. H. (2017). Cross-mode comparability of Computer-Based Testing (CBT) versus Paper-Pencil Based Testing (PPT): An investigation of testing administration mode among Iranian intermediate EFL learners. English Language Teaching, 10(2), 23–32.

    Google Scholar 

  • Leacock, C., & Chodorow, M. (2003). C-rater: Automated scoring of short-answer questions. Computers and the Humanities, 37(4), 389–405.

    Google Scholar 

  • Levenshtein, V. I. (1966). Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics Doklady, 10:707.

  • Manmatha, R., & Srimal, N. (1999). Scale space technique for word segmentation in handwritten documents. In International Conference on Scale-Space Theories in Computer Vision, pp. 22-33.

  • Marti, U., & Bunke, H. (2002). The IAM-database: An English sentence database for off-line handwriting recognition. Int. Journal on Document Analysis & Recognition, 5(2002), 39–46.

    MATH  Google Scholar 

  • Miller, G. A. (1995). WordNet: A lexical database for English. Communications of the ACM, 38(11), 39–41.

    Google Scholar 

  • Mitchell, T., Russell, T., Broomhead, P., & Aldridge, N. (2002). Towards robust computerised marking of free-text responses. Proceedings of the Sixth International Computer Assisted Assessment Conference, Loughboroug University, UK.

  • Mohler, M., & Mihalcea, R. 2009. Text-to-text semantic similarity for automatic short answer grading. Proceedings of EACL-2009, pp. 567-575.

  • Mohler, M., Bunescu, R., & Mihalcea, R. (2011). Learning to grade short answer questions using semantic similarity measures and dependency graph alignments. In D.Lin (Ed.), Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies, vol. 1, pp. 752–762.

  • Moschitti, A. (2006). Making tree kernels practical for natural language learning. In 11th conference of European Chapter of the Association for Computational Linguistics, pp. 113–120.

  • Nikou, S. A., & Economides, A. A. (2016). The impact of paper-based, computer-based and mobile-based self-assessment on students’ science motivation and achievement. Computers in Human Behavior, 55(2016), 1241–1248.

    Google Scholar 

  • Page, E. B. (1968). The use of the computer in analyzing student essays. International Review of Education, 14(3), 253–263.

    Google Scholar 

  • Paiva, R. C., Ferreira, M. S., Mendes, A. G., & Eusébio, A. M. J. (2015). Interactive and multimedia contents associated with a system for computer-aided assessment. Journal of Educational Computing Research, 52(2), 224–256.

    Google Scholar 

  • Sangwin, C. (2015) Computer aided assessment of mathematics using STACK. In: Cho S. (eds) Selected regular lectures from the 12th International Congress on Mathematical Education.

  • Sangwin, C., & Kocher, N. (2016). Automation of mathematics examinations. Computers & Education, 94, 215–227.

    Google Scholar 

  • Scheidl, H. (2018), Handwritten text recognition in historical documents. Diploma Thesis, Technische Universitat Wien.

  • Shute, V. J., & Rahimi, S. (2017). Review of computer-based assessment for learning in elementary and secondary education. Journal of Computer Assisted Learning, 33(2017), 1–19.

    Google Scholar 

  • Sim, G., Holifield, P., & Brown, M. (2004). Implementation of computer assisted assessment: Lessons from the literature. Research in Learning Technology, 12(3), 215–229.

    Google Scholar 

  • Smith, A. M. (1981) Optical Mark Reading - making it easy for users, In Proceedings of the 9th annual ACM SIGUCCS Conference on User services, pp: 257–263.

  • Tandalla, L. (2012). Scoring short answer essays. ASAP ‘12 SAS Methodology Paper.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Birla Institute of Technology Mesra, Ranchi, 835215, India

    Sujan Kumar Saha & Rushali Gupta

Authors
  1. Sujan Kumar Saha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rushali Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sujan Kumar Saha.

Ethics declarations

Disclosure of potential conflicts of interest

The authors state that there is no conflict of interest.

Research involving human participants and/or animals

We state that the research and results presented in this study do not involve any animals. We used humans only as evaluator of the system.

Informed consent

For this research formal consent is not required.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Saha, S.K., Gupta, R. Adopting computer-assisted assessment in evaluation of handwritten answer books: An experimental study. Educ Inf Technol 25, 4845–4860 (2020). https://doi.org/10.1007/s10639-020-10192-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10639-020-10192-6

Keywords

Search

Navigation