Researchers at IIT Kharagpur are making Sanskrit accessible with their Artificial Intelligence-based system for processing Sanskrit texts
There has been a renewed interest in Sanskrit since the announcement of NEP 2020. Various academic institutions both at school education as well as higher education are adopting various approaches for improving the reach of the language through training programs, research and outreach initiatives. While various digital resources have improved the accessibility and use of world languages and even regional languages, Sanskrit presents unique challenges in automated computational processing. In addition to the sheer volume and diversity, both stylistic and chronological, found in these texts, the linguistic peculiarities expressed by the language, pose several challenges in making these works accessible to the world.
Research group led by Dr. Pawan Goyal have developed a digital infrastructure for the efficient processing of Sanskrit texts, by effectively combining state-of-the-art machine learning techniques and traditional linguistic knowledge from Sanskrit. The proposed framework is based on Energy-based models and it enables the encoding of relevant linguistic information as constraints.
“Processing of Sanskrit texts poses several challenges owing to the high lexical productivity of the words, free word order in poetry, euphonic assimilation of sounds at the word boundaries and phonemic orthography followed in writing. Keeping these in mind, we proposed a generic graph-based framework that takes advantage of the free word order nature of the language. Further, we make use of linguistic insights from the traditional Sanskrit grammar for learning the feature function and applying the relevant constraints.” explained Dr. Goyal.
He further adds, “Our proposed framework substantially reduces the training data requirements to as low as 10%, as compared to that of the neural state-of-the-art models. In all the Sanskrit-related tasks discussed in the work, we either achieve state-of-the-art results or ours is the only data-driven solution for those tasks.”
This work is accepted for publication in the Computational Linguistics journal published by the MIT Press. This work has been carried by research scholar Dr. Amrith Krishna [Tweet @krishnamrith12], currently, a postdoc at the University of Cambridge, supervised by Dr. Pawan Goyal. The paper currently addresses the tasks of word segmentation (संधि विच्छेद), morphological parsing (पद विश्लेषण), dependency parsing (कारक विश्लेषण) and poetry to prose conversion of Sanskrit text (अन्वय). The team is now actively collaborating with several external research groups to extend the application of the proposed system for automatic speech recognition and question-answering in Sanskrit.
The classical language has a rich literary tradition spanning more than 2 millennia that encapsulates the cultural ethos of this civilizational nation. Works in Sanskrit, numbering more than 30 million extant manuscripts, include extensive epics, subtle and intricate philosophical, mathematical, and scientific treatises, and rich literary, poetic, and dramatic texts. The proposed AI-based system, used in conjunction with interactive tools such as the Sanskrit Heritage reader, can aid the users in the easier analysis of these manuscripts with word-by-word analysis and translation, the relation between words, poetry to prose conversion, search and question answering, etc.
Graphic by Suman Sutradhar
For more information contact Prof. Pawan Goyal at pawang@cse.iitkgp.ac.in or Dr. Amrith Krishna [Face: amrith.krishna, Tweet @krishnamrith12]
Media Coverage:
Before we delve into the issue of multiple regional languages, let us remind ourselves, that across the globe, classroom teaching is pursued in a variety of regional languages, be it France or Germany or Russia or a country like China, which has over 300 languages and dialects with 8 of them being the major ones. It has been observed that the human mind is more receptive to communication in the language in which it is accustomed to thinking from childhood. In my four decades at IIT Kharagpur, initially, as a student, research scholar and later as a faculty member, our ‘teaching and learning’ has been in English in its entirety. The practice, I would say, continues regardless of the first language of the students and difficulties at their end in adopting to English as the primary communication language, thus disrupting the process of learning. But when explained in regional languages, especially mother tongue, grabbing the concepts or expression of the ideas by the students becomes fairly easy. This can be observed frequently in practical classes and labs where the lab instructors often form a close relationship with the students and communicate with them in the languages known by the former. Even some teachers adopt this approach outside the classroom.
Chief Guest at the occasion Prof. V.K.Tewari, Director, IIT Kharagpur shared his views about the implementation of National Education Policy 2020 at Kharagpur. He emphasized the need for dynamic pedagogy keeping in mind the need for students coming from diverse linguistic backgrounds, competitiveness. To address their varied needs IIT Kharagpur offers flexible curricula and relaxation in degree completion period. Further elective and specialization courses such as the science of happiness, quality and reliability, leadership, classical and folk arts have also been introduced in addition to trending courses in financial engineering, entrepreneurship, management, IP law, healthcare etc. Prof. Tewari mentioned about the “do it yourself” lab where students can build products in a large range of engineering disciplines in accordance to the interest of students. The multidisciplinary educational experience at IIT Kharagpur would prove to be highly effective to implement NEP 2020, he said. [