Objective : Create an automated diagnostic tool to measure ASER reading level
Concept : Developing an Android-app based tool integrated with an ML model that would be trained on ASER data.
Reading Level: This tool can also be utilized to asses reading levels of children in English and Indic languages. The tool will accept audios of students reading sentences/paragraphs, stories and will then automatically evaluate this data to provide reading levels of students
Issues addressed: Oral evaluation is considered the most effective approach right now for basic literacy and spoken fluency. The need of trained evaluators for the same, puts a limit on the number of children that can be evaluated. Without a diagnostic evaluation it is difficult for teachers to provide learning content according to the child’s level.
Details : We are moving to the era of self and group learning; an automated assessment tool would be imperative to drive the motivation and check learning outcomes. Learning outcome assesment can be in the area of assessing the Numeracy levels and assesing the reading level
Reading Level: For the Asssesment of Reading/Literacy level, the model will take audio samples of children and predict his/her basic literacy or reading levels which would be classified in any of the 5 Reading levels(Beginner, Letter level, word level, short paragraph level, story level). For this classification, the features used will be pronunciation correctness, speaking rate and fluency.
● Text/pronunciation mistakes (analyzed through speech to text recognition)
● No. of pauses and length of pause (analyzed through audio signal processing)
Growing on the initial understanding of reading fluency, we hope to create oral assessment tools for science, humanities, communication skills and more. Moreover, we can recognize children's strengths and weaknesses and plan their courses accordingly.
Project Plan:
Reading Level:
| Step | Know-how | Status |
|---|---|---|
| An android app to collect audio data to be used for testing and training the ML model | Android Development | Done |
| Develop web-portals to transcribe the audios data | Web Development | Done |
| Audio sample filtering - remove background noise, silences | Audio-signal processing | In-Progress |
| Calculation of Word Error Rates for Different Speech to Text Services such as Google STT, Azure Speech to Text, IIT Bombay ASR tool, BBN Technologies ASR tool | Audio-signal processing | Done |
| Collaboration with BBN technologies and Testing their ASR Tool | Audio-signal processing | In Progress |
| Integration of IIT Bombay ASR tool to ASER app and pilot testing of the API based results | Audio-signal processing | In progress |
| Develop Speech to Text algorithms to convert the filtered audios to text | Speech Recognition | In-Progress |
| Develop and test ML Model to take text from the filtered audio samples and corresponding questions/text and labels (no. of mistakes and proficiency-level) as input and predict proficiency level | Machine Learning and Deep Learning Algorithms | Not Started |
| Conceptualize and integrate the designed ML model with an Android app (Deployment) | Android Development | In Progress |
**Progress: **
- Data Collection: An Android app has been designed to collect human evaluated training data. This app is being used in different villages of India (Maharashtra, Uttar Pradesh and Rajasthan) to collect audio samples of children reading out different levels of text (letter, word, paragraph and story), numbers (two digit, single digit) and solution to basic numeric problems(subtraction and division) along with the proficiency level marked by the evaluator.
- Speech to Text (Reading): Word Error rates for different speech to text services were calculated and the BBN Technologies ASR had the lowest WER. We would be collaborating with BBN technologies and would use and test their tools further on our datasets. There is a plan to pilot the IIT bombay ASR tool in the ASER app.
- Speech to Text (Reading): Plan to train speech to text model for Hindi and Marathi using Kaldi tool. In progress.
- Stuttering Identification in children : Functionality to identify students stuttering while speaking has been added to the Transcription portal. Scripts to identify stuttering from audio transcripts is being developed
- Model for story and paragraph has been developed by the IITB team and has been deployed to the ASER app
- Model for letter and word recognition has been developed and is being deployed
- BBN Technologies Speech recognition tool has been tested. Feedback to the team has been provided. Data has been shared for tweaking and improving the model further
Next Steps:
- Complete the transcription of collected data.
- Deployment and Testing of Letter and Word level Model
- Pilot Testing of Complete automated ASER app for Native Language