Friday, 25 March 2016

Reflection on LOTB Assessments

Baishakhi Paul & Jitesh Dhoot
(Baishakhi and Jitesh work in the Central Team of Learn, Out of the Box programme)

Learn, Out Of the Box programme started its journey with the introduction of a new technology called “WebBox” which was provided to low income schools with an objective to make classroom learning fun for students and to improve learning levels. Schools were provided with required hardware and WebBox had preloaded content for class 6th and 7th of Math and Science. Moreover, the exceptional feature of internet access through Webboxes was made available to all schools. In order to maximise the utilization, LOTB provided and still continues to give rigorous classroom support and teacher training.
To assess the usefulness of WebBox in classrooms, learning level and performance of students, enthusiasm and perception towards various teaching-learning practices and teacher skills, the programme underwent an evaluation through Internal and External assessment and inferred the following:

Internal Evaluation

To measure the student’s learning outcome pre and post tests was conducted which was pen/paper based in 50 random schools in 6 states. Mysore and Delhi had less than 50 schools so fewer schools were chosen from these states. Assam and Himachal Pradesh were not included as they had a different academic calendar. The assessment tools were designed to understand the changes in cognitive and content dimension among students. Significant improvement in performance was observed in high engagement schools where monthly subject specific usage of WebBox was more than 5 days. Qualitative assessments were conducted in 30 schools across 6 states followed by 12 and 6 schools respectively in Delhi and Mysore. In qualitative assessments, most of the students reported that they understood the content from WebBox, and that it promoted group work and helped them understand the concepts better. LOTB has deduced from its journey that most of the students prefer studying with friends more than doing it on their own. Science experiment videos were the most liked and accessed content from the WebBox amongst students. Majority of teachers accepted that technology helps to build interest amongst students. All classroom observations pictured that most schools had basic infrastructure for an interactive digital classroom but many classrooms had various problems like space, poor internet signal etc. WebBox related issues were a major challenge in the programme. From internal assessments it was also observed that teachers needed support in pedagogical skills and content knowledge. Also, additional support was required to integrate various activities into conventional teaching in classrooms.

External Evaluation

The external assessments were conducted in phases with changes in the design of study from being proto-type in first year to experimental-control design in third year. To understand the usage of Webbox, assessments were organized in Assam, Mumbai, Rajasthan and Uttarakhand in two ways that were pen/paper based and qualitative study on treatment and control groups. Only students who appeared for pre-test were evaluated for post-tests. With average changes in scores it was clear that the overall performance was improved in small steps and not in masses. Students and teachers both accepted that WebBox had improved the student attendance and their interest in learning. A large number of teachers reported that classroom participation had improved due to different activities conducted in the class.

There have been many positive impacts of the WebBox though many teachers felt that wireless technology would improve their teaching experience as that will give them mobility. The external assessment team also provided a suggestion to redesign a few areas of the programme as technology and teacher training is critical but at the same time not sufficient to improve the learning outcomes of the students. They also recommended on making the content error free, engaging school authorities and providing additional training and incentives to the teachers.

Way forward – Based on our past experiences and learnings throughout the programme journey in the last three years along with the impact assessment reports, we are trying to reflect through more focussed interventions and programme redesign.