The purpose of this presentation is to introduce the MICOOL App Rubric, a systematic evaluation tool intended for the assessment of key criteria for the suitability of content-based educational apps to be deployed in the primary and secondary school classroom. The rubric was designed as part of an Erasmus+ Project - ‘Mobile Intercultural Cooperative Learning with Tablets (www.micool.net) involving seven European partners who have come together to share expertise and key learning experiences on the deployment of tablet technologies in schools. An important project goal is to enhance digital integration in teaching and learning including increasing the media competence of teachers through the innovative use of mobile technologies in classrooms.
The MICOOL APP Rubric was produced as part of the research output of the project undertaken by Dublin City University, Ireland. The decision to design a rubric of this nature was prompted by the lack of suitable guidelines for teachers when choosing apps for use in classrooms to support teaching and learning. The proliferation of educational apps in the Apple App store and to a lesser extent in the Google Playstore and stores of other providers, while at one level welcome, is also problematic as very few apps have been rigorously tested to assess their suitability for use in formal education settings.
This presentation will discuss the limitations of existing approaches to classroom app evaluation by teachers and educators and propose as an alternative, a framework of thematic areas of assessment, on which the rubric is based. The design rationale for the MICOOL open-source rubric tool will be outlined and an argument presented about how this approach provides for efficient, useful and comparable metrics on relevant qualities of educational apps, resulting in reproducible structured data that summarises key assessment criteria for classroom deployment.
The presentation will demonstrate the MICOOL App Rubric which is structured around 5 sections; the first of which outlines essential and recommended descriptive information to collect, the latter four of which comprise thematic sets of evaluative statements, with 5-point scale agreement assessed using specific responses. The thematic sections are:
Each section contains a number of subsection statements scored by the app reviewer. Scoring for statement agreement is from 0 to 4. Thematic evaluative sections are enumerated 1 to 4, with section 0 for recommended descriptive data collection. Statements and responses in each section are sub-numbered in an expected hierarchy. Thus, a specific level of agreement with a particular thematic question may be directly referred to in reports based on the rubric.
Using this system an app’s score derived from the MICOOL App Rubric is presented as an overall percentage, calculated as the mean of the four thematic sections, which are also given as percentages. Thus, for example a score of 85:(96,80,72,92) gives a teacher an overall sense of the quality of an app (85%) and a more specific indication that this score is most strongly based on instructional content and technical robustness, with in-app assessment a weaker area. In designing the rubric allowance was also made for the fact some apps may contain no assessment elements at all but nonetheless are still relevant as educational aids, and as such can be usefully deployed in classrooms. Examples include reference apps, historical event apps and art appreciation apps, among others. In such cases the assessment score is marked as not applicable (NA), and the overall score is calculated as the mean of the other sections. Thus when reported it is clear to the teacher that the app contains no assessment, but the evaluation metrics of the rubric still provide an overall sense of the utility of the app, and of its content, design, and robustness - e.g. 75: (80, 70, NA, 75).
The applicability of the Rubric will be demonstrated using a small sample of educational apps illustrating the utility of the tool across a diverse range of subject areas. In addition to demonstrating the rubric’s technical features the presentation will make an argument for the development of more robust methods for the assessment and evaluation of technological and digital resources in the classroom and relevant metrics for teachers and scholars. The development of the MICOOL App Rubric is an initial attempt to respond to the absence in the literature or academic practice of a coherent and effective set of evaluative criteria for apps with its designers proposing that this rubric offers a flexible and accessible tool that can be valuably used in a range of educational settings.
