Measures of success

In our project proposal, we set forth three criteria for judging success of the project: i) acceptance by users, ii) uptake by three specific communities and iii) acceptance into upstream projects.

i) Acceptance by users during our three user evaluation activities

We planned three face-to-face user evaluations, but for time and organizational reasons were able to hold only two. We received a great deal of positive feedback and useful advice in both events. The specifics and outcomes are recorded on the project’s Google Code web site and are linked to from the Usability Evaluation Plan here:

http://code.google.com/a/apache-extras.org/p/rave-in-context/wiki/UsabilityEvaluationPlan

While most of the feedback was positive, outstanding usability issues were raised and these were recorded formally within the issue tracking systems associated with the software, both in the Rave in Context project site and in the Apache Wookie site. We feel that this method of finishing a relatively short project by recording and transitioning outstanding suggestions and requests greatly facilitates the ongoing life of the code, and indeed some of these issues have been resolved since the formal completion of the project.

ii) Uptake of our widgets by users of myExperiment, Simal and openDOAR

Although we have made contact with the communities associated with the projects whose functionalities we were seeking to generalise in producing the Rave in Context templates, the templates themselves have yet to receive significant uptake by any of those communities. To some extent our work has influenced work that was already taking place within MyExperiment to refine the user interface with particular reference to mobile access. Although we were unable to initiate extensive contact with OpenDOAR during the life of the project, we will continue to keep them informed of the ongoing development of our code. In the case of Simal, technical issues meant that in order to test the widgets functionality in a timely fashion we had to integrate them instead with the JISC’s project management registry PIMS. The fact that this was successful indicates that use of our widgets within Simal is entirely possible, because Simal similar to PIMS in that it records and displays data about software projects. We are keen to pursue this further in the future.

To some extent these issues can be attributed to the fact that it is still ‘early days’ and therefore too soon to assess the impact of Rave in Context. The ongoing support of our code within the Apache Wookie project should ensure that the code is available for those communities, and for reuse in other research-driven web sites, going forward.

iii) Acceptance of our templates and recommendations into upstream projects

Here we have achieved more uptake in a shorter time than we expected. The widgets have been enjoyed further refinement and expansion by the Apache Wookie development team. Indeed, they will be used in an ongoing project to replace the administrative interface to Wookie itself. The aim of this project is to standardise the interface across Wookie and another widget server project, in an interesting reflection of the aims of our project. They have also been adapted to create a widget for accessing Twitter, and will shortly be adapted to provide access to Facebook. The widgets are also to be used as a basis for the onward development of the MAAVIS (Managed Access to Audio Visual and Information Services) software developed at the University of Sheffield. Finally the Wookie team has also enhanced the templates to produce widgets that have no dependency on Wookie itself, thus making the templates useful in a broader range of use cases, such as being served from other widget servers and compiled into native mobile applications using development toolkits such as Phonegap.