Link: Cetis Analytics Series Vol 2, No 1. Case Study, Engaging with Analytics
Link: Cetis Analytics Series Vol 2, No 1. Case Study, Engaging with Analytics (MS Word .docx)

Jean Mutton, Student Experience Project Manager, University of Derby, shares with us some approaches she has been spearheading in terms of using data and analytics to help improve the student experience. Through their participation in Jisc development programmes, Jean and her team (including paid student interns) have taken a service design approach that focuses on the needs of end user first.

This case study explores the wider issues around using data to inform decision making, and the strategies the University of Derby are developing to improve their student enhancement processes by addressing key questions such as:

• What is actually happening to students, how can we find out?
• What are the touch points with between students and the institution?
• What are the institutional “digital footprints” of our students?
• What really matters to our students?

Continue reading →

CETIS Analytics Series Vol 1, No 10. Analytics for Teaching Practice (pdf)
CETIS Analytics Series Vol 1, No 10. Analytics for Teaching Practice (MS Word .docx)

Many strong claims have been made for Learning Analytics and the potential which it has to transform the education system, which deserve to be treated with caution, particularly as they regard teaching practice.

The introduction of these techniques cannot be understood in isolation from the methods of educational management as they have grown up over the past two centuries. These methods are conditioned by the fact that educational managers are limited in their capability to monitor and act upon the range of states which are taken up by teachers and learners in their learning activities. Strategies for simplification have been developed which classify the range of knowledge as a number of subjects, reduce the subjects to courses, and assign students to cohorts which carry out the same activities. Teachers, meanwhile, deal as best they can with the full variety of learners’ needs in their practice. Over the years, an accommodation has developed between regulatory authorities, management and teaching professionals: educational managers indicate the goals which teachers and learners should work towards, provide a framework for them to act within, and ensure that the results of their activity meet some minimum standards. The rest is left up to the professional skills of teachers and the ethical integrity of both teachers and learners.

This accommodation has been eroded by the efforts of successive governments to increase their control over the education received by both school and higher education students. Learning Analytics radically reduces the effort involved in gathering information on the way in which lecturers deliver the curriculum, and also to automate the work of analysing this information. An alliance of these two trends has the potential to constrain teaching practice, and therefore it is necessary to take a systemic view when assessing the impact of analytics on teaching practice.

Three types of analytics intervention are discussed, in terms of their impact on practice.

• efficiency in the wider functioning of the institution, which has few implications for teaching practice,
• enhanced regulation of the teaching and learning environment, which has potentially negative impact on teaching practice,
• methods and tools intended to help lecturers carry out their tasks more effectively, which have the potential to be a useful tool in teaching practice.

It is concluded that Learning Analytics should not be seen as a short cut to providing teaching professionals with universal advice on ‘what works’, and that its use to increase the accountability of teachers to management may have unintended negative consequences. Rather, the most promising area for enhancing teaching practice is the creation of applications which help teachers identify which of the many interventions open to them are most worthy of their attention, as part of an on-going collaborative inquiry into effective practice.

Continue reading →

Link: CETIS Analytics Series Vol 1, No 8. Institutional Readiness for Analytics (pdf)
Link: CETIS Analytics Series Vol 1, No 8. Institutional Readiness for Analytics (docx)

This briefing paper is written for managers and early adopters in further and higher education who are thinking about how they can build capability in their institution to make better use of data that is held on their IT systems about the organisation and provision of the student experience. It will be of interest to institutions developing plans, those charged with the provision of analytical data, and administrators or academics who wish to use data to inform their decision making. The document identifies the capabilities that individuals and institutions need to initiate, execute, and act upon analytical intelligence.

For the purpose of this paper, the term Learning Analytics (LA) is used to cover these activities using the definition of:

Analytics is the process of developing actionable insights through problem definition and the application of statistical models and analysis against existing and/or simulated future data. (CETIS, 2012)

The proposition behind learning analytics is not new. In the school sector particularly, good teaching practice has long involved record keeping with pen and paper and the analysis and reflection on this data to inform courses of action, and more recently using technology. Similarly, in different ways, all higher education (HE) and further education (FE) institutions use data to inform their decision making in assessment boards and course committees. However, as institutions increasingly use technology to mediate, monitor, and describe teaching, learning and assessment through Virtual Learning Evironments (VLEs) and other systems, it becomes possible to develop ‘second generation’ learning analytics. The large data sets being acquired are increasingly amenable to new techniques and tools that lower the technical and cost barrier of undertaking analytics. This allows institutions to experiment with data to gain insight, to improve the student learning experience and student outcomes, and identify improvements in efficiencies and effectiveness of provision.

Continue reading →

Link: CETIS Analytics Series Vol 1. No 4. Analytics for Understanding Research (pdf)
Link: CETIS Analytics Series Vol 1. No 4. Analytics for Understanding Research (MS Word .docx)

Analytics seeks to expose meaningful patterns in data. In this paper, we are concerned with analytics as applied to the process and outputs of research. The general aim is to help optimise research processes and deliver improved research results.

Analytics is the use of mathematical and algorithmic methods to describe part of the real world, reducing real-world complexity to a more easily understandable form. The users of analytics seek to use the outputs of analytics to better understand that part of the world; often to inform planning and decision-making processes. Applied to research, the aim of analytics is to aid in understanding research in order to better undertake processes of planning, development, support, enactment, assessment and management of research.

Analytics has had a relatively a long history in relation to research: the landmark development of citation-based analytics was approximately fifty years ago. Since then the field has developed considerably, both as a result of the development of new forms of analytics, and, recently, in response to new opportunities for analytics offered by the Web.

Exciting new forms of analytics are in development. These include methods to visualise research for comparison and planning purposes, new methods – altmetrics – that exploit information about the dissemination of research that may be extracted from the Web, and social network and semantic analysis. These methods offer to markedly broaden the application areas of analytics.

The view here is that the use of analytics to understand research is a given part of contemporaneous research, at researcher, research group, institution, national and international levels. Given the fundamental importance of assessment of research and the role that analytics may play, it is of paramount importance for the future of research to construct institutional and national assessment frameworks that use analytics appropriately.

Evidence-based impact agendas are increasingly permeating research, and adding extra impetus to the development and adoption of analytics. Analytics that are used for the assessment of impact are of concern to individual researchers, research groups, universities (and other institutions), cross-institutional groups, funding bodies and governments. UK universities are likely to increase their adoption of Current Research Information Systems (CRIS) that track and summarise data describing research within a university. At the same time, there is also discussion of increased ‘professionalisation’ of research management at an institutional level, which in part refers to increasing standardisation of the profession and its practices across institutions.

The impetus to assess research is, for these and other social, economic and organisational reasons, inevitable. In such a situation, reduction of research to ‘easily understandable’ numbers is attractive, and there is a consequent danger of over-reliance on analytic results without seeing the larger picture.

With an increased impetus to assess research, it seems likely that individual researchers, research groups, departments and universities will start to adopt practices of research reputation management. However, the use of analytics to understand research is an area fraught with difficulties that include questions about the adequacy of proxies, validity of statistical methods, understanding of indicators and metrics obtained by analytics, and the practical use of those indicators and metrics in helping to develop, support, assess and manage research.

To use analytics effectively, one must at least understand some of these aspects of analytics, and certainly understand the limitations of different analytic approaches. Researchers, research managers and senior staff might benefit from analytics awareness and training events.

Various opportunities and attendant risks are discussed in section 5. The busy reader might care to read that section before (or instead of) any others.
Continue reading →