CETIS Analytics Series: A Framework of Characteristics for Analytics

Link: CETIS Analytics Series Vol 1, No 7. A Framework of Characteristics for Analytics (pdf)
Link: CETIS Analytics Series Vol 1, No 7. A Framework of Characteristics for Analytics (MS Word docx)

This paper, the seventh in the CETIS Analytics Series, considers one way to explore similarities, differences, strengths, weaknesses, opportunities, etc of actual or proposed applications of analytics. It is a framework for asking questions about the high level decisions embedded within a given application of analytics and assessing the match to real world concerns. The Framework of Characteristics is not a technical framework.

This is not an introduction to analytics; rather it is aimed at strategists and innovators in post-compulsory education sector who have appreciated the potential for analytics in their organisation and who are considering commissioning or procuring an analytics service or system that is fit for their own context.

The framework is conceived for two kinds of use:

  1. Exploring the underlying features and generally-implicit assumptions in existing applications of analytics. In this case, the aim might be to better comprehend the state of the art in analytics and the relevance of analytics methods from other industries, or to inspect candidates for procurement with greater rigour.
  2. Considering how to make the transition from a desire to target an issue in a more analytical way to a high level description of a pilot to reach the target. In this case, the framework provides a starting-point template for the production of a design rationale in an analytics project, whether in-house or commissioned. Alternatively it might lead to a conclusion that significant problems might arise in targeting the issue with analytics.

In both of these cases, the framework is an aid to clarify or expose assumptions and so to help its user challenge or confirm them.

Continue reading

CETIS Analytics Series: Analytics for Understanding Research

Link: CETIS Analytics Series Vol 1. No 4. Analytics for Understanding Research (pdf)
Link: CETIS Analytics Series Vol 1. No 4. Analytics for Understanding Research (MS Word .docx)

Analytics seeks to expose meaningful patterns in data. In this paper, we are concerned with analytics as applied to the process and outputs of research. The general aim is to help optimise research processes and deliver improved research results.

Analytics is the use of mathematical and algorithmic methods to describe part of the real world, reducing real-world complexity to a more easily understandable form. The users of analytics seek to use the outputs of analytics to better understand that part of the world; often to inform planning and decision-making processes. Applied to research, the aim of analytics is to aid in understanding research in order to better undertake processes of planning, development, support, enactment, assessment and management of research.

Analytics has had a relatively a long history in relation to research: the landmark development of citation-based analytics was approximately fifty years ago. Since then the field has developed considerably, both as a result of the development of new forms of analytics, and, recently, in response to new opportunities for analytics offered by the Web.

Exciting new forms of analytics are in development. These include methods to visualise research for comparison and planning purposes, new methods – altmetrics – that exploit information about the dissemination of research that may be extracted from the Web, and social network and semantic analysis. These methods offer to markedly broaden the application areas of analytics.

The view here is that the use of analytics to understand research is a given part of contemporaneous research, at researcher, research group, institution, national and international levels. Given the fundamental importance of assessment of research and the role that analytics may play, it is of paramount importance for the future of research to construct institutional and national assessment frameworks that use analytics appropriately.

Evidence-based impact agendas are increasingly permeating research, and adding extra impetus to the development and adoption of analytics. Analytics that are used for the assessment of impact are of concern to individual researchers, research groups, universities (and other institutions), cross-institutional groups, funding bodies and governments. UK universities are likely to increase their adoption of Current Research Information Systems (CRIS) that track and summarise data describing research within a university. At the same time, there is also discussion of increased ‘professionalisation’ of research management at an institutional level, which in part refers to increasing standardisation of the profession and its practices across institutions.

The impetus to assess research is, for these and other social, economic and organisational reasons, inevitable. In such a situation, reduction of research to ‘easily understandable’ numbers is attractive, and there is a consequent danger of over-reliance on analytic results without seeing the larger picture.

With an increased impetus to assess research, it seems likely that individual researchers, research groups, departments and universities will start to adopt practices of research reputation management. However, the use of analytics to understand research is an area fraught with difficulties that include questions about the adequacy of proxies, validity of statistical methods, understanding of indicators and metrics obtained by analytics, and the practical use of those indicators and metrics in helping to develop, support, assess and manage research.

To use analytics effectively, one must at least understand some of these aspects of analytics, and certainly understand the limitations of different analytic approaches. Researchers, research managers and senior staff might benefit from analytics awareness and training events.

Various opportunities and attendant risks are discussed in section 5. The busy reader might care to read that section before (or instead of) any others.
Continue reading

CETIS Analytics Series: What is Analytics? Definition and Essential Characteristics

Link: CETIS Analytics Series Vol 1, No 5. What is Analytics (pdf)
Link: CETIS Analytics Series Vol 1, No 5. What is Analytics (MS Word .docx)

There is currently a growing interest in better exploiting data from various sources to help organisations to be more effective and a growing number of strategies for doing this are being developed across different industries. The term “analytics” is frequently being applied to these efforts but often without clarity as to what the word is intended to mean. This makes it difficult to make sense of what is happening or to decide what to appropriate from other industries or research and hinders creative leaps forward in exploring how to adopt analytics.

This paper, part of the CETIS Analytics Series, is aimed at strategists and innovators in post-compulsory education sector who are grappling with these questions in the context of a single institution or the sector at large. It does so by:

  • considering a definition of “analytics”; and
  • outlining analytics in relation to research management, teaching and learning or whole-institution strategy and operational concerns.

Continue reading

CETIS Analytics Series: Legal, Risk and Ethical Aspects of Analytics in Higher Education

Link: CETIS Analytics Series Vol 1 No 6. Legal, Risk and Ethical Aspects of Analytics in Higher Education (pdf)
Link: CETIS Analytics Series Vol 1 No 6. Legal, Risk and Ethical Aspects of Analytics in Higher Education (MS Word .docx)

The collection, processing and retention of data for analytical purposes has become commonplace in modern business, and consequently the associated legal considerations and ethical implications have also grown in importance. Who really owns this information? Who is ultimately responsible for maintaining it? What are the privacy issues and obligations? What practices pose ethical challenges?

This paper in the CETIS Analytics series covers legal, ethical and related management issues surrounding analytics in the context of teaching, learning and research and their underlying business processes. It is based on current UK law, set in the context of publicly funded Further and Higher Education and their mission. With a primary focus on personal data, it considers the rights and expectations of the data subjects (students, researchers, employees) and the responsibilities of institutions, above campus services, suppliers and funders.
Continue reading

CETIS Analytics Series: Analytics for Learning and Teaching

Link: CETIS Analytics Series Vol 1, No 3. Analytics for Learning and Teaching (pdf)
Link: CETIS Analytics Series Vol 1, No 3. Analytics for Learning and Teaching (MS Word .docx)

A broad view is taken of analytics for Learning and Teaching applications in Higher Education. In this we discriminate between learning analytics and academic analytics: uses for learning analytics are concerned with the optimisation of learning and teaching per se, while uses of educational analytics are concerned with optimisation of activities around learning and teaching, for example, student recruitment.

Some exemplars of the use of analytics for Learning and Teaching are to:

  • Identify students at risk so as to provide positive interventions designed to improve retention.
  • Provide recommendations to students in relation to reading material and learning activities.
  • Detect the need for, and measure the results of, pedagogic improvements.
    Tailor course offerings.
  • Identify teachers who are performing well, and teachers who need assistance with teaching methods.
  • Assist in the student recruitment process.

Conclusions drawn in this paper include:

  • Learning and academic analytics are underutilised in UK Higher Education.
  • There are risks to adoption of analytics, including that measures that are not useful are revealed through analytics, and that staff and students lack the knowledge to use analytics. However, there are visualisation techniques that help significantly in non-specialist adoption.
  • There is also a risk for institutions that delayed introduction of analytics may lead to missed opportunities and lack of competitiveness in the UK Higher Education market. Our judgment is that this is the most compelling risk to amortise.
  • Institutions vary in analytics readiness and maturity, and may to a greater or lesser extent be ready for the introduction of analytics or increases in the use of analytics.
  • We stress that there are different scales of analytics projects, from small projects that can be undertaken under a limited budget and time frame, to large projects that may involve, say, creating a data warehouse and employing experienced analytics staff to build complex models.
  • A good way of starting is to undertake small limited-scope analytics projects. These enable institutions to develop staff skills and/or raise the profile of analytics in the institution.
  • Large-scale analytics may involve the activities of staff who may variously be characterized as analysts, modellers or data scientists. These staff are often in short supply, but reference to local departments of statistics may provide expert help.
  • There are commercial solutions that work in conjunction with commonly adopted Virtual Learning Environments and student information systems. Some of these are ‘plug-and-play’ and do not require analyst input. However, before purchase, they should be evaluated for suitability, particularly with respect to intuitional pedagogic approaches and concerns.

Continue reading

CETIS Analytics Series: Analytics for the whole institution

Link: CETIS Analytics Series Vol 1, No 2. Analytics for the Whole Institution: Balancing Strategy and Tactics (pdf)
Link: CETIS Analytics Series Vol 1, No 2. Analytics for the Whole Institution: Balancing Strategy and Tactics (MS Word .docx)

The benefits afforded by the longitudinal collection and analysis of key institutional data are not new to enterprise IT managers nor to senior management more generally. Data warehousing and Business Intelligence (BI) dashboards are integral to the modern management mindset and part of the ‘enterprise IT’ architecture of many Higher Education Institutions and Further Education Colleges.

However some things are changing that pose questions about how business intelligence and the science of analytics should be put to use in customer facing enterprises:

  • The demonstration by online services ranging from commodity sales to social networks of what can be done in near-real time with well-connected data.
  • The emphasis brought by the web to the detection, collection and analysis of user activity data as part of the BI mix, ranging from clicks to transactions.
  • The consequent changes in expectations among web users of how services should work, what businesses could do for them, accompanied by shifts in legal and ethical assumptions.
  • The availability of new types of tools for managing, retrieving and visualizing very large data that are cheap, powerful and (not insignificantly) accessible to grass roots IT users.

Set against that backdrop, this paper aims to;

  • Characterise the educational data ecosystem, taking account of both institutional and individual needs.
  • Recognise the range of stakeholders and actors – institutions, services (including shared above-campus and contracted out), agencies and vendors.
  • Balance strategic policy approaches with tactical advances.
  • Highlight data that may or may not be collected.
  • Identify opportunities, issues and concerns arising.

Our focus is therefore not on technology but rather on high value gains in terms of business objectives, the potential for analytics and opportunities for new thinking across the organisation.
Continue reading

CETIS Analytics Series: Analytics; what is changing and why does it matter?

Link: CETIS Analytics Series Vol 1, No 1. Analytics; What is Changing and Why Does it Matter? (pdf)

This paper provides a high level overview to the CETIS Analytics Series. The series explores a number of key issues around the potential strategic advantages and insights which the increased attention on, and use of, analytics is bringing to the education sector. It is aimed primarily at managers and early adopters in Further and Higher Education who have a strategic role in developing the use of analytics in the following areas:

  • Whole Institutional Issues,
  • Ethical and Legal Issues,
  • Learning and Teaching,
  • Research Management,
  • Technology and Infrastructure.

Continue reading

The roles of libraries and information professionals in Open Educational Resources (OER) initiatives

Link: The roles of libraries and information professionals in Open Educational Resources (OER) initiatives (pdf)

Link: Executive Summary (pdf)

Executive Summary
This report contains the findings of a study carried out by the Centre for Academic Practice & Learning Enhancement (CAPLE) and Centre for Educational Technology and Interoperability Standards (CETIS), at the University of Strathclyde. The study focuses on the involvement of the Library as an organizational unit, and of individual librarians and other information science specialists, in open educational resources (OER) initiatives. This research study contributes to the current Open Educational Resources (OER) Programme [http://www.jisc.ac.uk/oer], an initiative by JISC and the HEA whose objective is to promote the creation, dissemination, access and use of OER. This programme represents a firm commitment by UK Higher Education (HE) institutions to the OER movement.
Continue reading

IMS Learning Tools Interoperability Briefing paper

Link: http://publications.cetis.org.uk/wp-content/uploads/2012/05/LTI-Briefing-Paper.pdf (pdf)

The IMS Learning Tools Interoperability (LTI) specification provides a standard mechanism for seamlessly connecting Interoperability learning applications and remote content to virtual learning environments (VLEs) and enterprise portals.

It is increasing in popularity as a method for providing integrations which are not dependent upon a particular VLE.

This briefing paper provides an overview of the LTI specification and illustrates the benefits for developers, VLE administrators, teachers and learners.
Continue reading

Delivering Web to Mobile

Link: Delivering Web to Mobile. Main link, please link to this page when citing the report.
Link: Delivering Web to Mobile (pdf). Direct access to report.

The use of mobile devices for the consumption and use of Web content and services has grown steadily over the last few years and continues to do so, with analysts predicting that mobile will soon exceed the traditional desktop PC as the most common means users interact with the Web and other Internet services.

This report looks at the growth of mobile, the state of the Web and gives an overview of approaches to delivering content and services optimised for the mobile context. This includes approaches to Web design for responsive sites, leveraging access to device functions and capabilities and the use of Web technologies to build mobile applications.

Continue reading