The Future of Interoperability and Standards in Education: A JISC CETIS Event
The stated intention of this working meeting organised by JISC CETIS, and held at the University of Bolton, UK, on 12 January 2010 was to:
'[...] bring together participants in a range of standards organisations and communities to look at the future for interoperability standards in the education sector. The key topic for consideration is the relationship between specifications developed in informal communities and formal standards organisations and industry consortia. The meeting will also seek to explore the role of informal specification communities in rapidly developing, implementing and testing specifications in an open process before submission to more formal, possibly closed, standards bodies.' [1]
Participation was by invitation, and delegates were requested to submit position papers prior to the meeting. By the day of the event, 17 papers had been submitted; there are now 20 available on the meeting wiki, as well as several post-event blog summaries and reflections [2]. Joining the 40 or so in-person participants were a number of interested parties who followed the meeting via Twitter, some adding commentary and questions (using the Twitter hashtag cetisfis).
Meeting Organisers
The UK's Joint Information Systems Committee Centre for Educational Technology and Interoperability Standards (JISC CETIS) [3] has been, since the late 1990s, the organisation central to the development, uptake and implementation of interoperability standards for educational technology in the UK. With JISC CETIS staff having served at various points on most of the international standards bodies operating in this domain, they were well-placed to bring together an experienced group for a working meeting to discuss the future of interoperability in this area.
Rationale
In his opening address, JISC CETIS Director Adam Cooper emphasised that the impetus behind this meeting was a sense of growing dissatisfaction amongst many involved in standards development and implementation within education. Where the original intentions of more-or-less formal bodies such as the Institute for Electrical and Electronics Engineers Learning Technology Standards Committee (IEEE LTSC) [4], the IMS Global Learning Consortium (IMS GLC) [5] and the International Organisation for Standardisation (ISO) [6] were laudable, there has been an increasing feeling that the resource put into supporting these standards has not always borne the hoped-for fruit.
Meanwhile a few locally driven interoperability efforts such as Simple Web-service Offering Repository Deposit (SWORD) [7] and eXchanging Course Related Information (XCRI) [8] have appeared to work fairly well, and implementation of light-weight, non-education-specific specifications such as the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) [9], Search/Retrieval via URL (SRU) [10], Atom and RSS have shown signs of affording some real benefits.
Developments in the Web environment have also changed the landscape, with cultural expectations of greater openness, grass-roots community involvement, and agile development methodologies becoming common-place. Moreover, after years of being confined to the realm of exciting-but-far-in-the-future, real potential is starting to emerge in such areas as linked data and the Semantic Web, and open standards for identity, rights and identifiers.
More recently, the current economic climate and attendant concerns about funding cuts prompt the question, as a remote participant asked via Twitter: '[w]hen budgets are cut, is standardisation a luxury or a necessity?' [11]
So, it was deemed time for a fresh look to find the best way forward in ensuring effective interoperability. While many angles and aspects of interoperability standards could have been tackled, for a one-day event, it was decided to focus on systemic issues around the formal / informal axis of standards development.
The Delegates
The meeting was wisely co-located with the 40th meeting of the European Committee for Standardization Workshop for Learning Technologies (CEN WS-LT) [12] on 11 January [13], which ensured a large attendance for both meetings, even with transport problems predicted due to heavy snowfall. Originally expecting a turn-out of around 20 from the list of invitees, the JISC CETIS organisers were surprised and delighted that nearly 40 people ended up attending, from a number of countries and standards bodies. While this made for a cramped meeting room, it also made for a rich discussion.
A University of Bolton meeting room full of standards enthusiasts. Photo courtesy of Tore Hoel *
International communities around standards development and implementation were well represented, including participants from:
- the Dublin Core Education Community [14] and other Dublin Core Metadata Initiative (DCMI) groups [15];
- several IEEE LTSC Working Groups [4];
- various ISO groups looking at educational technology interoperability [6];
- ICOPER [16];
- the IMS Global Learning Consortium and various IMS specifications groups [5];
- the European Telecommunications Standards Institute (ETSI) [17];
- Open Web Foundation [18];
- Advanced Distributed Learning (ADL) [19], an American organisation which works with an international community of developers and vendors using Sharable Content Object Reference Model (SCORM)[20];
- and of course, the CEN WS-LT [12].
About a quarter of participants came from outside the UK, including Austria, Belgium, France, Germany, Greece, Norway and the USA.
UK-based interoperability standards development organisations and communities in attendance included BSI [21]; UKOLN [22]; the SWORD and XCRI development communities; and Becta [23].
The Position Papers: Common Ground and Tensions
Many delegates were heard to say during breaks that the position papers submitted are such a rich resource in themselves that just collecting them would have made the event worthwhile. While there is not space here to summarise the complexities of positions put forward, it is safe to say that they represented broad agreement with the impetus behind this meeting.
There was also a large degree of commonality across the papers in terms of barriers to useful standardisation for interoperability, and possible solutions. There was little that could be deemed contentious; perhaps only opinions about the best business models for funding what is a very expensive process, a process that one participant noted could be seen as a waste of many talented people's time.
The role of public projects, open source developments, and commercial vendors within standards efforts has long been an interesting, and sometimes controversial topic within the educational technology domain. For instance, one paper stated that only open markets can stir the innovation needed to create really good educational technology [24], while others spoke from a paradigm of public funding. Later discussions at the meeting touched on how interoperability standards in education have sometimes been seen as a way to stop commercial vendors taking control of what some feel is inherently a public good. In contrast, someone made the point that innovation rarely comes out of Higher Education, and that the best teaching and learning standards emerge from industries and disciplines with a high stake in their students' successful learning, such as aviation and medicine.
However, these issues represented but a small proportion of the papers and discussion. Perhaps more destructive to the forward movement of effective standards development is the number of paradoxes which remain difficult to resolve. Some of the tensions that emerged at this stage included:
- Standards development vs. Innovation (i.e. are they mutually exclusive? Do (should?) standards drive, support, hinder innovation?);
- Open community development (seen in some communities and cultures as the most desirable) vs. Formal standardisation (required by some large vendors and governments, e.g. some Asian countries);
- Open processes vs. Closed processes (e.g. documents and discussions members-only, or behind walls; standards for purchase);
- Developing for future, innovative possibilities vs. Standardising current practice;
- Open, collaborative Web 2.0 paradigms vs. Traditional educational technology (e.g. that which mimics classroom teaching; locked away behind walls);
- Conceptual models representing common thinking across diverse communities (which can be plastic and responsive) vs. Standard information models required by formal standards bodies (which can be much more stable than that which they represent);
- Achieving perfect abstract standards vs. Achieving easy, immediately implementable standards for developers.
Knowing that discussion of these issues can suck up days of meeting time, the organising team at JISC CETIS decided that the group was going to need to focus on agreement on the most potentially effective solutions proposed.
Plenary Session
JISC CETIS Assistant Director Lorna Campbell ably facilitated the entire day's proceedings, starting with an open plenary discussion. Delegates were asked what they felt were the most important issues to go on the table, given the limited time available, with a focus on what they would like to improve about the process of developing interoperability standards for education.
JISC CETIS Assistant Director Lorna Campbell joins the group in laughing at some standards-related humour. Photo courtesy of Tore Hoel *
The need for less fragmentation of effort and more effective use of the available resources for standardisation underlay most of this discussion. Initial broad ideas included improving quality by increasing transparency and inclusiveness relating to review of standards throughout their lifecycle. Or, as JISC CETIS Director Adam Cooper said, 'motherhood and apple pie'.
What about more specific points raised?
Beginning at the Beginning
Some delegates raised discussion points about the initial stages of specification and standards development, including:
- 'Visioning' needs to be done correctly from the start.
- Better specification of requirements should happen early in the development cycle.
- The need for diverse communities with an interest in a particular area of standardisation first finding agreement on a shared conceptual map of their domain. The idea is to separate out the ideas of a conceptual map and the eventual information model agreed upon within a specification.
- Avoiding duplication of effort by better connections with other areas where relevant standards may be under development or already exist, such as e-business development.
- In other words, making sure that when we focus on education-specific standards, they really have an education-specific component or requirement.
- There was one call for a complete stop on formal standardisation within educational technology, due to the sector not being mature enough. Whether or not people agreed with this, there was general agreement that focussing on the informal specification part of the standards development cycle was vital.
Specification Development Issues
The need to balance effort between abstract and long-term formal standards development and concrete, implementation-ready specifications work was key to the entire day. Rather than asking what is the best standards development process?, the question should be what are different standards processes good for? Ideas about specification development included:
- Transparency/inclusiveness is important, but should be de-coupled from the idea of community size; sometimes a small development group at the start is best, with openness of process so others can feedback and test as robustness improves over time.
- Specification development should be agile: short-term and fail-fast principles apply.
- Failing well and failing fast is important because of the issue of not being able to replace standards once they are 'done'; this can prevent innovation.
- Fail criteria as well as success criteria are needed (failure and success; slow-burner specifications; recognising negative benefits (what a standard may prevent as much as what it achieves).
- Specifications and standards need to be written for implementation; usability of specification documents must be improved.
- Availability of reference implementations and tool support is crucial.
Standardisation Issues
Acknowledging that there are times when formal standardisation is appropriate, most participants agreed that actually teasing out the details of how and when (and if) that process should happen is important further work. Some other ideas raised included:
- IP and licensing of the standards themselves must be clarified and eased.
- Better connections between standards, or families of standards would be helpful: some kind of integration 'middleware' should be looked at.
- More join up of industry and academia.
- Take small steps – don't wait to build the whole thing (e.g. the Internet Engineering Task Force (IETF)[25] model).
- Trusted governance is important, including openness of process and written documentation.
- Written records of all processes are necessary wherever more than one language group is expected to be involved (e.g. in any international standards effort); while this slows things down it ensures involvement of those who don't speak the standards body's primary language.
Adoption Issues
Achieving interoperability, and quality of standards, is intimately connected with adoption issues. If specifications and standards aren't adopted, feedback cannot be included in the development cycle. And if appropriate support for adoption is not given, there will not be wide uptake.
- We need better connections between standards bodies and policy makers, including focus on ensuring decision-makers understand what different standards are for, and where specifications that aren't formal standards may be useful.
- Continuous development and support for standards after they are ratified is important both for uptake and for future development of the standard. This includes documentation and support materials written for different stakeholder groups (e.g. implementers, librarians, educators, managers), and training.
- Better understanding of adoption dynamics is needed, including 'worse is better', e.g. the necessity for many stakeholders of accepting 'good enough' standards.
Breakout Groups and Discussion
In the afternoon, the meeting broke into discussion groups, each of which had the task of discussing and feeding back on the following questions:
- How can the relationship between formal and informal standards bodies address the issues identified in the plenary discussion?
- What are the potential benefits of involving informal standards development initiatives in the system?
- Can you identify ways forward from the position papers?
Groups were then required to identify ideas under the following headings and produce a single statement of agreement under each of them:
- Opportunities for improvement
- Barriers that have to be overcome
- Potential solutions
The final output required was a concise statement to which the whole group subscribed.
When the groups reported back to the meeting, Lorna Campbell facilitated a session where the groups' responses were collated, and their points analysed to see if agreement could be reached on a final group statement. If any individual point was rejected by anyone in the room, that point was removed from the group statement. The rejected statements were left on the board however, and are still available alongside the agreed points on the meeting wiki [26]. The final group statements under each heading were as follows:
1. Opportunities for Improvement
- Learn from the culture and lightweight processes from the informal specification community
- Improve the diversity of participation in standards communities
- Rapid, iterative development of specifications and pre-standardisation work
- Improve quality through early implementation and evaluation
- Increased adoption through involvement of more stakeholders
- Build shared understanding of concepts between stakeholders
- Recognise, understand and work with bodies which differ across a range of dimensions - e.g. legal status, respect, trust, openness, business models
- Identify criteria for discontinuing work
2. Barriers That Have to Be Overcome
- Lack of inclusiveness in specification processes
- Public procurement policy that does not recognise standards & specifications from a variety of sources (e.g. only accepts standards from certain formal bodies)
- Lack of early implementation of specifications
- Conflicting understanding of the scope and purposes of standards
- The ability to create derivative works is an essential issue. There are cases when divergence is damaging but also when derivation is prevented. How to resolve this paradox?
3. Potential Solutions
- Learn from incubation models - moving/supporting community efforts to a state where they might engage with full-blown standards ratification. For example the Apache incubator [27].
- Identify criteria for candidacy for incubation and moving from one to the other
- Match agility level to the goal/stage of the specification process
- Support adoption, community engagement & advocacy throughout the whole lifecycle from incubation to adoption and beyond
- Ensure resources/funding are available throughout the whole lifecycle of standards, not just up until ratification/publication
- Document success and failure stories to identify success and failure criteria
- Raise awareness, and be transparent, about the way we want to move through this multi-dimensional space
- Increase effective co-ordination between the different bodies
- Improve policymakers' understanding of the diversity of standards and specifications
- Manage the expectations of policymakers
- Identify solutions for the patent and ownership issues with specifications
- Many bodies should more effectively disseminate within their existing 'rules'
Conclusion
The organisers of this meeting tried not to bite off more than they could chew. They created a fairly tightly defined remit for discussion, and kept the afternoon meeting firmly to an agenda of summarising only those points on which consensus could be reached. Even so, there were many people around the table, and not much time for coming to a deeply considered consensus. The issues raised have been bubbling around for a number of years in a number of forums, and there are no easy answers.
The richness of experience and knowledgeable opinion contained within the position papers and the plenary discussion notes is perhaps best not summarised too early by a few bullet-points. However, JISC CETIS Assistant Director Scott Wilson has made a brave attempt on the meeting wiki and is welcoming feedback [28]. So, thinking together has begun, and for those who have responsibilities in the technical standards development domain, there is an opportunity now to join the discussion.
Author's note: N.B.: JISC CETIS asks that if you blog or otherwise publish on this issue, that you use the tag cetis-fis and if you Tweet about it, use the hashtag cetisfis.
Acknowledgements
* I would like to thank Tore Hoel of the ICOPER Best Practice Network, for providing all the photographs in this report. Tore can be contacted via:
Email: tore.hoel@hio.no
Web site: http://hoel.nu/wordpress/ or http://www.icoper.org/
References
- JISC CETIS Event: The Future of Interoperability and Standards in Education, 12 January 2010, Bolton, UK
http://jisc.cetis.ac.uk/events/register.php?id=215 - JISC CETIS Future of Interoperability Standards Meeting 2010, position papers and post-event reflections
http://wiki.cetis.ac.uk/Future_of_Interoperability_Standards_Meeting_2010 - JISC CETIS http://jisc.cetis.ac.uk/
- IEEE LTSC http://www.ieeeltsc.org
- IMS Global Learning Consortium http://www.imsglobal.org/
- ISO http://www.iso.org/
- SWORD http://www.swordapp.org/
- XCRI http://www.xcri.org/
- OAI-PMH http://www.openarchives.org/OAI/openarchivesprotocol.html
- SRU http://www.loc.gov/standards/sru/
- Thomas, Amber. Tweet tagged #cetisfis http://twitter.com/ambrouk/status/7673130429
- European Committee for Standardization Workshop for Learning Technologies CEN WS LT
http://www.cen-isss-wslt.din.de/ - CEN WS-LT 40th Meeting, 11 January 2010, Bolton, UK
http://wiki.cetis.ac.uk/CEN_WS-LT_2009-01-11 - DCMI Education Community http://dublincore.org/groups/education/
- DCMI http://dublincore.org/
- ICOPER http://www.icoper.org/
- ETSI http://www.etsi.org/
- Open Web Foundation http://openwebfoundation.org/
- ADL http://www.adlnet.gov/
- The Sharable Content Object Reference Model (SCORM)
http://www.adlnet.gov/Technologies/scorm/ - BSI http://www.bsigroup.com/
- UKOLN http://www.ukoln.ac.uk/
- Becta http://www.becta.org.uk/
- Weston, Crispin and Barr, Avron. Position paper for JISC CETIS Conference. 7 January 2010. http://wiki.cetis.ac.uk/images/c/c6/Saltisletsi.pdf
- The Internet Engineering Task Force (IETF) http://www.ietf.org/
- JISC CETIS Future of Interoperability Standards Meeting 2010, plenary notes and summary of group feedback session http://wiki.cetis.ac.uk/Future_of_Interoperability_Standards_Meeting_2010
- Apache Incubator http://incubator.apache.org/
- Wilson, Scott. Draft Summary of CETIS-FIS. http://wiki.cetis.ac.uk/Draft_summary_of_cetisfis