Delivering Open Educational Resources for Engineering Design

A great deal of information is accessible on the World Wide Web which might be useful to both students and teachers. This material, however, is of variable quality and usefulness and is aimed at a wide spectrum of users. Moreover, such material rarely appears accompanied by guidance on how it may be most effectively used by potential users. To make information more usable it must be made more readily discoverable and there should be clear – and preferably machine-readable – indications of its provenance and quality and the legitimate uses to which it may be put.

The recently completed HEA/JISC-funded Delivering OERs for Engineering Design (DelOREs) Project set out to explore the creation of two Web-accessible ‘collections’ of university-level Open Educational Resources (OERs) relevant to Engineering Design. The research was led by the University of Bath in partnership with Heriot-Watt University, the former with staff having experience in engineering design and its teaching, while the latter’s staff were experienced in the provision of research and educational material using Web-based technologies.

The term ‘open’ in the context of OERs embraces the notions of free availability and as few restrictions as possible – for example, technical, legal and cost – on their use. The users for this project were identified as being, on the one hand, undergraduate students of engineering design, and on the other, the teachers of those students.

The first collection – known as Delores Selections [1] – is one of Open Educational Resources relating to engineering design relevant to and presented in a way appropriate for teachers and students at first-year undergraduate level.

The second collection – known as Delores Extensions [2] – is one of Open Educational Resources and other openly available resources useful to higher-level students studying engineering design and the teachers of such students.

In neither of these collections are the resources held locally; rather, the collections constitute indexes to the resources themselves which are to be found on the World Wide Web, sourced from individual examples of OERs and from collections such as MIT OpenCourseWare [3], JISC UKOER Phase 1 projects [4], OCWC [5], OER Commons [6] and the SEED collection [7].

DelOREs Selections

This static collection of OERs is presented using a weblog or blog interface. WordPress, a configurable blogging software package, was selected as the blog core partly because it has a very large user base and partly because of its plug-in architecture which provides the potential for relatively easy extension of its functionality. In addition to this, the ‘look and feel’ of the interface presented to the user is readily customisable.

The WordPress software was configured initially using standard components to allow DelOREs Project members to write single posts each of which is a formalised description of a single OER held at a Web location.

The description for each resource is based on a template which provides information relating to such things as subject content, resource description, location linking and licensing constraints. In addition, content is provided for RSS output to interested sites which can incorporate the resource descriptions automatically in their own pages.

The WordPress content is organised using a subject-specific taxonomy, devised by the engineering design experts within the team and evaluated by independent experts, and represented in the Delores Selections interface as a nested tree. The clickable tree, which shows, for each taxonomic label, the number of resources held at that level in the tree, provides the user access to appropriately classified content The classification tree appears as a standard element on the right in each page of Delores Selections.

The ‘custom fields’ functionality of WordPress has been used to record the resource URL, author name, publication date, licence URL, and a human-readable rights statement.

A widely used WordPress theme – Carrington [8] – was adapted so that the additional metadata fields, the category (i.e. topic) and tags (e.g. resource type) are displayed to the end user with each post, and also so that the link from the heading for each post (being the title of the resource) takes the user directly to the resource, rather than the more common blogging practice where the link is a permalink leading to the individual post.

DelOREs Extensions

In contrast to Delores Selections, Delores Extensions is a dynamic collection of resources which requires some means of automatically selecting and updating content. This requires in itself the means both of identifying potential content and discarding that which does not meet inclusion criteria. A means, too, must be chosen to allow the user to select appropriate material from the collection.

The building blocks are two software packages. To provide user access to the content, the Waypoint software package was selected [9]. This package implements an adaptive concept-matching algorithm – unique to Waypoint and developed by Chris McMahon and his colleagues – which allows browsing of material that has been classified against a set of faceted classification schemes. The user is able to interactively ‘prune’ a classification tree list which responds in real time to the user’s multiple selection of taxonomic labels. This means of exploration is particularly beneficial to the user who already has some conceptualisation of the ‘space’ being explored.

Delores Extensions takes advantage of the core material identified and selected manually for Delores Selections, but is augmented by using ‘conventional’ Web-based discovery techniques including RSS feeds, crawling, spidering and scraping.

Two approaches to ‘filtering’ potential content have been devised, the first is manual, the second automatic. The first is to provide a mechanism in the delivery interface to allow users to vote on the extent to which a proffered resource matches the classification that has been given. As a result of this, resources may be reclassified more appropriately. The second means of filtering uses the second main building block for Delores Extensions, this being an implementation of the sux0r [10] software. Sux0r has a number of functionalities, but crucially provides aggregation of RSS feeds which can be passed through a filtering system using naive Bayesian categorisation [11]. This function allows appropriate training sets to be used to train the sux0r implementation to distinguish between classes of input. Once trained, the sux0r filter retains or discards resources based on the system ‘knowledge’.

Identifying Candidate OERs for Engineering Design

Candidates for OERs cover a wide spectrum ranging from single stand-alone documents and Web pages, to fully structured Web sites, through webcasts to sets of lecture notes. To provide a useful resource, material must conform to the minimal ideal requirements, these being:

1. Subject appropriateness.
2. A sufficient quality.
3. Alignment with target audience needs.
4. Explicitness of licensing conditions which define an OER.

Inferring the extent to which a resource conforms to the first three criteria requires expert judgement or, when attempting to acquire the resource automatically, then some judgement-making algorithm of sufficient differentiating capacity. These items might also be inferred from the provenance or authority of the source. At present, entirely reliable automatic judgement-making is probably out of reach. However, work in progress may make this more attainable in the short term. Such work includes, for example, the Learning Registry [12] initiative which is seeking to facilitate the collection and provision of the information that would be necessary. Likewise, work on the assessment of information value (including measures of quality) will support this sort of judgement making [13]. The final item requires no judgement, merely a clear, preferable machine-readable, statement.

The conventional criterion for inclusion of a resource as an OER is that it available for reuse under a Creative Commons (CC) licence [14] or a licence of equivalent clarity providing similarly unrestricted use. For Delores Selections, the selection criterion was relaxed to include, in addition to CC-licensed resources, those which were offered to the reader for reuse in a spirit of sharing, and where the spirit of the CC licence (if not the letter) is conveyed by the provider. Such material is offered in manner which conveys the desire to share with limited or no restriction couched in informal rather than formal terms.

The key findings of the DelOREs Project relate to the availability of subject-related material, the extent to which such material is made functionally re-usable as a result of suitable licensing instruments, and the discoverability of resources, especially of those which have ostensibly been placed in repositories for reuse by an identified audience.

The Availability of Subject Resources

Contrary to the project team’s expectation, there is a very limited amount of material that is offered expressly for ‘open’ use and which conforms to the required licensing criteria.

The culture of making material available with minimal constraints on reuse, and clearly marked in this respect, is very immature. There are a number of OER collections and repositories whose brief is to do this; however, the material that has been provided specifically for engineering design students and teachers is very restricted in both scope and quantity. Whilst a good deal of material has clearly been placed in public sight with the intention of allowing its use for education purposes, the legitimate usage of this material remains unclear and, thus, it cannot be offered easily through a subject collection because of the danger of inadvertent encouraging the reuse of material in a non-legitimate manner.

Currently, there is no widespread culture of OER provision. The ideal ‘Open Practice’ culture will be one in which individuals and institutions default to making their material available to others under appropriate licensing. This is very far from the situation which exists at present.

Resource Visibility

Before a potential open educational resource can be used as such, it must first be discovered, either by the end-user or, in the case of OER collections, by the OER collection provider. Discovery can be achieved either using ‘pull’ techniques (i.e. search) or ‘push’ techniques (i.e. targeted information delivery).

The experience of the DelOREs Project shows that looking for suitable material, even when ostensibly the item to be discovered has been provided expressly as an open educational resource, requires time and effort which is not always well rewarded. The ideal situation would be where a resource is not only made available, but that the intention for its usage, its purpose, its applicability and so on, is clearly and explicitly described, and in a way that, ideally, is machine-readable, so that it can be processed automatically. In effect, any material that is presented for use as an OER should be self-advertising of that fact if not to machines then at least to humans. This could be achieved in a number of ways. An example of resources that go some way to fulfilling this ideal are those which have been placed on the exemplary MIT OpenCourseWare Web site [3]. Each resource introductory page contains a content description based on a locally standardised XHTML schema which includes such metadata as title, author, course details, version and so on. All actual resources (which are associated through child pages from the introductory page) are presented in pdf format. These are downloadable using a single mechanism which is easily automated, for this specific site.

Because of this standardisation and the reliance on an XML data representation and pdf format, automatic processing is possible to some extent, once the local schema is known.

Automatic Resource Discovery and Processing

The discovery of OERs is difficult at present because of the limited amount of machine-readable data that is provided.

During the project the most successful approach to automatic discovery consisted of using bespoke Google search scripts to encode a set of search terms. These terms are case-specific; for example, in the DelOREs Project, discovery was attempted using such terms as: gear, machining and other keywords more or less specialised to engineering and engineering design. However, discovery performance is dependent on the felicity with which search terms are selected and the extent to which their matching search engine index content results in OERs being identified.

Once a resource or resource repository is found – that is, the discovery element has been achieved by manual or automatic means – it is possible to identify by inspection the schema that has been chosen to encode the resource data and metadata. It is then possible to write a script that will extract the resource descriptive data. During the project, successful extraction scripts were written for OER Commons, MIT OpenCourseWare and JORUM.

Some resource providers (e.g. MIT OCW) provide content in XML format which assists script writing because this encoding is more amenable to processing than, for example, HTML or native formats such as pdf, MSWord, etc. Representing such data in other forms (e.g. RDF, json or microdata) would be equally beneficial. The standardisation of the content is currently being moved forward by the Learning Resource Metadata Initiative (LRMI [15]), a project co-led by the US-based Association of Educational Publishers and Creative Commons, to build a common metadata vocabulary for educational resources.

For fully automatic discovery and data processing of OERs to be achieved, which would return optimal results, machine-readable information should be supplied which satisfies the four qualifying items for OERs identified above. Ideally this would be represented in a way that the harvesting mechanism could access easily and use as the basis of inference without any manual intervention.

In the meantime, utilising the power of search engines could be improved by one simple expedient. The key ingredient is the explicit identification of a resource as being an OER. Even a statement of this fact in plain language would enhance the capacity of search engines to identify and index a resource as an OER and to deliver it in response to a search, ranking it as more valuable than material which may be appropriate by other measures but not explicitly OERs.

Conclusion

The general conclusions arrived at in the DelOREs Project concern resource availability for the engineering subject area specifically, the current non-standardisation of information relating to OERs and the current OER ‘culture’.

• On the one hand, it is apparent that there is a good deal of material suitable for use as educational resources, but currently not usable as such because of uncertainty about quality, fit in the educational framework, and the legitimate use to which the resources can be put.
• Resources provided for open use in the subject area do exist. These, however, are relatively few in number, and insufficient to provide a properly balanced population of resources which effectively cover the entire spectrum of teaching and learning needs across the university-level engineering design curriculum.
• There is a great diversity in style, type and presentation of the limited resources. This extends not only to the educational content of the material, but the way in which resources are identified as being OERs, the copyright and the licensing terms under which they are presented, and such information as the subject coverage, syllabus level and so on.
• Related to the above is the incomplete status of standards for OER provision either for expressing OER licensing and content in human-readable form or in a manner which is machine-readable. There is no standardisation yet of means of automatic exchange of OER material, and what content is currently machine-readable with precision is only so once the representation schema for a particular provider or resource is known.
• There is no strong culture of OER provision at individual or institutional level except in a very restricted set of institutions. The default position continues to be that of guarding rights to intellectual property, rather than of finding ways to share and add value to existing material.
• On a more positive note it is clear that there is a clear understanding amongst teachers of the benefits of not only having OERs to use, but to providing their material as OERs. This suggests that resources are needed for the provision by institutions of their educational resources for wider use. On the other hand, from the information gained from subject experts during the project, it is clear that there is uncertainty surrounding how to go about making their material suitable as OERs and the mechanisms for publishing it as an ‘open’ resource. This suggests that education is required in the benefits of releasing hitherto ‘private’ material for sharing and reuse, particularly within an OER framework, and how best to go about it.

Currently, as readers of Ariadne will know, the UK academic authorities are encouraging the active management of research data to ensure its availability for reuse and re-purposing; doing likewise in respect of teaching and learning materials would be highly beneficial to the cause of OERs. In this respect there are two important points to be made. First, nearly all the material that was included in the collections was from well-resourced institutions in the USA: there is no culture of sharing in the UK, and the current climate in HE does not support origination of shared teaching materials or indeed collaboration to achieve more efficient or effective teaching. Secondly, the DelOREs Project has enabled the SEED collection [7] and (the remaining parts of) the now defunct CDEN/Ryerson Collection to be made available once more on the World Wide Web. These are collections of very useful learning resources that had fallen into disuse because of the absence of financial mechanisms to sustain them – indeed much of the original CDEN material has been lost for all time. The SEED material was originated in the 1980s – a time that was much more amenable to the origination of shared material and to collaboration in teaching. New ways must be found to re-engender that spirit of shared endeavour and co-operation and to make it viable through appropriate financial and organisational means.

At present many of our institutions would regard the effort spent in the origination and sharing of educational resources more profitably channelled toward maximising REF scores. In the changing climate of increasing professionalisation of teaching in HE, there must be an expectation that such resource-provision activities will be required. It follows that the UK academic community, and the funding councils, need to find ways of rewarding those individuals and institutions who contribute open resources to the community, else little will be done.

Acknowledgements

The project [16] was funded by the Higher Education Academy/JISC through the strand of the Open Educational Resources Programme Phase 2 [17] of the e-Learning programme, in particular relating to the delivery of collections of OERs based within a thematic area.

References

1. The Delores Selections Web site http://www.icbl.hw.ac.uk/delores/selections/?page_id=2
2. The Delores Extensions Web site http://go.bath.ac.uk/delores-extensions
3. MIT OpenCourseWare http://ocw.mit.edu/index.htm
4. The UKOER Phase 1 Projects http://www.jisc.ac.uk/whatwedo/programmes/elearning/oer
5. Open Course Ware Consortium Web site http://www.ocwconsortium.org/
6. OER Commons http://www.oercommons.org/about
7. SEED OER Collection http://www.bath.ac.uk/idmrc/themes/projects/delores/co-design-website/
8. WordPress Free Themes Directory http://wordpress.org/extend/themes/carrington-blog
9. McMahon, C., Lowe, A., Culley, S., Corderoy, M., Crossland, R., Shah, T. and Stewart, D., 2004. Waypoint: An integrated search and retrieval system for engineering documents. Journal of Computing and Information Science in Engineering, 4 (4), pp. 329-338.
10. The sux0r software home page http://www.sux0r.org/
11. The Bayesian RSS Feed Filter Project home page
    http://bayesianfeedfilter.wordpress.com/2009/08/25/about-sux0r/
12. The Learning Registry http://www.learningregistry.org/
13. Darlington, M. J., Culley, S. J., Zhao, Y., Austin, S. A. and Tang, L.C.M., 2009. Defining a framework for the evaluation of information. International Journal of Information Quality, 2 (2), pp. 115-132.
14. The Creative Commons UK Web site http://www.creativecommons.org.uk/
15. The Learning Resource Metadata Initiative (LRMI) Web site http://wiki.creativecommons.org/LRMI
16. DelOREs Project Contact Details:
    Professor Chris McMahon (c.a.mcmahon@bath.ac.uk) – engineering design and engineering design education and Waypoint, and;
    Phil Barker (phil.barker@hw.ac.uk) – Web-based delivery of educational and research information and sux0r software and Bayesian feed filtering.
17. JISC Open Educational Resources Phase 2 Programme
    http://www.jisc.ac.uk/whatwedo/programmes/elearning/oer2.aspx

Author Details

Dr Mansur Darlington
IdMRC (Innovative Design & Manufacturing Research Centre) Centre Manager
Department of Mechanical Engineering
University of Bath

Email: M.J.Darlington@bath.ac.uk
Web: http://www.bath.ac.uk/idmrc/

Mansur Darlington is the Centre Manager for the IdMRC at the University of Bath and a research officer in the Design Information & Knowledge Group of the Centre. For the last 15 years he has been involved in research associated with the capture and codification of engineers’ design knowledge and the development of methods for supporting engineers’ information needs.