The evaluation of the SOLAS SIP project viewed the project as an innovative pilot experiment aiming at the clarification of a model of Assistive Technology (AT) provision, monitoring and support for pupils with a disability that is sensitive to a variety of contextual considerations. These contextual considerations included the nature of the educational setting, the pupil, the range of stakeholders involved, and the wider values of the educational system.

The evaluation approach was based upon a learning oriented model that examined the relationship between the specific innovation the project sought to introduce, its innovation model, and the adequacy of that innovation model in the light of the practices conducted by the project. This approach was supported by interviews with teachers who had experience of the project, interviews with members of the project Steering Committee, extensive discussions with the co-ordinator and a review of project documents and other relevant documents.

The evaluation describes the project's innovation model as composed of five elements. Firstly, AT is a qualitative innovation in that, in the cases covered by the project, it made a substantial ‘functional’ difference. Secondly, understanding and interacting with the context provided by the school setting is essential. Thirdly, practical or ‘lay’ knowledge carried by teachers must be given due weight in making decisions. Fourthly, there is a need to act positively on recent changes in policy principles that have not yet resulted in equivalent policy frameworks and policy implementation. Fifthly, flexible local provision, monitoring and support is essential.

A very positive response to this innovation model was reported by teachers interviewed and significant levels of approval, with some qualifications, was forthcoming from members of the Steering Committee. The Steering Committee response, while generally positive, does indicate that elements of the project's innovation model may be contested in the overall institutional context.

The practices of the project, in conformance with, and in support of, the innovation model, are described and evaluated in the report below. Evaluation is conducted in relation to the criteria of educational value and institutional impact. Educationally, drawing off the opinions offered by teachers, the project is evaluated as a significant success that offers an important base of learning for future AT provision. Institutionally, the project showed good dissemination capacities, engaged in limited concertation activities but constituted a most useful Steering Committee, and made a good attempt to gain institutional influence for its innovation model.

The evaluation concludes by specifying the important learning contributed by the project to the contextual requirements, constraints and opportunities of introducing AT into Irish mainstream educational settings. In the view of the evaluation, it did demonstrate that a good appreciation of the setting is vital to successful implementation and that locally-based technology supply, monitoring and support must be at the heart of future provision.

The project thereby offers an alternative model to existing top-down models of pupil assessment and technology-provision. Its work, both in this light and in the light of its general contribution to clarifying an expanded range of context-based social and educational criteria critical to AT provision, as an element of general ICT provision in education, should be of considerable interest both to educational policy-making in general and to the IT2000 initiative in particular.

This document reports on evaluation activities carried out over a nine-month period on the SOLAS SIP project. This project piloted a regional system of Assistive Technology (AT) provision and support in the south-west region from June, 1999, to June 2001. Assistive Technology refers to any device or system that helps to improve the functional capacity of people with disabilities.

SOLAS SIP was run by Boherbue Comprehensive School, Co. Cork. It was funded by the Department of Education and Science through the Schools IT2000 initiative and, specifically, through the Schools Integration Project (SIP) of the National Centre for Technology in Education (NCTE). SIP projects are intended to help determine models of good practice for the use of Information and Communication Technologies (ICT) across the Irish education system.

The report begins by briefly outlining the evaluation approach. It then documents the basic assumptions that have informed the project and the activities undertaken to operationalize and test these assumptions. In the evaluative section proper, the report considers to what extent the assumptions of the project have been borne out by the outcomes of project activities. On the basis of this assessment, a concluding reflective section considers the learning achieved by the project and considers future implementation prospects.

 SECTION 1: EVALUATION APPROACH

It should be noted at the outset that the scale of the evaluation was reasonable in the light of the project’s own size, but not very large. The evaluation and this report have therefore been narrowed to essentials in an endeavor to clearly report on the learning achieved and future prospects of the project, or related activities.

The first and perhaps most fundamental point relating to the evaluation approach is the value perspective that underlies it. The evaluators only undertook this project on the basis of establishing a degree of general agreement with its objectives and approach. This level of agreement by no means precluded a critical evaluation perspective. Outline general agreement on objectives might well dissipate in the light of more detailed knowledge of the project’s philosophy. It might also dissipate in the event of the project’s acute failure to implement a sound outline philosophy through poor project management or through unanticipated and insuperable constraints in its environment. However, the basic point remains that a small evaluation in the context of an experimental pilot project would be extremely difficult to execute if some measure of outline agreement was not present. The evaluator's first step, therefore, was to establish such agreement by reading background documentation and through discussion with the project co-ordinator.

This leads on to the characterisation of the project as an organizational structure requiring a certain type of evaluation. As a pilot project, exhibiting an approach largely innovative within an Irish context in relation to AT provision for people with a disability, SOLAS SIP clearly could not be regarded as in any way securely anchored in institutional structures and practices. Indeed, the very point of the project was to step outside such practices and to generate insight and demonstration of feasibility of new ideas and organizational frameworks that could advance, but also prospectively change existing philosophies and models of delivery. There was therefore no point in conducting the kind of external assessment against targets and standards appropriate to well-institutionalized activities, nor even of considering the project as a full-scale implementation pilot with evaluation of implementation against criteria of co-ordination, project management and outcomes.

Rather, SOLAS SIP has to be considered as an experimental, or learning oriented pilot project, that generates insight and indicates horizons of feasible innovation. It therefore has to be evaluated against this standard, which requires a looser, more contextual evaluation philosophy. The primary question here is to assess the degree to which a sound understanding of the basic innovation, and the factors influencing its future, wider implementation, has been achieved.

Put simply, the task of the project was to generate a new order of knowledge and to make a case for the wider implementation that should follow from the establishment of this order of knowledge. A basic premise of achieving such a goal was the creation, amongst a sufficient constituency, a shared set of understandings of the value of the innovation. The evaluation documents the degree to which this has been achieved.

The evaluation activities of the project consisted of detailed discussions with the co-ordinator, site visits and teacher interviews, interviews with members of the steering committee, project documentation review and contextualization of the project in a wider literature, policy and societal context. The core of the evaluation was the site visits and teacher interviews, seven in all, conducted with teachers in a representative sample of the live cases of the project. Interviews were also conducted with four members of the project Steering Committee, taken to constitute a representative sample. Discussions with the co-ordinator proved invaluable in establishing the history, philosophy and context of the project and the co-ordinator also supplied important documentation on the activities of the project. The co-ordinator also supplied some of the extensive documentation of the project for review.

TOP

 SECTION 2: KEY ASSUMPTIONS AND ACTIVITES OF THE PROJECT

To speak of the assumptions of an experimental pilot project is not simply to speak of a set of testable propositions to be confirmed or disconfirmed in practice. In this kind of context, basic assumptions are often not wholly confirmed or disconfirmed by the relatively limited activities of a project. It might even be said that, in view of the experimental nature of innovative pilots, that implementation horizons are of necessity only speculative, though of course such speculation can be very confident.

Also important in relation to the assumptions of a project is whether the assumptions are themselves appropriate and adequate in relation to environmental contexts and, therefore, even if only partly demonstrated, can nonetheless be said to be of sufficient standing to justify the commitment of various kinds of resource to their further investigation, preliminary or full implementation.

A further vital dimension of innovation assumptions is that they should translate into commitments of key project stakeholders. These commitments should involve both a willingness to commit material and immaterial resources - finance, trust, energy, persuasion, dissemination - and a kind of value conversion, a principled commitment to the basic ideas of an innovation rather than simply a pragmatic or opportunistic agreement that is dependent on favourable circumstances. The question of adequacy has to do with the setting of the project philosophy in its wider environment, the question of resource and value commitment to do with the degree of shared understanding that the project succeeds in achieving amongst its primary constituency.

If these reflections appear somewhat abstract, they will quickly be brought down to earth when we consider the basic initial, orienting assumptions of the SOLAS SIP project, the various activities that follow from them, and the degree of commitment it has succeeded in eliciting from its core constituency. The basic assumptions of the project, mostly present at the outset but modified and refined through its lifespan, appear to be the following.

• Assistive Technology (AT) is a qualitative innovation for people with a disability. It is not simply a substitutive or ameliorative innovation but one without which a sufficient level of participation in the educational process is made very difficult. A key criterion is the achievement of functional literacy, which in many cases cannot be attained to the same level without AT. AT is therefore greatly more significant for people with a disability than IT competence for the general population, notwithstanding the justifiable importance attached to such competence at a personal, societal or policy level.

• A second assumption of the project is the importance of context. Context manifests itself in a combined educational and technology implementation model that draws attention to the importance of the values contained in educational organization and practices - the ethos of a particular school, the importance of communication, the presence of specific constraints and opportunities. It emphasises the view that the context must be appropriately organized in the first place, and subsequently appropriately monitored and supported in the second place, if the innovation of introducing an AT system is to be successful. However widely implemented AT provision of a given type is in general, the introduction of AT in a public, social setting of a school is always an innovation in that specific setting. It is so because the range of factors that are important to a successful outcome are numerous - attitude and initial capabilities of teachers, perceptions of other students, the effect of AT on the wider classroom setting, perception and aptitude of the impaired pupil, the extent of disability, the range of technological options, the extent of financial resources available, amongst others.

• A third assumption is the significance of practical, or ‘lay’ knowledge as having equal status with expert, or clinical and/or technical knowledge, in the area of AT provision. The idea here is not to devalue expert knowledge as such, as for example clinical assessment of impairment, but not to give it determining status over other kinds of knowledge. This issues, in a way, a challenge to expert knowledge to explore wider frameworks that take on board how variations in social context could effect, even modify, expert judgement. In the case in question, AT in education, the project emphasises the importance of social factors in making pedagogical judgements as well as psychological and technical ones.

Other kinds of knowledge that are important therefore are practical knowledge of how to teach, how to judge perceptions of status, how to choose technology on the basis of the kind of contextual matrix listed above, how to monitor how the pupil fares with the technology and to appropriately adapt it, how to make assessments of appropriate technology to introduce in a given case. The claim to knowledge here is a revisable, multi-perspectival one rather than a typologically ordered list of possible dysfunctions and appropriate technological solutions.

This distinction overlaps with another, relatively well-known one; the distinction between the social and the medical model of disability. The former model stresses how environment constructs disability, often exacerbating the effects of functional impairment, while the latter looks upon disability as largely a medical condition characterised by the absence of certain ‘normal’ physical and/or mental competencies. Both this assumption and the previous one gave substance to a key belief of the project, that any innovative structures on a SOLAS-type model should be fundamentally located within the educational environment and not in organizational environments dedicated to the supply of technology, however sensitive to disability or educational issues these organizations may be.

• Another assumption of the project was that social and legislative change in attitudes to disability, summed up in a hesitant and sometimes contradictory but nonetheless real move from a bio-medical to a social model, amounting to a belated move away from antiquated and inadequate models, has involved a change towards more progressive models, stressing manifest and hidden environmental barriers, full participation and equal rights. This also involves an imperative to develop appropriate policy initiatives if such attitudinal and legislative frameworks are to count as more than fine ideas and words. An assumption of SOLAS SIP is that it reflects the kinds of spirit present in the new thinking today and that its innovation should form part of the realisation of such ideas in actual organizational structures and practices.

• A further assumption of the project, at the outset, was a spatio-organizational one, stressing the importance of the regional and the local as a basis of an information and product repository, flexible delivery, sustained monitoring of experience, and rapid provision of support of all kinds. The idea here owed something to manifest problems with existing institutional structures due to slowness, fragmentation, de-contextualisation, inadequate resources, poor responsivity and others, along with the corresponding need for corrective, responsive structures.

It also owed something to the importance of taking ownership, of building specific coalitions of interested parties - teachers, parents, pupils, and the various assessment, monitoring, technology delivery and maintenance functions of the proposed regional support system within some form of consensual, mobilized community. In that sense, if proposed to build from the grassroots up, to mobilize enthusiasm and hence to create the will to solve problems.

As is common in many experimental innovations in the public or voluntary sector, much depends on the pioneer or pioneers. It was apparent, no doubt in the project as it certainly was in the evaluation process, that the project initiator, co-ordinator and key figure in the project, not alone internalised these assumptions as a set of values and orientations but could also maturely communicate them through written and oral means and by example of practice. In this he was clearly supported by the principal and Board of Management of Boherbue Comprehensive School who facilitated the project throughout and provided important support for the co-ordinator.

The internalization of key values in a consistent way by the initiators or organizers of projects is not, of course, a sufficient condition of success since they also have to find the assent of other stakeholders, who have their own existing beliefs and interests. However, it is an important condition of possible success. We will return to this ‘diffusion’ question below but first we have to document the kinds of practices that the project engaged in, springing from, realizing and testing these assumptions.

Below, we provide an outline of project practices in a number of selected, key dimensions. Before commencing this exercise, a few brief comments may be made about the temporal unfolding of the project. Like many projects of this kind that do not have existing organizational structures or personnel, very considerable time has to be spent in initial set-up activities that provide a basic technology for conducting operations - gaining knowledge, establishing a niche which may involve a number of wrong turnings, becoming known in the specific environment, selecting between goals and trimming or eliminating over-ambitious ones in favour of the more important and most intrinsic ones.

Because the SOLAS SIP project emanated from the earlier SOLAS project, initiated in 1997, and learned from its close association with the DATE project as well as from other activities such as the APHRODITE project, there was undoubtedly some reduction of this kind of time and resource dissipation.

Nonetheless, as the co-ordinator frankly admits, there were some costs of this nature including an operational linkage at the initial stages of the project with another SIP project, the SPEAK project. SPEAK is based on a network of special classes in west/south Cork and its focus is principally on the provision of AT to pupils with learning difficulties, some very severe and some with associated physical difficulties. This linkage was not as productive for SOLAS as was initially anticipated in the view of the SOLAS SIP co-ordinator, firstly, because the area was very specialized and, secondly, because there was an attempt to implement text-book high-tech solutions which were perhaps too advanced for the context and the SOLAS project at that particular time.

In the latter sense at least, this collaboration did provide a base of learning as to what SOLAS SIP should be doing as well as helping it to better specify its own philosophy, based on the contextual approach documented above, and also it clarified the need to focus on "middle-tech" solutions that could be successfully introduced in the cases that the project was involved with.

Though not explicitly specified by the co-ordinator, it is reasonable also to suppose that the hiring, training and then subsequent loss, to another AT agency, of the only project worker in the first year could also count as a set-up cost. Twelve months does not seem an adequate time period to get an adequate pay-back on training investment, though the co-ordinator has praised both the professionalism and enthusiasm of the individual involved.

Loss of personnel after a training period is a classic problem with innovative organizational forms whose instability cannot offer long-term career paths, but whose innovative nature provides good training and, for very understandable reasons, individuals may choose to move to organizations that offer them longer horizons, better rewards or other incentives.

The loss of the project worker, which in this case could not have been avoided because of the project's funding structure, led to the co-ordinator taking over direct responsibility for the project, which certainly involved him putting in more time on the ground and actually may have contributed to the success of the project as instanced by the extraordinarily positive view of the project amongst the teachers interviewed (see below).

These specific ‘costs’ of set-up were therefore not utterly without value and, though they slowed the project, they did not really deflect it from exploring its key objective of demonstrating the practical feasibility and pedagogical-contextual appropriateness of its model of regional, flexible provision. What they probably did was place a huge burden on the co-ordinator both internally, in relation to project organization and administration, and externally in relation to all facets of provision. In any case, the project's wide range of activities continued and these can now be documented.

The first of these activities was the constitution of a body of empirical and orienting knowledge. The project needed to constitute its own knowledge framework, not least because there is no register of people with disability and a generally under-developed research focus on the relationship between disability and educational opportunity and experience. In speaking of empirical and orienting knowledge, we signify not the learning emanating from the project so much as the inputs required to get started, to develop a methodology, to investigate the range of possible cases and to specify and categorize the cohort of cases that is to be considered in detail.

In relation to the population selected for consideration in this report, the project decided, at the beginning of its second year, to concentrate on primary school children. This concentration arose from the fact that the earlier SOLAS project, along with its association with the DATE project, had provided a good understanding of the second level situation and that, in any case, the current project could not satisfy the anticipated demand that would arise from making its services available to the second level sector.

The project sent a circular to all primary schools in Cork and Kerry in late autumn 2000, outlining the project aims and services. The responses suggested an apparent need for an integrated physical/learning type service that could support the activities of newly-appointed ‘resource’ and ‘learning support’ teachers and, ideally, also to extend the competence and capacity of such teachers, along with ‘mainstream teachers’, in dealing with AT for students with disabilities.

In addition to this categorising knowledge gained by responses and enquiries from a circular and reflection on the characteristics of its body of cases, the project was also able to draw off student-centred investigations (e.g. Hidden Voices) and broader educational perspectives deriving from the academic work of the co-ordinator on the implications of context, expressed in value preferences, school ethos, pedagogy, class atmosphere, ergonomic and physical lay-out considerations, psychological issues, technical change, equality and participation issues, and others.

What does not appear to have been successful is any attempt to integrate the contextual approach with more clinical, de-contextualised approaches of bodies who conduct centralized assessments of pupils' abilities. The project’s emphasis on alternative sources of practical knowledge would involve a discourse on appropriate divisions of authority between contextualised, practical knowledge and such de-contextualised formal-theoretical bodies of knowledge.

Notwithstanding, as can already be gleaned from these remarks and the depth and maturity of the assumptions underpinning it, the project was well-served by its pre-existing and evolving knowledge framework which clarified what it was for and what, by and large, it was against - rigid technologies of every kind, bureaucratic, procedural, assessing and assistive.

The second activity to be considered is that of pupil assessment which, in the case of the project, is hard to distinguish from implementation. While the co-ordinator expressed unease with the use of the word ‘assessment’, as it suggests ‘expert’ intervention and decision-making rather than the collaborative model envisaged, the term is retained in use. Assessment was conducted in a contextually sensitive manner and initial implementation of provision followed from it. Three words are probably adequate to sum up the assessment philosophy of the project - contextual, dialogical and consensual.

In relation to the contextual dimension, the assessment typically sought to involve all concerned parties, one or several teachers depending on who had a stake, the pupil, the parents, the personal assistant of the pupil if relevant, the school principal if relevant or necessary.

But above and beyond the human agents, assessment was also directed at the physical situation - lighting, classroom lay-out, ergonomics and others. The key dimension to this activity, as stressed by every single teacher interviewed, was that the assessment took place in the school setting. This locational choice was regarded as important by the teachers because it was real and practical, but also because all parties could be present and could understand, and buy-in to the proposed approach. In other kinds of assessment, conducted in distant settings, it can be very difficult for the teacher to attend due to classroom or other commitments. .

The dialogical dimension follows from the constitutive contextual one and involved an attempt not simply to involve stakeholders but to get them to communicate from their respective standpoints. The consensual dimension involves the constitution of technological solutions and associated practices that could gain the commitment of all. The last dimension proved more difficult to realize as there was some disagreement on the proper role of parents in the educational process, as we will come back to below. The assessment process tried to have available a good suite of hardware and software technologies so that different options could be tried relatively quickly. The assessment process also considered the child’s medical history, where relevant. The general impression created by research for this project evaluation, confirmed by the co-ordinator, is that assessment was largely confined to physical conditions such as Cerebral Palsy and Spina Bifida, and sometimes associated learning difficulties.

SOLAS SIP did aspire to explore a potential working relationship between a central national agency and the project, as a regional agency, as a potential national model. However, in the first relatively complex case where SOLAS SIP collaborated with a central national agency, that agency did not include the project in any further involvement in the case, even though SOLAS had co-ordinated the assessment procedure at a local level, involving also an existing regional service provider. Involvement with another central agency, in this case a specialist hospital, did result in a successful collaboration, with a broad strategy being developed jointly and the detail being implemented locally by the project.

The wider dimension of implementing follows closely on from assessment. The specific assessment activity carried out by SOLAS SIP was not an in-depth assessment of the child’s physical and psychological conditions so much as their manifest capacities to use specific kinds of assistive technology. The implementing activity that arose therefore involved the identification of possible technologies and strategies, on the basis of experience, testing and discussion with staff, that were judged to suit the child’s needs and the child’s educational context. The choice of which option might work best was usually then reached by consensus, with the teacher sometimes making the final judgement. Where Department funding was required for the technologies, the project then ‘prescribed’ a suite of hard and soft technologies in a formal report that conformed to existing practice.

This activity of the project, in a sense its absolute raison d’etre, drew much of its purpose from the manifest difficulties most teachers demonstrated in relation to knowledge of what options were available and in relation to the actual customizing of these technologies to the child’s and the educational setting’s best advantage.

Very often, quite simple interventions would be made by the co-ordinator – slowing the speed of a standard mouse, recommending continuation with a standard mouse rather than a special one for status reasons, exploring alternatives to potentially ‘intrusive’ voice technologies – that would otherwise not have happened. These apparently simple interventions, however, were very important in ensuring the real effectiveness of the technology in the given case and also, just as importantly, its personal and social legitimacy. Of course, more complex solutions were also recommended on the basis of knowledge of new product offerings by technology suppliers.

Though some teachers had good training and networks, they still required the kind of specialized insight and advice that the project was able to supply. In those cases where isolation was greater and training and network benefits less advanced, the project was able to suggest implementation solutions that it would be difficult to envisage occurring without it. It was also able to pre-empt the ordering of the wrong kind of equipment through the Department of Education and Science’s existing systems, a problem compounded by delays in supply and the slowness and difficulty of rectification.

A significant feature of the project’s implementation activity was the ability both to go to the classroom setting – already noted above – and to bring technological solutions to this setting. The latter, of course, took some time. Even though the project had its own bank of equipment for assessment, and some for loan, it did not have a stock of ready to supply equipment stored locally and schools had to go through existing Departmental procedures to acquire it. However, this did appear to radically speed up delivery of AT which seems to have moved to a six week cycle when the project was involved as opposed to a cycle of up to a year when it was not.

The latter timescale, according to most teachers, often had a deleterious effect on a child’s development, as the impaired pupils could not effectively participate in class activity, falling behind educationally and running the risk of being stigmatised as ‘backward’.

A further related activity of the project was monitoring activities. Monitoring here does not mean assessment of educational outcomes – this was left to teachers and the wider educational process – but rather monitoring of whether the technology did really allow the pupil to participate up to a satisfactory level in the learning process and judging whether further modifications would prove effective. Monitoring in this sense was an important project activity with the project co-ordinator regularly being requested to offer advice on a particular problem. This advice-giving often took the form of follow-up visits where particular issues or problems of implementation were addressed. Availability in this sense could be considered an important activity of the project as teachers alluded repeatedly to the value of having an expert on-call who would take responsibility to address the particular issue or problem.

The project envisaged a further, more longitudinal form of implementation and monitoring that would prepare for, and track, pupils’ movements across different levels of education, e.g., from primary to secondary levels. However, for reasons beyond the project’s control, this did not prove possible.

A further activity of the project was that of concertation. This mainly amounted to putting together a representative steering group and using this group as a forum for developing ideas, assessing ideas and practices and opening institutional doors. This Steering Committee was undoubtedly representative and, judging from its composition, interviews with its members and reading its minutes, performed its function well. However, it was hard to generate specific co-operation activities even in cases where members of agencies were present on the Steering Committee and largely sympathetic to the project. It was also hard, as it generally is in this sector, to get Department of Education and Science representation or feedback for concertation activities.

The co-ordinator indicated that one of the main values of the Steering Committee was the contribution of key individuals, on a needs basis, outside of the formal Committee. These members also formed an important cushion and sounding board, without which the work would have been conducted in relative isolation.

One of the most significant areas of the project’s activities was its dissemination activities. Not alone has the co-ordinator contributed numerous papers to appropriate seminars, settings and conferences, but he undertook to produce a path-breaking guide to the use of assistive technologies in the Irish context. He also evidently engaged in making the project well known in a whole variety of relevant forums. The circular distributed to all primary schools made the activities of the project known across the South West Region. Finally, the project publication, Enabling Technologies, and its web-site, are significant dissemination tools.

A further activity of the project could be described as its attempts at institutionalization. This activity, the attempt to stabilize the project by embedding it in institutional structures that guarantee access, resources and legitimacy, was an important goal of the project. This task is, of course, ongoing, and at time of writing it is unclear what institutional options will lie before SOLAS. However, the will of both the co-ordinator and the most immediately affected constituencies is for SOLAS to continue, which is not at all in our experience a probable outcome of pilot experiments of this kind. To that extent, the Steering Committee can be seen as in part an attempt to build in institutional relevance and to open a pathway to wider institutionalization.

The other major activity related to the project was the proposal sent to the Minster for Education and Science, in late 1999, emanating from the South West Regional Authority, Boherbue Comprehensive School and the Central Remedial Clinic, for the establishment of a pilot national/regional assistive technology service. This proposal effectively built upon previous project work and encapsulated many of the assumptions of the SOLAS project in relation to flexibility and speed of delivery, and the importance of an adequate support environment.

It can be clearly seen from this document that SOLAS SIP saw its institutional relevance as running along with broader agenda change, expressed in such developments as a greater emphasis on equality and fairness for people with disabilities in policy circles, the reconceptualisation of disability as a social rather than medical issue, and the experience of technology diffusion which, recurringly in disability and non-disability areas, moves from a kind of utopian view of the positive implications of new technologies to a more sober realization of the social organization and support systems that have to be built to make the technology really useful, and sometimes even just usable. No reply to the proposal was forthcoming from the Minister.

TOP

 SECTION 3: EVALUATION OF PRACTICES AND ASSUMPTIONS

The evaluation framework developed in this document distinguishes between assumptions and practices. The assumptions of a given project define its innovation model while practices follow more or less tightly from this innovation model. In the case of some projects, practices follow intrinsically from the innovation model, in other cases they significantly depart from it. Practices are never simply slaves to the innovation model but extend and modify it. Often practices involve the de-emphasis of core ethical or evaluative assumptions contained in the innovation model because the affected constituency might not share them, or even be hostile towards them. The practices of a project could therefore be quite successful without confirming the innovation model, opening the possibility of a disjunction between what a project does and what it wants to do.

This in turn opens the possibility of introducing a quite different innovation model that could largely deliver the desired practices of the constituency and which might come from outside of a particular project. On the other hand, practices and their results that confirm the innovation model are likely to strengthen both the identity and perceived relevance of a project.

For reasons of simplicity and coherence, the practices of the project outlined above are divided into two basic categories, those that have to do with their educational value and those that have to do with public and institutional impact.

The criterion of educational value in the context of this report has to do not simply with pedagogical contribution but also, in the case of SOLAS SIP, with organizational forms of assessment, delivery and monitoring. While the project understands itself as primarily concerned with the latter, there is no doubt that important pedagogical judgements are also made about students’ and teachers’ capacity to deal with certain technologies.

However, there is an important distinction to be made between judging the appropriateness of technological innovation in interchange with the overall educational context and making judgements that are pedagogical in the first degree, i.e., that decide pedagogical appropriateness on the basis of the assumed fit between characteristics of the technology and the user. That distinction is critical because the former approach leaves fundamental pedagogical questions to teachers, including the question of whether the technology actually works in the real setting, while the latter prescribes how it should work on the basis of the fit between the characteristics of the technology and that of the learner. In this latter model, then, decisions are taken by experts without extensively taking into account the socio-educational conditions in which the technology is used.

In relation to pedagogical questions considered in the dual sense of both the perception of the need, in specifying actual configurations of technology, to take full account of the socio-pedagogical conditions in which it is used, and the approach taken to doing this by this project, the judgement of the teachers was unanimously positive. The most basic question that teachers had, in relation to SOLAS SIP or any other possible intervention, was whether the technology/context arrangement could enable the student to participate in class. The question was quite stark in most of the cases examined in the evaluation: Can the student participate in class after the intervention in circumstances where they could not before it? The answer to this question in the majority of cases was that the project did make that difference and that its contextual, situated approach was the main factor in it being able to do so.

It could be argued that it did so because it was able to arrange the supply of equipment more rapidly than conventional channels. This was indeed one of the main benefits of the project and is intrinsic to its rationale. But, beyond this, teachers were clear, even in cases where they were both well networked, resourced and trained, that the extra familiarity with the technology and the additional appreciation of how various contextual and psychological factors intertwine in its use was an important, additional advantage offered by the project. The evaluation would claim that these judgements were fundamentally pedagogical, from assessment through to implementation and support, and legitimately so. The legitimacy rests on the claim to validity of the socio-pedagogical approach in general, the insight generated by experience of many similar cases at similar levels of complexity, through acting in co-operation with teachers who have the status as experts in their own right, and by a willingness to co-operate with other bases of expertise. It also derives from the project’s willingness to concentrate on the middle-range of cases - physical disability with, at most, mild learning difficulties, in which the subjects and variables were relatively well known.

The question does arise, however, given the circumscription to particular kinds of cases, as to whether the SOLAS SIP model alone could be the basis for a universal service. We will take up this question below.

A vital point about the judgement of the educational value of the project, as made by teachers, was that the willingness of the project to come to the educational setting was perceived as vital to its overall contribution. The act of going to the school had the effect of bringing together, in one place, the range of relevant actors - teachers, pupils with a disability, other pupils, project and parents, and who were each important stakeholders in the overall success of the innovation. It had the wider effect of giving a personal feeling to the service offered in the setting rather forcing these actors to entirely depend on distant experts and organizational structures, which could not be appealed to in the same ‘reflexive’ way. When teachers encountered problems, the project co-ordinator could be relied upon to make himself available at relatively short notice.

The very clear sense from teacher interviews was that this change of emphasis gave them a sense of ownership over the provision process that the normal approach could not equal. The model facilitated the provision of services by taking account of their own inputs. The fact that they were entirely satisfied with the response of the project when they requested back-up and support of various kinds does not preclude the fact that they could also express dissatisfaction were the response not forthcoming in a reasonable time frame and to standard. It should be made clear, at this point, that it is not the evaluation brief to criticize bureaucratic organization per se, which in many contexts has distinct advantages, merely to observe on teachers’ perceptions of its inadequacy in the case of AT provision for pupils with a disability.

If the action of the service supplier going to the school involves an inversion of the previous relationship between space and power, then the greater urgency of the project’s time scales involved an inversion of the previous relationship between time and power in the case of the schools investigated. The project’s time-frame derived from the urgency generated by the case, a pupil unable to participate fully, or in some cases at all, in the early, critical experience of education because of delays in assessment or in technology provision. In the existing system, the first order delays in getting assessment and resources for appropriate equipment is often compounded by second order problems with the inappropriateness of the technology in use and by the almost complete absence of satisfactory support.

The mode of assessment and provision currently in mainstream operation is simply too slow, too inflexible and too much lacking in appreciation of context and in adequate support in context, in the opinion of the teachers interviewed. This view was held by almost all of the teachers.

This perspective, of course, also raises questions about the role of technology suppliers and the inadequacy of their wider support systems. It appears at least plausible that schools are not in the best position to insist on quality given that their individual transactions are normally not extensive in this area and do not have significant revenue implications for the technology suppliers. This in turn leads to the question of whether an intermediate layer of service provision – between Department and schools - of the kind proposed by SOLAS SIP, would involve some improvement in the relative bargaining position between the public sector and AT vendors. One member of the Steering Committee of the SOLAS SIP project was firmly of the view that this would be the case. This is, of course, additional to the primary question of whether it would improve the appropriateness, flexibility and timing of supply?

Perhaps the biggest question arising in relation to the educational practices of SOLAS SIP is the uncertainty attached to the role of parents. On the whole, the opinion of teachers would be that parents could have a background, supportive role but no real role in the educational process as such. It is really beyond the brief of this evaluation to consider this question in depth, as to some extent it lay beyond the brief of the project itself which sought to involve parents in a consensual, stakeholding framework. However, there appears to be a significant issue about the appropriate role of parents in the educational process that has to do with professional roles, ownership of technology, and the precise educational support parents can provide.

The question of impact under consideration here is not that of capacity to shape emerging policy directly so much as the attempts made and strategies adopted to influence the policy agenda. Any judgement of activities of this kind must take centrally into account the status of the project as a small, experimental pilot. In many ways, notwithstanding the best efforts of projects of this kind at demonstrating need and disseminating results, it actually takes others to make fundamental judgements about what such a project contributes. The outputs of projects of this kind therefore depend on constellations of interests, prevailing policy beliefs, and topicality of the issue (policy urgency).

Taking this consideration into account, the evaluation viewpoint is that SOLAS SIP also performed very well in this regard. All the major players in the field, including the Department of Education and Science – the Minister wrote a foreword to the Enabling Technologies publication, the previous Minister was in receipt of a proposal as outlined above, the Department received regular reports from the project – are aware of the project and its activities. The Steering Committee was evidently helpful in this respect but, in turn, the Steering Committee was well constituted by the project. The booklet produced by the project was of high quality and badly needed in the area. In addition, interesting and pertinent seminar contributions and other presentations have been delivered by the project co-ordinator.

Beyond formal matters, there is good evidence of intensive informal networking done by the co-ordinator. The members of the Steering Committee interviewed claimed good knowledge, not just of the wider policy issues relevant to the project, but also of its technical activity. It is reasonable to suppose that this knowledge was correspondingly diffused amongst the specific networks of these members.

Five assumptions of the project that constituted its innovation model were outlined above. These are:

1. AT is a qualitative innovation
2. the importance of context
3. the importance of building in practical as well as theoretical knowledge
4. the necessity of acting on expressed policy principles
5. the importance of flexibility and locality in delivery of services

Of these five, teachers tended to support them all as did some members of the Steering Committee. Inferring from interviews and wider knowledge, it is probably that there is significant support at the level of special needs teachers and sectoral organizations for (1), (4) and (5) but (2) and (3) are more likely to be disputed.

In relation to (1) there is general agreement that AT can play a vital role if it is appropriate in the first place – not always the case, even where it is recommended - and if it is pedagogically and materially supported after initial implementation in the necessary ways. In relation to (4), there is a widespread perception that policy principles have been laid out and enacted in legislation but with as yet little sustained action to implement policies that match these principles. In the view of some, certain escape clauses in the legislation actually facilitate delay or inaction in turning principles into implementation practices. This especially applies, in the view of one Steering Group interviewee, to the primary educational sector where it would be hard to identify any clear policy line, outside of the availability of equipment funding.

So, there is no real question that implementation must be advanced to catch up and, possibly in time, to redefine policy principle, e.g., a greater move than is presently the case from the medical to the social model of disability. There is also general agreement in relation to (5), emerging from (4), that local, flexible provision is also a necessity, even if this were to prove relatively costly. The psychological and material benefits of having a service close by, and of not having to travel to distant places for once-off assessment, tended to receive broad agreement.

It would the assumption of the evaluation, partly supported by the interview process but mainly by observation of wider developments, that the main questioning of the assumptions of the project would lie in assumptions (2) and (3). The combination of contextualized service provision and the importance of practical knowledge does offer a challenge to the existing organizational model of mainly top-down provision and also to the existing model of expertise, primarily anchored in technical and psychological roles.

The real division of opinion may well lie in basic models of educational organization and their pedagogical implications, that is, between those who support a centeralised model of organization with some degree of regional flexibility and those who support more radical flexibility and wider community mobilization. This difference informs debates such as what is assessment, who should carry it out, and what constitute adequate standards or, as to what is the respective role between technology providers and those who may be classed as educationalists, or between educationally qualified technical specialists and psychologists.

This division, which appears to be important to the future of SOLAS and similar initiatives, may well not be an either-or choice as elements of both positions can well be accommodated and, indeed, the project has been keen to recognize the validity of expertise in complex cases. However, what is the case is that there is both uncertainty and opportunity in the wider institutional environment of educational policy implementation for people with disability, which clearly makes for turbulent and ‘political’ times in this field.

Important to the SOLAS SIP position on these matters is that the regional, flexible delivery and support system proposed should lie within the educational sector. Its argument for this has to rest on pedagogical grounds and opposes the idea that technical expertise, or technical and psychological expertise combined outside of the educational sector proper, should offer the basis of the service. This is a position that would probably in turn be opposed by others.

The general evaluation perspective is that the assumptions of the project have been well formulated, bear a true and meaningful relationship to its practices and receive from some constituencies – notably teachers, but also others – support on all the key elements while, from others, support would probably be forthcoming only for some of them.

An interesting perspective arising is whether these disputed assumptions, which interface with wider institutional differences and uncertainties, will have to be addressed as central issues in future debates on the relationship between technology and pedagogy in education as a whole and in disability-relevant education in particular.

TOP

 SECTION 4. PROJECT LEARNING AND INSTITUTIONAL PROSPECTS

It is the viewpoint of the evaluation that the project has succeeded in generating an important base of learning concerning its central focus, the existing situation and future options for appropriate delivery and support of AT services for the educational needs of students with disabilities. This learning has been both negative, a salient critique of the existing situation, and also positive, a set of cases and publications that at the very least illustrate some important micro-level practices that will need to be taken into consideration in any future initiative, whether explicitly based on SOLAS SIP or other.

In relation to its critical import, the position of the project in relation to problems of inflexibility, slowness, lack of support, distance of service providers, lack of co-ordination, lack of training, inadequate numbers of special needs teachers, and insufficient special needs training amongst other teachers, and inadequate ICT training in general, tend on the whole to be confirmed by teachers, members of the Steering Committee and wider reports of various kinds.

As illustrated above, not every level of SOLAS SIP’s innovation model is likely to be supported by all constituencies but at least each of its assumptions should give rise to interesting debate. On the positive side, the most important contribution of the project has been the demonstration of how flexible IT delivery and support could actually work in educational settings. In this connection, it has provided vital ‘bottom-up’ insight into cultural, psychological and practical aspects of providing a service. In these dimensions, the project provides a significant base of learning that should be further availed of.

Already, however, some judgements can be made. It appears to be generally agreed that the project offers illustration of how one model of delivery and support could work. It therefore offers the basis for either affirming or critical reflection on what schools, students, teachers, parents and other stakeholders need in the light of the project’s assumptions and practices. Whether such reflection is affirmative, reconstructive or even opposed to the project’s innovation model, the project offers a standpoint for debate that is grounded in real decisions, practices and outcomes.

The key issue facing the project at the present time is whether it is to continue and the possible form of that continuation. There are a variety of options ranging from continuing as is, being more directly administered by the NCTE, becoming mainstreamed in some form by the Department of Education and Science, becoming associated with an NGO, or attached to a regional Education Centre. It is beyond the scope of this evaluation to comment on the merits or demerits of any of these models beyond expressing the general view that the learning encapsulated in the project should be retained in future institutional designs and practical actions.

There is one point arising, however, that does lie within its brief. Flexibility and openness to learning between people with different competencies and with different stakeholder positions should as far as possible be retained. This would suggest that a continuing learning model, remaining open to the lessons of innovation, should be preferred before the structure is given a permanent institutional shape.

The evaluation would recommend that the following dimensions of the project are particularly important to the related questions of future institutional designs and practical activities.

The first arises from the desirability of extending the project’s contextual, inclusive model from the demand side to the supply side. That is to say, different competencies ranging from special needs teachers, to psychologists, to educational AT providers, to pedagogues, to ICT specialists, to those with special knowledge of particular disabilities and others, should be enabled to fruitfully interact and provide a balanced service, even if the core of that service, in the case of AT provision, is to be provided by the educational AT provider on the SOLAS model.

The second is that the delivery of any services that follow from advances in supply side co-operation must involve a sea-change in current thinking, involving a re-balancing of priorities from excessive dependence on bureaucratic forms of decision-making, the judgement of distant experts, excessive dependence of technology suppliers who do not provide adequate support, to a greater appreciation of how ‘assessment’, implementation and support must take account of the school context in which the pupil is located.

The third is that practices in any such future model should be ‘reflexively monitored’ so that learning is maximized, much in the way that SOLAS’s activities have already led to a publication that makes a considerable contribution to the area.

The fourth is that SOLAS SIP, in its ideas and approach, transcends the current state of policy and provision in Ireland and implicitly supports the idea that any innovation model applied in the national case should draw from models and practices already implemented in other jurisdictions in the area of using AT in special needs education.

TOP