Consulting Services

An article published in the Journal of Telemedicine and Telecare, Vol. 2, No. 2, 1996, pp.81-86 (reproduced with permission of the Royal Society of Medicine Press)

Authors Benjamin Mitchell*, John Mitchell*, Dr Alex Disney**

*John Mitchell & Associates, ** Renal Unit, The Queen Elizabeth Hospital, South Australia, Australia.

Summary

We carried out a longitudinal study to evaluate the users’ attitudes to the introduction of telemedicine into the dialysis units of a renal ward in South Australia. The first questionnaire was distributed to all members of staff involved with the introduction of the system. There were 44 responses (80%). Staff were fairly positive about the telemedicine system, and felt that it was easy to use and reliable. They also clearly felt that the confidentiality and privacy offered by the system in an open ward were unsatisfactory. A second questionnaire was distributed to all staff about six months later and there were 40 responses (66%). Of these, 22 could be matched with the responses from the first survey (a response rate of 50% from the first sample). There were no significant changes in staff members’ feelings between the two surveys, except in two cases: there were significant changes in staff opinion about the degree of confidentiality (P<0.05) and privacy (PP<0.01) offered by the system, with attitudes becoming more positive in each case. The results indicate the need for dialogue with users, in order to address their concerns regarding the system and practical difficulties. This study highlights the importance of planning, effort, cooperation and an appropriate culture within a renal unit in order for telemedicine to be accepted.

Introduction

The Queen Elizabeth Hospital’s Renal Dialysis Telemedicine Project was undertaken from May 1994 to June 1995 and included the installation of videoconferencing facilities at its four renal dialysis centres at Woodville, Wayville (10 km from Woodville), North Adelaide (8 km) and Port Augusta (300 km)(Fig 1). The renal unit was dialysing a total of 145 patients at these four centres, with each patient normally dialysing three times per week and attending an outpatient clinic once every two months.

Fig 1 Location of the four sites involved in the project

Initially, four videoconferencing units were used (PictureTel System 4000), which were connected to document and miniature probe cameras. After the first year of the project four more desktop units were purchased. Approximately A$450,000 (where A$1 is 0.6 ECU or US$0.75) funding was provided by the South Australian Health Commission in November 1993.

The aims of the telemedicine project were to assess the feasibility and cost-effectiveness of telemedicine as a means of improving the quality of patient care, to determine the need for the further education of dialysis staff, and to monitor dialysis processes and equipment at sites remote from the main dialysis institution. These aims were later expanded, based on early experiences with the project, to include the development of strategies to accelerate user adoption and to maximise both the number of users within the Unit and the breadth of telemedicine applications. An additional aim was to assess the value of desktop videoconferencing for clinical consultations.

A formal evaluation was carried out to assess staff members’ attitude towards the videoconferencing at the time of installation, as well as the change in their attitudes after six months of experience with the system. The surveys were used by the project management team to identify key areas of staff concern, and to measure the effectiveness of interventions designed to reduce these concerns.

As reported by other authors (1) there have been few studies dealing with the acceptance of telemedicine by physicians or users. Those studies which have been conducted have typically had small and restrictive sample sizes, often with under a dozen users (1-3). Of these studies many have questioned the project management staff, or practitioners who have not used the technology (4). Typically there has also been a focus on the specialist or physician with many studies ignoring allied health or nursing staff.

Methods

Participants

A longitudinal survey was carried out.. The first questionnaire was distributed in December 1994 and a follow up questionnaire was distributed in May 1995. The initial survey was conducted at the main centre at Woodville, and the Wayville and Port Augusta satellite centres. The final survey also included the North Adelaide satellite centre. The participants included all renal unit staff who would use the systems - consisting of physicians, nurses, allied health staff and technicians.

A two-page questionnaire, using both open- and closed-response questions, was designed by the project management team. The questionnaire requested information, using closed-response questions, about the following:

• importance of the telemedicine system to the unit;
• number of times the system was used, along with current and expected frequency of use;
• perceived ease of use, ease of access, reliability, confidentiality and privacy of the telemedicine system;
• perceived effectiveness of the induction program and organisation of the network (e.g. timetabling, site co-ordination).

The closed questions dealing with attitudes were assessed using a seven-point Likert scale. An example was the question ‘How reliable to do you believe the system is?’ with respondents indicating their response on a scale from 1 (‘very reliable’) to 7 (‘very unreliable’). The directions of the anchor points was varied to reduce possible response bias.

Open-ended questions were asked about practical difficulties experienced with the system and suggestions for further improvement. In the initial survey closed questions were also asked about benefits, both personal and organisational, and initial concerns regarding the system. An example of this type of question was ‘Please indicate your three main areas of concern regarding the use of the system’, with subjects writing their response on the sheet provided.

Procedure

The questionnaires were intended to be distributed to all potential users of the telemedicine system, although in practice a subsample, chosen for convenience, was selected. Participants were asked to complete the questionnaire in their own time and return it to the project officer. The data collection lasted approximately two weeks.

In order to obtain a more complete understanding of staff members’ attitudes and beliefs about the telemedicine system, it was decided that the survey should be confidential. In order to compare responses between the initial and final survey, a coding system was used (participants recorded the first three letters of their month of birth and their mothers’ maiden name)

Results

The responses to both the open and closed questions were analysed from the first survey in order to gain a complete picture of staff members’ beliefs about and attitudes towards telemedicine. In the second survey only the closed responses were analysed in order to gain an understanding of how beliefs and attitudes had changed.

The first questionnaire was distributed to all members of staff involved with the system, approximately 60 people. There were 44 responses, a response rate of about 75%. The second questionnaire was also distributed to all staff, and there were 40 responses (66%). Of these, 22 could be matched with the responses from the first survey (a response rate of 50% from the first sample). A summary of the responses from the first and second surveys is presented in Table 1.

There were several reasons for the lower response rate to the second survey. First, the renal unit experienced high staff turnover, and large numbers of staff had moved into non-dialysis wards of the unit. Secondly, there were problems with the coding system, with some staff completing it and some completing it in an indecipherable or incorrect way (12 responses).

The responses from the first survey indicated a wide range of perceptions about how frequently the staff would use the telemedicine equipment (Table 2). Nearly half the staff said that they would use the system at least once a week, while about a third were unsure of the frequency. Thus most staff expected that the telemedicine system would become a part of their weekly routine, with some staff waiting to see the applications of the equipment before committing to a particular level of future use.

Table 1 Summary of the respondents to both surveys

Table 2 Expected future use of the system

The responses from the first survey indicated a wide range of perceptions about how frequently the staff would have used the telemedicine equipment (Table 2). Nearly half the staff said that they would use the system at least once a week, while about a third were unsure of the frequency. Thus most staff expected that the telemedicine system would become a part of their weekly routine, with some staff waiting to see the applications of the equipment before committing themselves to a particular level of future use.

The responses to the closed-response questions are summarized in Table 3. The results indicated that staff felt fairly positive about the telemedicine system, and felt that it was easy to use and reliable. They also felt that the induction programme was effective. The staff felt on average that they had barely adequate amounts of time during their duties to practice using the system. The staff clearly felt that the confidentiality and privacy offered by the system in an open ward were unsatisfactory..

Table 3 Means for variables assessed in the initial survey

Note: all variables were measured using Likert scales (range 1-7), where higher scores represented more positive appraisals, and where a score of four was considered average or mid-range.

The most frequently cited benefits for the individual respondents are presented in Table 4. Factors most often mentioned related to increased communication between sites, easier access to staff and patients, and improved education. A secondary set of personal benefits seemed to involve the application of this increased quality of communication to problem solving and clinical management, resulting in an improved quality of patient care. An interesting finding was the perceived personal benefit arising out of the opportunity to work with a new technology.

Table 4 Main personal benefits perceived from the use of the telemedicine system

Staff suggested a smaller range of benefits that the technology would bring to the unit than to themselves (Table 5). The main benefits mentioned were administrative (travel, time or money savings) and those to do with improving the skills of the staff of the unit (through greater communication, education, access and cohesiveness).

Table 5 Main benefits to the dialysis unit perceived from the use of the telemedicine system

The major areas of concern for staff were mostly related to the physical nature of the equipment and the ward environments were it was used (Table 6.). The most commonly mentioned areas of concern were confidentiality and privacy. Written responses indicated that staff felt awkward using the system when it was placed in the ward because the conversation was audible throughout the ward. Some of the centres had alternative rooms outside the general ward area. However, this strategy generated its own problems, as a secondary set of concerns related to equipment mobility, because the cabinets were heavy and difficult to steer. For these reasons the videoconferencing equipment was sometimes not taken out into the dialysis areas, but parked in withdrawal areas set up for private telemedicine consultations. A final set of concerns were related to a lack of information about the videoconferencing system and its intended uses ( expense, reduction in personal contact). It is interesting that the technology itself was not the major concern of staff, with only a few concerns over its clinical effectiveness (quality and adequacy for acute problems).

Table 6 Main areas of concern about the use of the system

As well as concerns about the use of the system, staff were asked to identify the major problems they encountered when they actually used the system. Responses indicated that issues to do with the integration of the unit into the clinical setting were the major difficulties (mobility, lighting, privacy, position of equipment) (Table 7). A secondary set of difficulties concerned education and training (lack of practice time, scheduling difficulties, and requests for more information about the operation and use of the equipment).

Table 7 Practical difficulties listed by respondents

Staff indicated that their preferred method of becoming more confident and consistent users of the equipment involved more use and practice with the equipment, modification of the equipment to improve its mobility and privacy, and improved co-ordination and education (Table 8).

Table 8 Suggestions from users about how they could become more confident and consistent

Table 9 Variables measured in the initial and final survey ( n is the number of matched pairs)

Responses were compared between the initial and final surveys using a paired t-test (Table 9). There were no significant differences in the staff members’ feelings, except in two cases: there were significant differences in staff opinion about the degree of confidentiality (P<0.05) and privacy (P<0.01) offered by the system, with attitudes becoming more positive in each case.

The two surveys revealed that most staff felt positive about the telemedicine system and felt that it would become a part of their working routine. The major concerns were to do with integrating the physical units onto the working areas of the dialysis centres. The initial induction programme was considered effective, although staff continued to express the desire for further training and experience. Finally, the issues of confidentiality and privacy were more often expressed as attitudes towards using the system rather than as practical difficulties that had occurred with use. With experience and some interventions described below, attitudes improved.

Discussion

The responses to the surveys demonstrate the need for the following processes when introducing telemedicine technology into clinical wards: training and support for increasing usage, modifications to the videoconferencing of boardroom-oriented equipment to serve in a clinical environment, management structures to facilitate change, evaluation and feedback cycles in the establishment of the technology.

The surveys were invaluable to the project management team and provided the following successful interventions: introduction and continuing investigation of the most comfortable and functional headsets for use by the patients and staff, the addition of larger wheels to and the removal of the glass doors from the cabinets carrying the telemedicine equipment, and the provision of more training opportunities for staff. These strategies and the problems they have addressed are discussed below.

One of the major challenges for telemedicine is for the technology to become easily adopted on wards. The many benefits of telemedicine will be realised only occur if the technology can be easily integrated into existing work practices. This study highlights some of the steps that need to be taken to achieve these benefits. The user is often overlooked in much of the discussion about telemedicine. Many of the issues involved in establishing videoconferencing in the present study were to do with human factors rather than problems with the technology. There was very little question that videoconferencing could be used successfully.

There were several issues underlying the initial concerns of staff. Some related to attitudes to the technology, and others to problems with the design of the equipment. These problems were dealt with by two strategies: in order to overcome negative attitudes and beliefs, education and dialogue were used; in order to overcome equipment problems, modifications were made to the designs to make the equipment more appropriate to a clinical setting. It is worth noting that there were very few problems arising from the actual quality of the picture or sound transmitted via the rollabout videoconferencing units.

There has been some debate in the telemedicine literature on the technical capabilities of videoconferencing equipment. This study demonstrates that the equipment can be clinically useful even at low bandwidths (128kbit/s), if the technology is absorbed into existing work practices rather than replacing them. There were many clinical uses of videoconferencing during the project, if the term ‘clinical’ is extended to include all uses by physicians, allied health staff and nurses. Some of the most frequent users of the system were the registrars, the pharmacists, social workers, dieticians and several key nurses. The study demonstrated that telemedicine is more likely to make an impact if range of staff become active users of the technology, not just the leading physician.

The introduction of telemedicine technology is complex, and requires constant feedback and dialogue with the staff. In the present study, many of the barriers to the increased use of the technology involved negotiation and compromise between the users and their managers. For example, providing a suitable location for the telemedicine units was difficult. Some staff felt uncomfortable when using the system in the dialysis areas, and preferred it to be used in other rooms. However, this defeated the purpose of using the equipment while the patients were dialysing, as well as reducing the likelihood of an impromptu use of the equipment. Another example involved the addition of a call indicator. Staff were becoming annoyed that a senior staff member could appear on the monitor screen without warning. Also, it was difficult for the caller to attract the attention of staff because they were unaware that the person was there. A solution to this problem was the addition of a flashing light and ringing sound when an incoming call was received. These issues and others, such as the use of headphones and microphones to improve confidentiality and privacy (Fig.2), further emphasise the need for sensitive project management and cultural negotiation.

Following the successful introduction of the dialysis telemedicine system in South Australia, its evaluation is continuing into a second year, with a cost-effectiveness study as a focal point. This continuing evaluation satisfies Bashshur’s advice (5) that ‘Optimal evaluations can only be performed if optimal systems are in place. Otherwise, the evaluation simply reflects imperfections in design rather than capability.’ Further research is also being undertaken into staff use and acceptance of both room-based and desktop videoconferencing equipment in order to make the technology even more accessible and user-friendly.

References

Allen, A., Hayes, J, Sadisvan, R., Williamson, S.K., Wittman, C. (1995). A pilot study of the physician acceptance of tele-oncology. Journal of Telemedicine and Telecare, 1, 34-37.

Ball, C.J., McLaren, P.M., Summerfield, A.B., Lipsedge, M.S., Watson, J.P. A comparison of communications modes in adult psychiatry. Journal of Telemedicine and Telecare, 1, 22-26.

Brown, F.W. (1995). A survey telepsychiatry in the USA. Journal of Telemedicine and Telecare, 1, 19-21.

Dakin D. Mayo presentation offers insight into attitudes about telemedicine. Telemedicine Newsletter 1995;3:1

Bashshur RL. Base evaluation on access and quality as well as cost. Telemedicine Monthly Newsletter of Telecommunications in Healthcare 1994;2:8

Graph 1 Role of respondents to initial and final survey.