Measurement properties of the Illness Perception Questionnaire in practitioners working with patients with co-morbid schizophrenia and learning disability

Rationale, aims and objectives: The widely-used Illness Perception Questionnaire (IPQ) adapted for use in patients with major mental health problems and those who care and work with them, was originally developed for evaluating the illness perceptions of patients with physical pathology. There have been concerns expressed regarding the appropriateness of the IPQ and derivative measures of it in axis I disorders such as schizophrenia. The current investigation examined the measurement characteristics of a modified version of the IPQ in mental health practitioners working with patients with schizophrenia co-morbid with learning disability. Method: Two hundred and ten mental health practitioners working with patients with schizophrenia co-morbid with learning disability participated in the study. Contemporary models of the measurement structure of the IPQ were evaluated using confirmatory factor analysis. The internal reliability of the IPQ was also evaluated. Results: Single factor, 5-factor and 6-factor models were evaluated against data. All models were found to offer a poor fit to the data. The internal reliability of the measure was also found to be unsatisfactory overall. Conclusions: Consistent with one previous study that has found poor measurement characteristics of the IPQ when used in the context of schizophrenia, the current study found the measure to be a poorly performing index of illness perceptions. Echoing previous observations of the use of this tool in axis I disorder, it is recommended that a new measure of illness perceptions be developed using contemporary research methodology sensitive to the context of psychotic illness in order to provide a more effective tool for the provision of person-centered psychiatry and healthcare.


Introduction
It is increasingly recognized, within the context of the provision of comprehensive patient care, that the role of illness perceptions are of critical importance to the course and outcomes of a range of pathologies [1][2][3][4][5].Appreciation of the role of illness perceptions to outcome has not only enlightened the opportunities for holistic, person-centered care delivery [4,[6][7][8][9], but has also precipitated awareness of the potential for interventions that attempt to positively influence clinical outcomes by improving illness perceptions directly [1,[10][11][12][13].Mechanisms that mediate the influence and role of illness perceptions on clinical outcome remain largely theoretical and to a degree speculative [14][15][16][17][18].However, the evidence from a range of clinical studies highlights, incontrovertibly, a clinically pertinent and relevant role for these mechanisms.One of the main and certainly most widely used measures of illness perceptions is the Illness Perception Questionnaire [5].The IPQ was developed within the theoretical context of Leventhal's self-regulatory model to be a conceptually and psychometrically accurate multi-dimensional measure of self-report patient illness appraisal.Five illness perception domains were envisaged to comprise the main illness perception factors of relevance to patients, these being: (i) illness identity; (ii) cause; (iii) timeline; (iv) consequences & (v) control/cure.
Thirty-six items were included in the original IPQ [5].Initial evaluation of the basic measurement characteristics of the IPQ was overall positive and promising, but, recognising some less than optimal measurement properties, the instrument developers suggested further work to evaluate the tool [5].Evaluation of the measurement characteristics of the IPQ has been extensive and led directly to the development of a revised version of the tool, the IPQ-Revised (IPQ-R), which has demonstrated improved psychometric properties [19].The original IPQ and the IPQ-R have been applied to a broad range of disease and the pre-eminence of the tool within clinical research to evaluate patient illness perceptions has led to the measure being the instrument of choice for clinical research and practice [20,21].
Recognising further the influence of carers, both professional, family members and advocates of the support of patients in coping with disease and engaging with treatment, the IPQ/IPQ-R has been developed into versions that can be readily used in these particular groups [22,23].The reliability of the instrument applied to these contexts (e.g., clinical staff and carers) has been less extensively evaluated, a surprising observation given the influence of these groups on the patient's response to illness, compliance and engagement with treatment and thus its influence on clinical outcome.Interestingly, within the arena of mental health and major axis I psychiatric disorders, such as schizophrenia, very little research has been conducted to evaluate the measurement properties of the IPQ/IPQ-R, a finding made noteworthy by the recognition of the importance of illness perceptions within recovery models that are highly influential both to the delivery of evidence-informed, person-centered mental healthcare, and to the contextual alignment of treatment intervention to outcome [15,[24][25][26][27].

Evaluation of the measurement properties of the IPQ/IPQ-R
The internal consistency of the IPQ has been reported in a number of studies as support for the measurement validity of the scale, measures of Cronbach's coefficient alpha being generally satisfactory, though not always across all of the subscales [5,20,23].The utility of Cronbach's alpha as a singular index of instrument measurement fidelity is limited both theoretically, for example, (high) internal consistency does not equate to instrument unidimensionality [28][29][30][31] and also from a measurement perspective in that Cronbach's alpha is a lower-bound measure of instrument reliability and may significantly underestimate the reliability of a scale [30,32,33].
The structural validity of the IPQ has also been evaluated using principal component analysis, exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), with mixed results.One of the few studies which has evaluated the psychometric properties of the IPQ in clinical staff caring for patients with axis I disorders was that of Fleming et al. [23] using the schizophrenia carers version of the IPQ (IPQ-SCV) developed by Barrowclough and colleagues [22].Fleming and colleagues [23] found the (IPQ-SCV) to have poor measurement characteristics in this group and suggested the tool was unsuitable for use in this group due to poor concordance of data to anticipated factor structure and low internal consistency as assessed by Cronbach's alpha on a number of IPQ subscales.However, these researchers specifically highlighted these limitations to a specific context of evaluating psychosocial intervention training (the participants in their study) and thus generalising more broadly on the suitability of the tool for use in other clinical groups working with people with schizophrenia may not be appropriate.
The issue of the validity of the IPQ for use in practitioners working with patients with axis I disorders and indeed co-morbid axis II disorder is still, therefore, to be addressed.Recognising this important vacuum in the literature, the current study sought to evaluate the measurement properties of the IPQ-SCV in practitioners working in practice with patients with significant axis I and axis II co-morbidity, specifically schizophrenia co-morbid with learning disability.

Methods Population
Participants (n=210) were a range of mental healthcare workers (both professionally qualified and unqualified) with a mean age of 46.4 years (SD 8.17) and mean length of service of 20.36 years (SD 10.82) working with patients with a dual diagnosis of schizophrenia and learning disability.Fifty-four of the participants worked in community settings, 8 in specialist settings, 102 in inpatient care settings (missing data on 4 participants).The majority of participants were female (n=138).A minority of participants were unqualified (n=71), while the largest group (n=100) were Registered Nurses -Learning Disability (RN-LD).A small number (n=5) of Registered Nurses -Mental Health (RN-MH) and dual-qualified (n=4) nurses (RN-LD & RN-MH) and those with an appropriate vocational qualification or higher national certificate (n=21) comprised the remainder of the participant cohort.

Ethical approval
Ethical approval and authorisation for the study was provided from the appropriate academic and health organisations prior to commencing data collection.

Statistical analysis
Replicating the approach taken by Fleming et al. [23] 3 measurement models of the IPQ were evaluated using CFA.The first model (model 1) was the original 5-factor model of the IPQ.The second model (model 2) tested split the timeline fact into 2 distinct factors of chronic and episodic as suggested by Barrowclough et al. [22], thus presenting a 6-factor model for evaluation.A unidimensional model of the IPQ with all items loading on a single factor was also evaluated.A maximum-likelihood (ML) estimation method was adopted with robust standard errors generated using the Satorra-Bentler scaled correction [34].Consistent with contemporary CFA model evaluation practice, a range of goodness-of-fit tests [35][36][37][38][39][40] were used to evaluate the specified models.
Internal consistency of each IPQ sub-scale was evaluated using Cronbach's alpha [41] and McDonald's [42,43] omega reliability statistic.The omega hierarchical (ω h ) test statistic [43] has been found to be a more accurate measure of reliability compared to Cronbach's alpha [30].The omega total (ω t ) test statistic [42] provides an index of reliability within the context of the total saturation of the test.Hierarchical and total omega levels were calculated based on 5-factor model and a 6-factor model from a minimum residual factor analysis of the dataset followed by a Schmid-Leiman transformation to establish general factor loadings and from which ω h and ω t are calculated.Data were analysed using the R statistical modelling program [44].

Procedure
Participants are asked how much they agree with the statement in each item on a 5-point scale (Strongly agree to Strongly disagree).Basic demographic questions relating to age, gender, length of service and clinical area were also asked on a supplementary sheet.

Results
The mean scores on the IPQ subscales are shown in Table 1.Individual item skew and kurtosis characteristics were scrutinised which revealed no evidence of excessive skew (individual item max = 1.95) or kurtosis (individual item max = 3.87).Evaluation of the skew and kurtosis characteristics of the scale reveal that the data appear distributionally normal and suitable for an ML estimation approach to CFA.
The factor models evaluated using CFA and accompanying fit indices are shown in Table 2.The 6-factor model offered the best fit to the data compared to the 5-factor and single factor models; however, evaluation of the fit indices of all models revealed a consistent pattern of poor model fit.Cronbach's alpha was sub-optimal for a number of the IPQ subscales with the exception of the consequences-relative and timelines-chronic and timelines-episodic subscales.The Cronbach's alpha of the total scale was also sub-optimal (0.66).Evaluation of the hierarchical and total omega levels are summarised in Table 3.These reveal identical findings for 5-and 6-factor specified models.The ω t statistic, which is relevant to the proportion of variance in the scale in relation to all common factors was 0.81, whereas the ω h statistic which indicates the variance associated with a common factor, was sub-optimal within a range of 0.32-0.36.

Discussion
A combination of schizophrenia with co-morbid learning disability may represent additional challenges for both the patient themselves and staff engaged in optimising patient function within contemporary treatment and recovery models of care.Critical to patient outcome are the attitudes, beliefs and opinions of care staff to both schizophrenia and learning disability.Accurate and reliable assessment of the illness perceptions of care staff working with this group is therefore essential, both for maximising the opportunities for facilitating positive patient outcomes and for identifying potential training and education opportunities for staff in the event of negative or counterevidence findings of staff illness perception of this group.Central to this activity is an accurate and reliable measure of illness perceptions.However, consistent with the findings of Fleming et al. [23] in those professionals working with patients with schizophrenia, the IPQ was found to have unsatisfactory measurement characteristics in the current study, essentially, a poor-fit to contemporary measurement models of the tool and poor overall internal consistency in relation to Cronbach's alpha.However, the Omega statistics suggest a more complex picture in relation to overall instrument reliability.It was observed that ω t was satisfactory (>0.7) for the measure when evaluating the variance due to all common factors, but conversely unsatisfactory (ω h ) when evaluating the variance due to a general factor.This may indicate support for the intrinsic multi-dimensionality of the measure, in contrast to the CFA findings which emphasise that the current conceptual measurement models of the IPQ are miss-specified.
The above findings raise a number of important considerations.Firstly, are the measurement limitations of IPQ specific to this group of care workers working within this clinical population or is this representative of a more generalizable problem with the IPQ?In some respects this question is difficult to address comprehensively, since there are many studies in other groups which emphasise the adequacy of the IPQ.Secondly, the version of the IPQ used in the current study has been adapted for use in a schizophrenia context, therefore representing a modification/adaptation of an existing tool.Consequently, the question can only be addressed in relation to this clinical context (schizophrenia and learning disability) and the participant population.However, comparison with the findings of the current study with those of Fleming et al. [23] reveal a remarkably consistent pattern of results, including similar subscale mean scores and variances, poor fit to data and overall poor internal consistency.The current study thus replicates Fleming et al.'s [23] study in terms of design and findings and extends it to a carer group working within the context of a more complex psychopathology (schizophrenia co-morbid with learning disability).However, the salient conclusion regarding the comparison of these two studies is that the IPQ represents a poor index of illness perceptions from a measurement perspective in the context of major axis I disorder.This study consequently yields compelling corroborative evidence for the unsuitability for the use of this tool in this group.
Fleming and colleagues [23] highlighted a vacuum in assessment tool availability in illness perceptions emphasising that there was little alternative to using the IPQ.Giving the findings of the current study, it would seem that the development of a new and contemporary model of illness perceptions, specifically for use within the context of axis I disorders, particularly schizophrenia, is now long overdue.A challenge in the development of such a novel measure would be the lack of availability of a comparison measure or 'gold standard'.

Conclusion
The findings from the current study indicate the lack of suitability of the IPQ for such a role.A mixed-method [45] approach may therefore be the most suitable and parsimonious approach, starting from first principles and from both patient and staff qualitative interviews to determine the lived experience of both diagnosis and provision of care.Taking such an approach would reduce contamination due to bias that may accompany preconceptions held regarding illness perceptions in relation to this group while providing a 'clean sheet' context to develop the a priori specified domains and individual items that would comprise the new scale.Taking such an approach, rooted in the evidence-base and derived from the accounts of experts by experience and experts by care provision, would be an appropriate and robust approach toward the development of a diseasespecific measure of illness perceptions that could be pursued with methodological robustness, psychometric confidence and clinical appropriateness.

Table 1
Mean, standard deviation, range and confidence interval (CI) of the Illness Perception Questionnaire sub-scale scores

Table 2
Evaluation of the structure of the IPQ determined by CFA

Table 3
Evaluation of omega reliability measures based on the 5-and 6-factor models