|
|
|
|
Assessing Goal Attainment at Mobius
Initial analysis of goal attainment data has been limited to descriptive statistics and preliminary calculations for coefficients of rater reliability. Next steps for data analysis include: 1) expanded collection of goal attainment data and 2) identification of variables for correlation with goal attainment data.
The second graph depicts mean ratings by rater and domain types. The graph directly above is an error bar graph that shows the
average domain ratings for three rater types: consumers (RCLIENT); program
managers (RMAN) and independent case managers (RISC). Family members responses
were too few in number to be included in this presentation. Some comments about
this graphic follow. - When the bars overlap, that means that there are not
statistically significant differences between the means of the raters. When the
bars do not overlap, that means that there is a significance mean difference
(DeSisto, 2007). Updated: July, 2011 The Mobius goal analysis program produced it’s most significant work to date with the completion of a research project jointly supported by the Maine Health Access Foundation and the Maine Office of Adults with Cognitive and Physical Disabilities. The following final report was submitted in April, 2011. Program Goals and Objectives: The goal of the project was to collect and study data contained within service plans from two agencies operating within DHHS Region II (central), and from samples in DHHS offices in Region 1 (south) and Region III (north). Specific objectives of the project were to 1) implement assessment of goal attainment in the Region II agencies, 2) collect goal statement samples from all sources; 3) classify goal statements according to health activity and other goal and service recipient characteristics, and 4) analyze resulting data for frequencies distribution and correlations, One purpose was to identify the degree of goal activity that addresses individual health issues, and to explore associations between and among goal and subject characteristics based on available data. Key Program Activities February 1, 2010: Internal data collection initiated. Collaborators: Additional project staff included: Emma Stephenson, Mobius Quality Manager, for data classification; Jacob Lowell, MA Statistics, for data analyses; and Romy Spitz, Ph.D. and Michael DeSisto, Ph.D. for consultation on project design. Other collaborators were critical to the program and participated in several ways. Initial input was received from the project advisory group to advise priorities and from a second meeting convened to review early study results and recommend areas for further study. Advisory group participants included: Jane Gallivan, the Director of the Office of Adults with Cognitive and Physical Disabilities (OACPD); Jay Yoe, Director of DHHS Quality Improvement; Margaret Rode, OACPD Director of Quality Management; Laurie Kimball, OACPD Regional Quality Improvement Staff; and project staff. Implementation included extensive collaboration between Mobius project staff and The Progress Center, Norway, Maine and coordination with managers of OACPD regional operations in the Caribou and Portland offices of DHHS. The implementation timeline ran from 2/1/2010 to 4/31/2011 (as extended). All activities described above (advisory input, design detailing, data collection, analysis) occurred within time frames that enabled the project to complete key activities on schedule. Data was collected from recipient plans and BMS 99 forms. Under advisement from the DHHS Office of Quality Improvement, a project-specific coding method was used to protect the identity of service recipients. 1700 goal statements from 133 service plans obtained from the 3 State service regions were classified according to 18 variables representing 6 goal and 12 individual characteristics. Chi Square Analyses Chi square analyses were conducted using the following dependent variables and classifications. Goal Characteristics
Individual Characteristics
Mutli-Variate Analyses Logistical regression analyses were conducted using the following dependent variables.
Key Correlations Pertaining to Health Chi Square Analyses: Physical Health (n=566) vs Other Goal Types (n=1096) Physical health goals were more directly correlated with being present in the community (p value <.0001) but negatively correlated with presence of communication content (p value < .0001). Discussion: The results are generally consistent with the well-established challenges that health providers of all types face in effectively interacting with many patients who have intellectual disabilities. A recent meeting of the Family Medicine Education Consortium focus group on primary care for persons with developmental disabilities (Project Director is a member) identified provider training as the highest need for improving primary care practice and a North Carolina–based group will be developing a curriculum for PCP education over the next two years (Hersey, PA, October, 2010). Goals that addressed physical health and personal development were more likely to involve skill learning, compared with other goal domains. Chi Square Analyses: Physical Health Sub-Domains (n = 571) Sub-Domain Categories: Fitness (n=153), Hygiene (n=37), Medical Care (n=137), Nutrition (n=98), Personal Safety (n=99), Other (n=47) 10 variables were identified to have significant differences (<.0001). Medical care and fitness were directly correlated with being present in the community, while all categories were negatively correlated with communication content. Other key sub-domain results follow. The younger subjects grouping (age 18-39) presented higher rates of participation in fitness, medical care, and nutrition activity. Discussion: The analysis permitted a more detailed look at specific categories pertaining to physical health by supporting and expanding on the findings of the other chi square and regression analyses involving the broader domain categories. The decreased goal activity in self-guardians may be due to those individuals having attained a level of independence where skill development and support are not priorities for service plan goals. Other relationships are discussed in the next section. Logistic Regression: Physical Health (n=482) vs Other Goal Types (n=982) The analyses produced results indicating that physical Health goals were 30 % more likely to include skill development and 70% less likely to have communication content. Physical health goals were 1.6% less likely to be present with each additional year a subject ages. Discussion: The regression results for relationship with communication content were consistent with the chi square analysis. The direct relationship with skill development was consistent with the analysis of physical health sub-domains and interesting in that it suggests that the tracking of gains is more likely to occur with physical health goals, which might logically include targets such as losing weight (nutrition, fitness), increasing endurance (fitness) or performing hygiene with greater independence. In addition, the Maine system has undertaken initiatives to increase recipient awareness and participation in leading healthy lives, which may have contributed to this finding. Of the variables resulting in direct correlations with goal attainment (fitness, nutrition and medical care), two correlated directly with skill development. Further study could explore the interrelationship between skill development and goal attainment. The reduced likelihood of physical health goals as subjects age could reflect reduced emphasis on self-management of health in later years of life. Other Correlations of Interest Skill Content & Communication Content & Goal Attainment All analyses showed a direct correlation between the presence of communication content and skill development content in goal statements. Regression analyses found that these two characteristics were 2.2 X more likely to be present together, one of the two highest ratios reported. The analyses found a strong relationship between skill development and goals involving more than one-time events (6.6 X) which was is logical given that skill development involves continuous work. Higher goal attainment was directly correlated with receptive communication ability (p value <.004) and with the mid-ranges (moderate and severe) of intellectual disability (p value < .02). Perhaps worth mentioning was the less significant direct relationship between goal attainment and communication content (p value < .09), in light of the previously stated receptive communication relationship. Discussion: Maine OACPD regulations specify expectations for habilitation (skill development), however the importance of communication using non-traditional methods that address individual communication limitations has only begun to be recognized in the field of developmental disability services. Given that analysis for assessment of expressive and receptive communication ability (BMS 99 assessment forms) identified 45% of all subjects as having either moderate or very limited communication limitations, there appears to exist significant unmet need in this area. Further analyses of all sub-domains could provide more detailed insight into the inter-relationships among goal attainment, communication content, skill development present, and receptive communication ability. Considered in total, these findings suggest that further investigation is warranted to distinguish between: intentional and unintended communication, and traditional and non-traditional communication activity, including the impact of each one on outcomes (goal attainment). Independence and Skill Development Generally, characteristics of independence (self-guardianship, higher intellectual dx, less disability, greater expressive and receptive communication ability, independent health maintenance) were negatively correlated with the presence of skill development. Discussion: More independent individuals may be living more satisfied lives and have less need or desire for learning functional and life skills. Services for these individuals typically involve between 5-25 hours of support per week in order to maintain living situations and receive transportation to needed community resources. That contrasts with higher need individuals receiving high levels of support who have more goals, many which feature targeting skill development (and goal attainment as noted in the section above). High Performing Subjects Each sample source was asked to identify up to 10 subjects who would be considered high performers as described by project staff. This variable exerted greater weight on samples from the OACPD regional offices (50% of each), less than the Mobius and Progress Center samples (25% of each), the combined effect weighting the entire study group towards higher achievement. This additional measure was included in the study to see if correlations of interest could be identified to help explain higher performance. Chi square analysis of the high performer variable resulted in significant differences for five variables. Discussion: Aside from community participation, other correlations appeared to be as expected. The negative correlation with community participation may be due to the fact that related community-skills are already acquired in high performance subjects, and therefore not the subject of service plan goals. The correlation with one-time events would be consistent with the explanation provided in the sections above regarding lower levels of skill development with greater independence. MeHAF Recommended Chi Square Analyses Guardianship with Disability Level. A significant difference was identified in this analysis (p<.0001). Subjects identified as being their own guardians were directly correlated with being higher functioning. Health Maintenance Ability with Guardianship and Disability Level. Significant differences were identified in these analyses (p<.0001). Subjects rated as independent in health maintenance were directly correlated with being their own guardians and with being higher functioning. Medication Administration Ability with Guardianship and Disability Level. Significant differences were identified in these analyses (p<.0001). Subjects who were rated as independent or needing skill training for medication administration were more likely to be their own guardians and to be assessed as higher functioning, compared with subjects requiring support administering medications. Goal Domain-Type with Disability Level. No significant differences were found in the analysis of disability level with domain. However, as reported above, the analysis using physical health sub-domains did find significant differences with disability level, with individuals assessed as higher functioning being more likely to have goals involving fitness, medical care and nutrition. Discussion: Generally, these results were as expected, higher functioning aligning with increased independence in areas studied. As such, these and many other anticipated findings, generally serve to validate the data collection and analysis methods employed within the project. In medication administration, the finding that the two categories of `independent’ and `skill training needed’ were both more likely to be their own guardians could merit further study. Additional analyses found health maintenance ability directly correlated with expressive and receptive communication ability and medication administration ability directly correlated with goal attainment. Project Limitations Study limitations include at least the following elements. The exclusion of other variables influential to outcomes due to the limited available data. Meeting Goals The MeHAF/DHHS study `Identifying Health Goals in Service Plans’ project was designed to be exploratory in nature, as an initial inquiry into the content of service plans, as it relates to physical health and other characteristics, and as an opportunity for basic analyses examining variable relationships. These purposes were achieved. In addition, the Director of Quality, DHHS, has indicated interest in supporting further analyses of the data to gain greater detail about variable inter-relationships. The project produced several outcomes that can be useful to future data collection and analyses of a developmental disabilities service system activity.
MeHAF Role In addition to the critical role of direct funding and leveraging other support, MeHAF staff participated in the review of initial data and identification of additional areas for analyses (see MeHAF Recommended Chi Square Analyses above). Next Steps Analyses and discussion has resulted in identification of other areas for study, dependent on available resources. They include: Conducting similar study that addresses study limitations by 1) using a larger sample size, 2) employing randomized selection from the entire state population of service recipients, and 3) identifying other data sources (EIS). This is currently being explored with the OACPD.
For further information about assessing goal attainment at Mobius, contact our director at: webmaster@mobiusinc.org |
|
Send mail to
webmaster@mobiusinc.org with
questions or comments about this web site.
|
