|
Assessing Goal
Attainment at Mobius
Beginning in 2006, Mobius began phasing-in the assessment of attainment of goals
identified within person-centered plans. Currently, over 40 individuals have had
goals assessed utilizing a process which was modified from scales used with
other service populations. The organization was seeking a method for assessing
the effectiveness of individual plans and services in supporting positive
outcomes for individuals.
Goal attainment scaling provides a means to assess the amount of relative change
by considering information from any combination of measurement, observation
and/or reporting sources. The process does not determine whether goals are
relevant (meaningful to a person's wishes and
needs) or reasonably challenging for an individual. Those assurances can only be
derived from qualitative review of an organization's person-centered planning
process, and/or specific review of an individual plan. A manuscript, 'Assessing
Goal Attainment for Quality Improvement (Lawlor & York),' describing the
approach has been published in the September, 2007 issue of the Journal of
Intellectual Disabilities.
Assessment of goal attainment is most readily useful for designing individual
plans on a case-by-case basis. The application of data resulting from aggregated
ratings is more challenging to demonstrate at the program and organizational
levels. However, it can be useful for monitoring or troubleshooting purposes
when applied with other organizational information.
Mobius currently utilizes a variety of information sources, in addition to
assessing goal attainment, to assist with quality improvement efforts at the
individual and program levels. The following instruments are utilized.
The National Core Indicator Project: A national instrument administered by
provider agencies in Maine according to a specific schedule determined by the
State. Includes key outcome indicators for service recipients within
participating state systems.
Consumer Satisfaction Survey: Implemented annually by Mobius to determine
consumer satisfaction with services and processes.
Person-Centered Planning Tools: Includes extensive strength and needs
assessment, risk assessment, and any of fifteen optional tools.
Goal Attainment Data Analysis
Updated February, 2007
Initial analysis of goal attainment data has been limited to descriptive
statistics and preliminary calculations for coefficients of rater reliability.
Next steps for data analysis include: 1) expanded collection of goal attainment
data and 2) identification of variables for correlation with goal attainment
data.
The first bar graph above is a simple presentation of distribution of goals
assessed by domain type.
The second graph depicts mean ratings by rater and domain types.
The graph directly above is an error bar graph that shows the
average domain ratings for three rater types: consumers (RCLIENT); program
managers (RMAN) and independent case managers (RISC). Family members responses
were too few in number to be included in this presentation. Some comments about
this graphic follow.
- The mean is in the middle of each bar displayed.
- The bars represent a 95 confidence level.
- The length of the bars is a function of numbers of observations and
the variability of the observations (ratings). As the number of
observations increase and the variability of the observations decrease, the bars
get smaller.
- When the bars overlap, that means that there are not
statistically significant differences between the means of the raters. When the
bars do not overlap, that means that there is a significance mean difference
(DeSisto, 2007).
The graph shows visually that the primary difference among the three rater types
is between the RISC and the RCLIENT goal attainment ratings in the social
domain. This consistently reflects other findings in the literature which
suggest that goals and outcomes within the social/inclusion/community-type
domains are more difficult to predict, given the limited control that support
programs have on community environments. (Gardiner & Carran, 2005; Krispjin,
et. AL, 1999; Leon et. al., 1999).
Updated October 1, 2007 (DML)
Additional analysis conducted with Michael DeSisto
involving assessment of goal attainment has been underway to examine
associations between goal attainment and other variables related to goal
statements ,consumers and programs (see `Study Variables’ table below). Data
collected can be grouped according to one of the following three types:
1.
goal-specific (ex. average ratings; goal statement quality; quality of
life domain type; skill or support type, maintenance or growth type)
2.
consumer-specific (ex. satisfaction; degree of involvement in setting own
goals; degree of intellectual disability; community involvement)
3.
program-specific (ex. program type; related staff experience)
Correlations were found for the following variables:
The extent that one:
- Sets own goals
- Exercises in the community
-
Attends community clubs
-
Sees friends other than staff or family
-
Participate sin community entertainment
-
Chooses work staff
An exploratory regression analysis to determine predictors
of average goal attainment resulted in the following variables explaining 25% of
the variance in goal attainment.
The extent that one:
1.
Sets own goals
2.
Exercises in the community
The regression was limited to variables representing only
12 individuals and therefore is not conclusive. However, all of the analyses
cited above do suggest that further analysis would be worthwhile. Discussions
with quality assurance representatives of the Maine Department of Health and
Human Services (DHHS) have resulted in identification of the following next
steps to improve the quality of the study of assessment of goal attainment:
-
Develop and define a variable (or variables) to assess the quality of
goal statements
-
Identify variables that, based on our preliminary analyses and the
literature, hold promise for future meaningful study, and collect current data
for 100% of the consumer population.
-
Evaluate constructs for quality of life domains for use in future
analyses
Mobius is committed to continuing the study of individual
goal attainment both to improve the quality of its programs and to study quality
of life outcomes for persons with intellectual disabilities. To these ends,
Mobius will seek to secure funding to support further study.
For further information about assessing goal attainment at
Mobius, contact our director at:
dlawlor@mobiusinc.org
Hit Counter
| 
