Results
Descriptive statistics are presented by CF program cohort in Table 2 and
represent aspects of the Inner Setting. There were no statistically
significant differences in characteristics of CF programs across cohorts
and thus, analyses were conducted across programs.
The first aim was to analyze the top-ranked barriers and successes of
this implementation effort. Barriers were rank-ordered in terms of their
significance, and successes were evaluated by their frequency of
endorsement.
Barriers to implementation. Respondents were asked to rank order
a list of 9 potential barriers each year from 1 (most significant) to 10
(least significant). Rank ordering of barriers for Cohort 1 across the 3
years was highly consistent, with staff time, dedicated space (e.g., to
screen adolescents privately from parents), and perceived patient burden
(time and availability) ranked as the top 3 barriers to implementation.
Rank ordering of barriers across all cohorts and years yielded similar
results, in a slightly different order: #1) patient burden, #2) space
limitations, and #3) limited staff time. These barriers represent
characteristics of the Inner Setting that are difficult to ameliorate.
Successes of implementation. Respondents were asked to report the
top 3 successes of implementation. The 4 most frequently endorsed
successes in 2018, representing the 3rd year of
implementation were (representing Cohort 1 year 3, Cohort 2 year 2, and
Cohort 3 year 1) were: #1) early identification of depression/anxiety
(mental health issues); tied –#2) improved access to
psychological services and interventions and #2) increased
awareness/education in the CF team about depression/anxiety. The
4th most frequently endorsed success was reduced
stigma about mental health/normalization.
The second objective was to evaluate the success of implementation in
the first year and whether it improved over subsequent years, using the
survey scoring criteria outlined above. As can be seen in Table E.1
(supplement), implementation scores were very similar across cohorts in
Year 1, ranging from 21.12 to 26.17. Implementation scores increased
significantly over time (see 95% confidence intervals), nearly doubling
with each year of implementation. The longitudinal trajectories of these
implementation scores were then modeled over time for individual
programs. The estimated implementation scores and 95% confidence
intervals for each cohort across the 3 years of the study are depicted
in Figure 2, along with the trajectories for individual centers. As
expected, there was a pronounced increase in implementation scores over
time, with improved implementation reported each year. In addition,
there was marked variability among programs both in terms of initial
implementation and progression across time.
The third aim was to identify predictors of implementation success,
considering Characteristics of the Inner Setting and of Individuals.
Predictors for the first year of implementation are listed in Table 3
and indicate that the mental health providers’ years of experience on
the CF team was significantly related to higher implementation scores.
Predictors for changes across time for Cohort 1 are presented in Table
4. Similar to the first-year model, years of experience on the CF team
was significantly related to both implementation scores in Year 1 and
changes in implementation over time.
Convergent validity using registry data. An external source of
data from the Outer Setting, the Cystic Fibrosis Foundation Patient
Registry (CFFPR),19 was used to evaluate convergent
validity with the survey results. Annual data from the CFFPR on national
rates of mental health screening were obtained for the year priorto the initiation of MHC grants in 2015 and for the 3 years of the
implementation study (2016-2018). Responses to the following 2 questions
were recorded: “Was the patient screened for symptoms of classic
depression using the Patient Health Questionnaire (PHQ-9) or other valid
depression screening tools?” [yes, no, unknown]; “Was the
patient screened for an anxiety disorder using the Generalized Anxiety
Disorder Tool (GAD-7 or similar?)?” [yes, no, unknown]. As
expected, rates of screening in 2015 for depression were quite low:
21.4% for those 12-17 years (adolescents) and 24.1% for those 18 and
older (adults). Similarly low rates of screening for anxiety were
documented: 17.6% for adolescents and 19.5% for adults. In 2016, when
the MHC grants were launched, depression screening rates more than
doubled to 57% for adolescents and 61.2% for adults; anxiety screening
rates increased to 53.2% for adolescents and 59.2% for adults.
Screening rates for depression continued to increase in 2017 and 2018 to
69% and 73.2% for adolescents and 75.1% and 80.1% for adults,
respectively. Anxiety screening demonstrated similar increases across
those two years: 67.2% and 72.5% for adolescents and 73.9% and 79.5%
for adults.19