ECHO Colorado Implementation Profile

ECHO Colorado

The chancellor of the University of Colorado was instrumental in bringing ECHO to the campus and to the State of Colorado. At a gathering of leaders from academic medical centers in 2012, the chancellor of the University of New Mexico’s academic medical center asked the University of Colorado (CU) chancellor, Don Elliman, “Why don’t you have an ECHO?” Intrigued about the general concept, Elliman convened a small focus group with representatives of the two hospitals and five graduate schools on campus to discuss what they knew of the ECHO concept. A task force was established to explore what ECHO might look like in Colorado, led by Tim Byers, MD, and representatives from the hospitals and graduate programs. Fred Thomas, PhD, in his role as director of telemedicine at Children’s Hospital Colorado (CHCO), was the representative from CHCO.

The task force met monthly under the co-leadership of Thomas and Byers and conducted interviews on campus with the various representative entities. Byers approached the Colorado Health Foundation for an initial grant of $180,000, to travel the state and conduct focus groups and a listening tour of health systems, community-based organizations, traditional advocacy groups, and organizations that had an existing referral relationship with the CU Anschutz. The listening tour laid the foundation for ECHO Colorado’s structure and mission to be state-focused, community-driven, and directed by a board representing non-CU entities. Leaders wanted to recognize the bodies of expertise around the state, as well as on the Anschutz campus. As one respondent stated, “This is ECHO Colorado, not CU Anschutz ECHO. And I think that has worked quite well.”   

The Colorado Health Foundation designated the ECHO work as an area they would invest in over several grant cycles. Their investment of more than $6 million was complemented by funds from the CU Anschutz chancellor who wanted to demonstrate how ECHO could be integrated into the CU Anschutz Medical Campus. Just as ECHO Colorado was hiring its initial team and establishing its programmatic efforts, Byers retired and Thomas took the lead. Thomas prioritized efforts with ECHO’s partners and set a goal of establishing ECHO training programs for the top 20 priorities identified during the statewide listening tour. In 2018, Leah Willis joined ECHO Colorado as the director of programs, reporting to Thomas. Prior to joining the ECHO Colorado team, Willis worked in ECHO and related roles in telemedicine, continuous professional development, and network development in Utah and Tennessee.

ECHO Colorado began transitioning away from the Colorado Health Foundation as a primary funding source in early 2019, more than a year before the grant ended. As collaborations with various partnerships grew, ECHO Colorado began to initiate more clinically oriented ECHO programs in partnership with the CU faculty who delivered care through the CU Anschutz hospitals. The clinically focused programs enabled ECHO Colorado to diversify its funding and achieve a goal of “braided funding” from grants, contracts, CU Anschutz, and the state. By focusing on clinical programs, ECHO Colorado increased grant and project-based support through Medicaid Upper Payment Limit funding, which, on the CU Anschutz campus, was managed through the Office of Value-based Performance. In a 2020 merger with the campus eConsult program, ECHO Colorado transitioned from the School of Public Health to the School of Medicine, reporting to the senior associate dean for clinical affairs and was awarded umbrella funding vis a vis Medicaid Upper Payment Limit dollars by the Office of Value-based Performance.

With the increase in support and the shifting of focus to clinical programs, ECHO Colorado became more tightly integrated into the CU Anschutz Medical Campus. However, the long-term relationships with organizations around the state still provided community specific opportunities. The change in funding resulted in a disbanding of the statewide ECHO board. All ECHO programs had someone from the university engaged, in addition to community-based experts.

Autism Case Review    

In 2019, developmental pediatrics specialists Ann Reynolds, MD, and Rebecca Wilson, PsyD, wrote a request for Medicaid Upper Payment Limit funding for an ECHO focused on Autism Case Reviews. Reynolds first learned of ECHO when her section head introduced her to the model during the first implementation of ECHO within Developmental Pediatrics. Wilson was aware of ECHO through personal and professional contacts who had been involved in ECHOs. Wilson explained, “When we conceptualized our Medicaid Upper Payment Limit grant, we both saw that ECHO would be a good component of this work, to foster greater community outreach. It just fits so perfectly with our conceptualization of involving primary care in rural remote and underserved areas in identifying and caring for our patients.” Wilson further commented that the ECHO program allowed their interdisciplinary team of specialists to see patients (both indirectly via ECHO presentation, and sometimes, directly through clinic) that they would not otherwise see because the patient might not make it far enough into the health system to access all the expertise available.

The program leads and Willis implemented two autism-focused ECHO programs. One program was a six-week “introductory” series repeated several times per year, and the other program was an ongoing, monthly Autism Case Review ECHO program —the program that was included in this case. Both programs were supported by the same multidisciplinary team, creating continuity in the educational experience, and promoting relationship development between the program leads and participants.

The Autism Case Review ECHO was a unique adaptation of the ECHO Model. It did not have a formal curriculum, and didactics were offered only as needed to supplement a specific participant-submitted case. Participation in the six-week series or special permission from the program leads was required to join the Autism Case Review ECHO. Cases were typically provided by participants who were primarily from rural practices and at a geographic distance from the CU Anschutz campus. A six-to eight-person multidisciplinary team heard the case presented by a provider. This team included physicians, a psychologist, a social worker, and a genetic counselor. Reynolds explained that the primary care physician may “fully realize the child has autism. They really don’t have any questions about that. But they really don’t feel comfortable with what comes next. So, we’ve been mentoring them.”  Registration was open to eligible providers and typically had ten to twelve participants per session.

Adolescent Reproductive Health 

This program was developed by gynecologist Eliza Buyers, MD. Buyers first learned of ECHO through her extensive work in telemedicine and her engagement in the BC4U (Birth Control For You) program that focused on birth control for adolescents and teenagers. Although Buyer’s expertise (adolescent reproductive health and adolescent gynecology) was distinct from the initial purpose of BC4U, Buyers’ energy and expertise were undeniable. Learners clamored for additional opportunities involving Buyer, and a partnership was formed. BC4U contributed funding and Buyers contributed expertise and clinical leadership of the Adolescent Reproductive Health ECHO series.

Buyers’ attraction to ECHO stems from her commitment to being “a champion of the cause” for high- quality adolescent reproductive health care in primary care. Explained Buyers: “I’m in the community. I know what pediatricians are doing. Too often, the word ‘birth control’ or ‘sex’ frightens them. If we make this about adolescent reproductive health taught by a gynecologist, you are going to get pediatricians to buy in based on the fact that they have girls in their practice who have concerns about their periods. I can just tell you that then leads to discussions about contraception later on.”

The curriculum for the Adolescent Reproductive Health ECHO program included a 30-minute didactic presentation and a case. All didacts were presented by Buyers, who had invested considerable time and energy into the development of curricula, including rewriting the original curriculum—from one that targeted prescribing providers to a new curriculum that targeted an audience of non-prescribing nurses. In addition to leveraging quantitative and qualitative data from evaluation and assessment, Buyers conducted informal interviews with current participants and potential target audience members to inform the curriculum and the cases. Cases, also presented by Buyers, were often based on questions that came to Buyers who was on a pager and providing real-time consultations during her regular clinical rotation. Registration to the program was limited to about 30 participants per cohort. When registration exceeded this number, applicants were placed on a waitlist for the next cohort.

The ECHO Model seeks to build a learning community where “all teach, all learn.” This is done by leveraging technology, by sharing best practices, though case-based learning, and using data. We asked respondent to tell us what “all teach, all learn” meant to them. The ECHO hub encouraged all program leaders to integrate participant needs into program implementation. Willis commented, “Put yourself in the shoes of people who you’re intending to collaborate with on raising the scope of practice around this topic. It really is a collaboration, because if they’re not buying in and don’t feel like they’re getting something valuable out of this for their patients, they’re not going to show up.” The hub leaders framed “all teach, all learn” as everyone teaching something and learning something. Typically, the experts were delivering the content, and the participants were delivering an understanding of the true scope of practice and comfort levels with topics. Based on experiences across multiple ECHO programs, the hub staff realized that “it’s not all teach, all the same—we’re learning different things from each other.”

ECHO Colorado placed a strong value on the role that coordinators played in building an “all teach, all learn” culture. The respondent with the Adolescent Reproductive Health ECHO shared, “I really think the coordinator, although often associated with the logistics and coordination role, really has the responsibility to help bring that “all teach, all learn” into each session. It’s really important for the coordinator to provide regular feedback to the facilitator. Like, ‘Hey, next time let’s create more space for participants.’ Or ‘Hey, we need to call on some participants to show them that they do belong in that space.’” 

ECHO Colorado recognized that different approaches to ECHO were warranted and adapted the model to create “buckets” or an ECHO typology. Some series (programs) were more about information delivery and some were more about community practice but all were based on the needs of participants. The buckets included:

• Learning Series (didactic and facilitated discussion)
• Enhanced Case Management (didactic and participant-submitted cases)
• Case Review (participant-submitted cases only)
• Community of Practice (moderated discussion only)
• Co-management Conferences (quality improvement series focused on eConsults)

The buckets were not static and changed over time.

ECHO staff and leaders had many ways to reinforce the “all teach, all learn” model, including a formulaic approach to listening to goals from program teams and participants, working with academic and community-based experts, and using assessment and evaluation to inform continuous program improvement.

Autism Case Review

Case Review was developed as the bucket for the Autism Case Review to provide a structured and ongoing support system for primary care providers managing children with autism. Case Review was a synchronous “co-management care” model, which was “eConsult, but with context.” This type of session worked well because the “nature of our practice is to listen and ask questions because our field has complex diagnostic presentations that don’t lend themselves to easy flow charts or algorithms.” Both medical leads emphasized how much they have learned from the rural providers, emphasizing they had a stronger recognition of how trauma might impact children with autism. One respondent commented, “I grew up on the Western slope and thought I knew it, but I didn’t. This has been a powerful clinical and personal lesson for me.”

The multidisciplinary program team members – which included physicians, psychologists, genetic counselors, and social workers – were learning from each other. The safe space for participants to ask questions extended to the experts who “felt safe to say, ’We don’t know the answer to that question.’” The leaders displayed how to work together and the respect they had for each other’s roles.

Their safe space was built over time and by going out into the “towns and sitting in clinics with providers. That helps with authentic, genuine connections with people.”

Adolescent Reproductive Health

The Adolescent Reproductive Health ECHO was categorized as a Learning Series because it addressed a known gap in knowledge coupled with a desire to appeal to all providers, regardless of personal or strongly held community beliefs influencing adolescent reproductive health care. Respondents acknowledged that “all teach, all learn” was challenging because it wasn’t how physicians were trained. Yet the physician lead, Buyers, embraced this principle. Buyers commented. “Participants need to teach me what they need to know, because I don’t know their knowledge gaps. And then they often have fabulous resources to share with each other.” The coordinator commented, “There’s real value in hearing from our rural providers whose communities look different than the metro areas, which is where you’re going to find the academic hubs.”  

Across the ECHO Colorado hub, program leaders and participants were not necessarily teaching and learning in the same way. Buckets or a typology of ECHO programs allowed for differentiating the approach to meeting needs. Still, constant across the programs was that program leaders taught content and provided specialty consultation, while also learning about participants’ scope of practice, comfort with topic/care delivery, the availability of local resources in communities, and the challenges participants’ patients face in seeking specialty care.

Studies of program implementation identify context factors that can shape how a program was implemented. Such factors include leaders or champions, state and federal policies, funding, partnerships or collaborations, staffing, internal structures and processes, and monitoring for quality and fidelity. Not all of these factors play a role in how EHCO was implemented here or elsewhere. Below, we identify factors that emerged during interviews that appear to influence how ECHO was implemented at ECHO CO, and the Autism Case Review and Adolescent reproductive Health ECHO programs.

Funding

ECHO Colorado described their funding model as “braided, a very diversified funding model.” The Colorado Health Foundation invested more than $6 million to jump-start ECHO Colorado as a hub to serve the entirety of the state. These funds allowed for a thoughtful approach to the role of ECHO Colorado and the hiring of staff, including an experienced director, coordinators, and learning specialists. The chancellor of the CU Anschutz Medical Campus guaranteed funding to ECHO Colorado although the hub had not looked to those funds as their sole or primary form of support. ECHO Colorado also had grants from the State Health Department—including federal pass-through grants—and external funding that played an increasing part of their budget. The funding structure evolved when Medicaid Upper Payment Limit funding identified ECHO Colorado as a state strategy for value-based care innovations.

As previously mentioned, both programs studied received Medicaid Upper Payment Limit funding. However, the flow of the funds for each program was different and this impacted implementation. For the Autism Case Review ECHO, the program team leaders held the award. As a result, they were free to evolve their ECHO programming—almost without restriction—as long as they meet the overarching objective of increasing access for Medicaid patients. For the Adolescent Reproductive Health ECHO program, the funds flowed through BC4U, a Children’s Hospital Colorado department that was separate and not integrated with Buyers’ home department (Adult Obstetrics and Gynecology). The impact was that Buyers had to seek approval for any programmatic changes (including content, frequency, audience), in addition to continually justifying funding for her ECHO time.

Champions and Leaders

The early support from the CU Anschutz chancellor for ECHO Colorado marked it as a “priority” strategy for the university. The chancellor reached out to Byers, a well trusted clinician researcher with long-standing campus-community relationships, to lead a task force. Other task force members were drawn from the two hospitals on campus—Children’s Hospital Colorado and University of Colorado Health—and representatives from the various professional schools—School of Nursing, School of Medicine, School of Dentistry, and School of Public Health. Initially Thomas served as the representative from Children’s Hospital. As the statewide interviews were being conducted, Thomas and Byers became co-leaders of the effort and became co-directors of the ECHO Colorado organization once it received its initial funding. ECHO Colorado was initially governed by a board consisting of state-wide entities. Although this board eventually disbanded, their early engagement championed ECHO across Colorado. The ECHO Colorado hub also invested in hub leadership by recruiting an experienced director, Willis, who had worked with ECHO hubs and programs at the University of Utah and Vanderbilt University.

Training

ECHO Colorado hub leaders drew on a depth and breadth of training and work experiences to inform their strategy and approach. During the initiation phase, Thomas, Willis, and other ECHO Colorado staff attended ECHO immersion trainings at the ECHO Institute. Thomas said, “I went to immersion four different times. I went to just look at it from an entrepreneurial perspective, I went from a technology perspective, I went from a healthcare delivery perspective. I tried to listen to it differently and interact with different people at different times.” Staff were encouraged not to sit next to one another, but “instead to go and immerse themselves, to listen, to learn, and find out who’s doing something groovy, and bring that back.” Staff attended multiple MetaECHO conferences. Willis also engaged in other ECHO-adjacent informative experiences to better understand how ECHO was situated in the broader landscape of tele-enabled care.

The ECHO Autism Case Review and Adolescent Reproductive Health program leaders we interviewed had not participated in an ECHO immersion training, although many had attended one or more MetaECHO conferences. Program leaders received orientation to the ECHO Model and ECHO facilitation training from the ECHO Colorado staff. In some instances, program leaders were familiar with the model through attendance at other ECHO programs.

Organizational Characteristics

ECHO Colorado sat within the University of Colorado Anschutz Medical Campus. Initially the hub defined itself as a distinct entity, including non-University branded marketing (e.g., logos, website domain) and experts drawn primarily from outside the university system. Grant partners were community-based entities without university affiliation. The intent of this framing was to promote ECHO Colorado as a statewide “utility” rather than a university-centric project. After the branding was revealed, Thomas received a call from the chancellor’s office about it not looking like ECHO was connected to the Anschutz Medical Campus. Thomas explained that communities that hear “we’re here from the medical campus and we’re here to help,” sometimes heard instead, “we are here to tell you what you are doing wrong or tell you how to do something.” Thomas explained that they were establishing a different kind of partnership where communities were equal partners and needs were identified by the whole state.

ECHO Colorado evolved toward more integration with campus resources and support, while maintaining an emphasis on promoting access to specialty care across the state. Increasing internal integration shaped expert selection and program prioritization. A merger with the campus’s eConsult program led to a transition from the School of Public Health to the School of Medicine, resulting in administrative process changes.

Staff

Leaders in both ECHO programs expressed gratitude and dependence on the support they received from Willis and coordinators in the ECHO Colorado hub. One respondent commented, “You need somebody who is a close working colleague, who’s engaged and inspired by your mission, who really helps you mentor from the very beginning. Who can give you a sense of the to-dos and not to-dos. Here’s how you talk about it. Here’s what’s helpful. Here’s how to brand it. Those basics really helped us get off to a confident, good start. I think an ECHO mentor should be built into the program.”  A respondent from the other program commented that her program coordinator “taught me so much. There’s a couple of things she did for me that I was not able to do and this allowed me to focus on delivering the content and trying to hear what I was saying and slow down . . . And then after the session she gave me concrete advice. She has been my role model for every other coordinator.” 

Partnerships and Collaborations

From the beginning, ECHO Colorado focused on building partnerships across the state and within the Anschutz Medical Campus community. With a change in funding to align with upper payment limits (Medicaid), the internal partnerships became stronger; however, ECHO Colorado remained focused on expanding care access to disadvantaged populations across the state. Thomas and Willis both mentioned multiple state-wide partnerships that support a range of ECHO Colorado efforts. Leads of both programs created unique partnerships between ECHO Colorado and their employing unit. Further, these leads commented that their role in ECHO programs strengthened their partnerships with providers across the state.

Policy 

To a modest extent, minor consent laws regarding confidentiality and interactions with patients/families influenced the advice provided during the Adolescent Reproductive Health clinics. The stronger policy impact is associated with Upper Payment Limit via Medicaid. Specifically, ECHO programs with Upper Payment Limit funding were required to meet certain metrics of improving Medicaid access to specialty care.

When discussing the vision and sustainability of ECHO CO, including Autism Case Review and Adolescent Reproductive Health, several themes emerged.

Integration

Thomas and Willis envisioned ECHO Colorado as becoming more integrated with the university’s infrastructure, particularly in value-based care delivery and community engagement/outreach strategies. They hoped to expand into the state’s infrastructure, with particular emphasis on financial partnerships and collaborative program development with the state Medicaid office and the state health department. ECHO Colorado leaders also talked about ECHO and CORE [eConsults] coming together as a package and creating a new model of peer-mentored collaborative care. Leaders in both programs discussed the possibility of their ECHO becoming more integrated with eConsults.

Financial Sustainability

ECHO Colorado had a braided funding model with ongoing program center support through the CU School of Medicine. However, goals included further diversifying their funding using a multi-faceted strategy, including drafting innovative value-based payment model proposals, and leveraging existing fee-for-service structures at the state level. As a partner with the campus eConsult program, ECHO Colorado benefited from the continued adoption of eConsults by payers, as well as its dissemination outside of the university system. In thinking about future funding, Thomas commented “I would like one more braid in our braided-funding and that would be a fee-for-service, or some sort of built-in healthcare payment model. It’s just having all the different pieces in a way where we don’t have to depend on any particular one or worry about losing one particular braid and then we go out of business.” 

The ECHO program respondents expressed an interest in more sustainable funding including funding that would enable them to buy out more clinical time. A respondent in one program commented, “We can only do so much. We have some buyout for our clinical times, but we don’t have enough buyout to really run an ECHO-based clinical pathway.” A respondent in the other program shared, “I need more funding for my time because I work for the university. Every month I get a report card saying how much money I’ve made. If I’m not seeing patients, then I’m not making any money and I’m not funding my salary, which we’re all expected to do. So, unfortunately, that’s what I have to figure out.” 

Showing Value

Sustainability and integration were dependent on showing the value of ECHO to the university, the state, and others. Thomas said, “We have to publish more than feel-good articles, that we’ve done something with ECHO that is meaningful to our different partners.” This includes showing impact on costs and access to care. One ECHO program leader said, “We’re getting actual results. People are doing things that they weren’t doing before that align with best practices. If someone, perhaps a state health department could study the ECHO and show the results, then I could expand it.”

Expanding Programs

In both ECHO programs, leaders emphasized expanding their programs to more participants and thereby increasing their impact on patient access and health. The Autism Case Review talked about their current work as a “new care pathway, which is educating primary care physicians to do their own developmental history and identify patients who need to be expedited and see us.” They wanted to see their ECHO turned into a “robust secondary screener. An expedited care pathway with patients that are really a good fit for us. We want the complex patients. The easier patients, we want the physicians to take care of themselves.” The lead of the Adolescent Reproductive Health ECHO wanted it to be easy for physicians to reach her, saying her vision of ECHO in the future “would just be like an eConsult, but more communication. I want to support the cohort more. I want to know what they’re doing, what their questions are.” She also expressed an interest in expanding the program into other states.

Eliza Buyers, MD
Program Lead, Adolescent Reproductive Health ECHO

Sara Elizabeth Hunt
ECHO Project Coordinator, Adolescent Reproductive Health ECHO

Ann Reynolds, MD
Program Lead, Autism Case Review ECHO 

John “Fred” Thomas, PhD
Executive Director, ECHO Colorado

Leah Willis, MS
Director of Programs, ECHO Colorado

Rebecca Wilson, PsyD
Program Lead, Autism Case Review ECHO 

Willis, L. & Larson R.S. (2022). ECHO Colorado Implementation Profile. Diffusion Associates. https://www.diffusionassociates.com/echo.