Project ECHO at the Robert Wood Johnson Medical School, Rutgers University Implementation Profile

The story of how Project ECHO found its way to the Robert Wood Johnson Medical School, Rutgers University, begins with The Nicolson Foundation. The mission of The Nicholson Foundation was to improve the health and well-being of vulnerable populations in New Jersey. Joan Randell, who was chief operating officer (COO) for the foundation, wanted to introduce the ECHO Model to health care policymakers and influencers across New Jersey. She organized a team of people from multiple state health divisions and offices, along with foundation staff, to attend immersion training in New Mexico. The Nicholson Foundation also invited Sanjeev Arora, MD, to New Jersey on two occasions—once to help think through a strategy for ECHO and the second time to speak to state leaders, including two state senators and the chancellor of Rutgers. The foundation was careful in selecting an ECHO partner. It took the foundation “a bit of time, almost two years, to find a partner to lead this project.” This partner was the Robert Wood Johnson (RWJ) Partners Accountable Care Organization (ACO). At that time, Kathy Dodsworth-Rugani was the COO and tasked with leading Project ECHO at the organization, launching mid-2016.

Dodsworth-Rugani had a background in health care and experience in consulting and building service businesses from the ground up. From the start, she wanted to ensure that they “had the right amount of resources. If you don’t have people on board that can do the job and pay them so they want to do it and stay in the job, you end up having trouble.” Dodsworth-Rugani hired two “really strong people” one with a background in education and the other with advanced work in public health and evaluation. Dodsworth-Rugani invited Chicago and Colorado ECHO hub leaders to New Jersey to help kick off their efforts.

In 2018, RWJ Barnabas Health took over the RWJ Partners ACO when they acquired the Robert Wood Johnson University Hospital. Shortly after this acquisition, RWJ Barnabas closed down the ACO. ECHO would have been included in that shutdown had it not been for The Nicholson Foundation grant. With the leadership of Eric Jahn, MD, the senior associate dean for community health, who saw the value of ECHO and continued to promote ECHO with the leadership at the Rutgers schools, Dodsworth-Rugani moved ECHO, her staff, and the grant to its current home at the Rutgers RWJ Medical School. Transitioning to the medical school was a logical choice, but moving the team to an academic institution was not without its challenges. They needed to align the existing teams’ salary and titles in a new system and respond to new policies such as higher fringe rates and overhead.

The Nicholson Foundation Invested $4 million in ECHO over a five-year period. As of March 2021, their funding of the four ECHO programs ended, with The Nicholson Foundation in New Jersey closing down. Two additional programs were funded by the Department of Mental Health and Addiction Services at that time, another program by the Department of Health, and two by the Rutgers, New Jersey Medical School (NJMS). Another benefactor grant was replacing the Nicholson funding. The total annual budget for Rutgers Project ECHO was about $1.7 million per year and they had nine ECHOs running. As of fall 2021, the Rutgers Project ECHO and Telehealth Office, led by Dodsworth-Rugani, employed three program administrators, a full-time assistant director of evaluation and quality improvement, and a full-time finance and operations staff member. They had two open clinic coordinator positions.

Community Health Worker ECHO

The Community Health Worker (CHW) ECHO was started in collaboration with the New Jersey Department of Health (DOH) as a response to the COVID -19 pandemic. The state was looking for a way “to best support the various frontline workers and the outreach workers, to raise awareness and knowledge on the basics of COVID-19, and to identify the impacts that COVID-19 had on vulnerable populations and the community.” Lisa Asare, assistant commissioner of the Division of Family Health Services in New Jersey, said that ECHO was an “abstract concept” and she wasn’t initially “really sold” on it. When COVID struck, however, she was “pushed to ECHO. I had run out of options. I had nothing to lose.”  Others in the Department of Health shared a “heightened interest in the use of the ECHO platform” in response to COVID.

The CHW ECHO was in addition to an existing CHW educational plan where the New Jersey DOH Family Health Services Colette Lamothe-Galette Community Healthworker Institute (CLG-CHWI) was working with community colleges to provide “a more traditional learning experience where folks [CHWs] sat through hours of classroom time.”  The CHW ECHO complemented the existing program by focusing specifically on COVID and opening it up to frontline workers who might not be considered community health workers such as people in “food security, childcare, mental health, midwives, and doulas. These are people who interface with the populations that were at risk during the pandemic.”  ECHO became a way to “get timely information out to many people and give them real hands-on tools.”

Not all participants we interviewed had participated in ECHO Institute immersion training. The new program administrator went through onboarding and training with the hub and benefited from sitting in on other ECHO programs to see how ECHO was implemented and attended other trainings offered by the ECHO Institute at the University of New Mexico. Meg Fisher, MD, who was affiliated with the CHW ECHO, had attended immersion training when working on an infectious disease ECHO.

The CHW ECHO was initially funded through the New Jersey Department of Health. Sessions frequently included up to 200 spokes, with many coming to each session. The initial series had ended. At the time of this interview, the team expressed an interest in another series targeting the same group of first responder participants. [Note: Subsequent to the interview, and as a result of this pilot, Rutgers Project ECHO supported a proposal to the Centers for Disease Control and Prevention that has been awarded to the DOH for three years to continue and expand training of CHWs; ECHO was part of that solution.]

Complex Endocrinology ECHO

The Complex Endocrinology (Endo) ECHO was started in 2016 with the aim of providing a multi-disciplinary approach to treating patients with endocrine issues. The Endo ECHO had a panel of experts that included two endocrinologists, an advanced practice nurse, a peer support specialist, a clinical social worker, and a pharmacist. They were supported by a program administrator. Louis Amorosa, MD, was the lead endocrinologist and had been with the program for five years. He first learned about ECHO from the chair of the Department of Medicine who came to him and said, “Why don’t you participate in ECHO? This is a novel program. Get out to New Mexico and learn about it.” The Endo ECHO panel also included a pharmacist—Mary Bridgeman, PharmD. Bridgeman’s director said to her, “We think ECHO fits your wheelhouse. What do you think? We want you to come out to New Mexico.” Both attended immersion training and had been with this ECHO since its initial launch. The longevity of this program was attributed to a “handful of primary care providers who have been with us from the beginning.” These providers had repeatedly presented patient cases over the years. Amorosa said, “Their continuing willingness to engage in discussion is motivating for the hub and stimulated our persistence during times when our participant number drop to less than 20.”

Endo ECHO was initially funded through The Nicholson Foundation. That funding ended the first quarter of 2021 and it was funded through internal funds for the remainder of 2021. The team was looking for new funding sources.

The ECHO Model seeks to build a learning community where “all teach, all learn.” This is done by leveraging technology, by sharing best practices, though case-based learning, and using data. We asked respondents to tell us what “all teach, all learn” (ATAL) meant to them. In each interview, respondents discussed ATAL as every participant and expert bringing their knowledge to sessions so “everyone can learn from each other.” One respondent described ATAL as “walking into a room and sitting at a table where you don’t know who anyone is, you don’t know their titles, no distinguishing factors, and then having a very equal conversation and learning from each other.”  But it was more than bringing expertise; the collaborative use of this expertise was emphasized by many. One respondent said, “ECHO is like the puzzle that is patient care. Each of us has our own little piece of this puzzle. When we all get together you can realize the full picture.” This collaborative learning environment occurred in a “safe space” that “fosters relationships.” Dodsworth-Rugani emphasized, “Everyone has something to contribute, through their own knowledge and experience, and ECHO enables all voices to be heard.”  

ATAL was also described as gaining “the perspective of people who are at different places in their life, or in their jobs, or in their experiences.” Amorosa with the Endo ECHO said he gained a “perspective on what’s going on out there” and that “out there” does not always look like what he and others see in an academic medical center setting. “Out there” took on a different perspective when Asare described ECHO as a “litmus test for those of us working at the state and policy level. The policies we develop, are they working on the ground? Because according to these folks, the answer is ‘no,’ in some cases.” 

The Endo ECHO and CHW ECHO had similar and different ways of structuring how they achieved ATAL. Both used didactics. In the Endo ECHO, didactic topics were chosen by polling spokes and based on the expertise of a multi-disciplinary panel of medical experts. The Endo ECHO was experimenting with curricular blocks after having a “hodgepodge of content” that was nimble and responded to needs and opportunities. Didactics in the Endo ECHO were provided by the panel of experts, but if the panel was “uncomfortable with the topic” or there was “controversy” they invited external speakers because they “want to hear the other perspective.” The CHW ECHO also had didactics. Topics were chosen by the panel and reflected changing conditions and knowledge associated with COVID. The expert team in the CHW ECHO series included professionals outside of medicine (e.g., policymakers, community stakeholders) and participants that were frontline caregivers from all types of background.

In both ECHOs, didactic presenters were prepared in advance. For the Endo ECHO, this consisted of requesting the presentation in advance and a 30-minute pre-briefing to “talk about and ask questions of the presenter, any clarifying issues, so that we’re all unified in our presentation.” The CHW ECHO “created an onboarding guide and list that we provide to the hub members and presenters joining us every week,” developed a template for presentations, and met with presenters in advance. Because each week was a new presenter, staff “had to spend considerable time prepping presenters.”  One respondent commented that “ECHOs takes a lot of work. Otherwise, the ECHO session becomes a webinar.”

Spokes presented patient cases in the Endo ECHO, though it was a struggle to get cases—which they actively recruited, offering a $30 incentive. When they did not have a spoke presenting a case, the “hub will stand in, as a backup” though this was infrequent. In contrast, the CHW ECHO did not use or recruit cases. Initially they tried, but because this ECHO was not “clinically or medically based, we took a couple of different routes to create dialogue and participation with the participants.”  This included panel members presenting a client or patient scenario or story based on what they observed. Participants were given a “question submission link (via chat)” to submit their questions if they didn’t want to unmute.

Both programs raised concerns about cases. In the Endo ECHO they tried not use the word “case” all the time because it could mean, “Oh man, that’s too much work.” Spokes could send in a question and the team would put it into the template as they “try to make it as user friendly as possible.” The team had moved away from using the word “case” which was “not the right terminology when you have a diverse audience including dieticians, social workers, and case managers.” The team had been using the language of a “patient scenario” and asked spokes, “Tell us about a situation that you would like some advice on . . .” Similarly, the CHW ECHO didn’t ask for cases, but instead asked questions such as, “Is there anyone who has experienced this in their work or field? Is so, where are you from? What is the scenario?” 

Both programs were very focused on weaving the constraints and concepts of health equity into their sessions. They led discussions about access to care, structural racism, and addressing implicit biases.

Studies of program implementation identify contextual factors that can shape how a program was implemented. These factors include leaders and champions, state and federal policies, funding, partnerships, and internal organizational structures and processes, monitoring for quality and fidelity, and staffing—including how people were trained and the characteristics of the people leading and supporting the program.

Not all of these factors may play a role in how ECHO was implemented here or elsewhere, and some factors were more important than others. Below, we identify factors that emerged during interviews which influence how Project ECHO at the Rutgers Robert Wood Johnson Medical School, and the Complex Endocrinology and Community Health Worker ECHO programs were implemented.

Organizational Characteristics

Rutgers Project ECHO journeyed from foundation funding and a home in an ACO to a more diversified funding base and a home in an academic medical center. The ECHO hub reported to Jahn, senior associate dean for community health, and operated independently of other units in the medical school. There was “prestige of being in an academic medical center and having access to talented specialists” but buying their time came at a “serious cost.” In addition, the complexity of working in a large university system, including costs (fringe and overhead) impacts growth and competitiveness and could limit options. The Rutgers Robert Wood Johnson Medical School had gone through multiple changes since ECHO moved there, including changes in leadership and a more recent shift in leadership’s focus on wanting more RVUs per person which had some clinicians saying, “I can’t do ECHO. I have to focus on my clinical time.” And the change was not over as the Robert Wood Johnson Medical School was transitioning its clinical services to RWJ Barnabas Health and faculty were concerned about managing the increased demands on clinical time as well as balancing their faculty education and research commitments. The independence of the hub— it generated its own income and supported ECHOs inside and outside of Rutgers— coupled with experienced leadership enabled ECHO to grow despite organizational uncertainty.

Quality and Fidelity Support

Rutgers Project ECHO had a number of methods for ensuring fidelity to the ECHO Model, from having a “prep-session” prior to each ECHO to prevent technical issues and ensure everyone understood the session structure and timing, to providing presenters with an overview of ECHO along with tips and guidelines for presentations. Administrative staff were encouraged to “be very honest and take feedback that is actionable and do something with it.”  If the hub thought an ECHO was “starting to fizzle out” they were okay with “stepping in and going back to the drawing board. What worked? What didn’t work? What do we need to do to make it better? We’re not just keeping ECHOs running for the sake of keeping them running. We want them to resonate and meet the needs of providers because their time is so valuable.”  Rutgers Project ECHO hub had a full-time evaluation and quality improvement staff member, an indication that evaluation and quality improvement played an important part in their programs.

Leadership

The Nicholson Foundation took the initial leadership for ECHO; they were “100 percent committed.” Their commitment extended to having Arora come to New Jersey twice to educate and influence key stakeholders. The Nicholson Foundation was in a position where they could take their time to find the “right” partner. Dodsworth-Rugani was a strong leader, and the foundation continued to be her advocate through a five-year relationship. Dodsworth-Rugani led the ECHO hub and programs through many changes. She had invested in maintaining a strong team over time and building relationships with multiple state organizations to create a coalition of organizations that worked together to address public health issues through collaboration in ECHO.

Partnerships and Networks

The Rutgers Project ECHO hub had many strong relationships inside and outside of Robert Wood Johnson Medical School at Rutgers University. Their network on campus extended to “star faculty” who were trained educators in addition to medical experts that provided a multidisciplinary approach to addressing the needs of participants. While this internal network was important and strong, Rutgers Project ECHO also had strong relationships outside the institution. Multiple external partners were identified during interviews including the Department of Health, the Department of Mental Health and Addiction Services, the state’s Medicaid office, the Division of Family Health Services, the State Center for Health Policy, and The Nicholson Foundation. Some of these relationships began years ago when The Nicholson Foundation invited state officials to join them at immersion training. These external relationships, especially with state agencies, helped with the flexibility needed to pivot ECHO series toward urgent issues related to the COVID-19 pandemic. They had also built relationships with providers across New Jersey, including other medical schools and health systems, which allowed the ECHO team to draw on experts outside of the RWJ Medical School.

The partnership with The Nicholson Foundation was especially impactful in shaping Rutgers Project ECHO. The foundation, and in particular Raquel Jeffers, former program officer for The Nicholson Foundation, provided guidance and input on strategy, sustainability, building relationships across the state, managing the financial budgets, and use of social media—all of which helped build the model that exists today.

The Rutgers Project ECHO hub has relationships with other ECHO hubs, with mention made of talking through issues with leaders at the Chicago and Colorado hubs. Staff attended ECHO trainings and engaged in collaboratives. The Endo ECHO taps into a larger community of endocrinology ECHOs, especially the ECHO Institute’s Endo Collaborative.

Funding

The Nicholson Foundation invested $4 million over a five-year period of time (2016-2021). These funds supported the initial development of the ECHO hub, including market research, hiring staff to support three ECHOs throughout the five years, launching a successful pilot on electronic consults, building an initial quality improvement and evaluation protocol, building QI processes, and providing payment to the faculty hub team of subject matter experts who lead the discussions.

That funded had ended. A reliance on grant funding was problematic as “each grant funds only a portion, so you never really get ahead.” Still, grants were important to keeping the nine ECHOS and $1.7-million-dollar operation going. Many of these grants were “seeds planted during immersion” that had grown into multiple grants and sponsorship for ECHO programs. Dodsworth-Rugani was spending about 40 percent of her time on development that reinforced current partnerships and created new partnerships—which could keep the funding gates open.

The Rutgers Project ECHO hub was engaged in a strategic planning process and focused on broadening their services. They were considering “diversifying our ECHOs so that they’re not all focused on healthcare.”  This included education ECHOs, citizen ECHOs, and ECHOs to meet the needs of the incarcerated. They may also focus on additional training ECHOs. The CHW ECHO provided “the education that enabled participating doulas to qualify for certification from the Medicaid department, which then enables them to bill for their services.” This was a “different model from our current traditional ECHOs and we want to do more of that.” Training that leads to certification may also generate revenue by charging a fee— something that was being considered. And they were considering offering new ECHOs to address clinical needs, such as an ECHO focused on the high maternal mortality rate for Black women in new Jersey and “doing something with patients and families, not just providers, would be another interesting place to go with ECHO.”

The future was also about expanding relationships. Dodsworth-Rugani said she wanted to have further discussions over the next year with other institutions in Rutgers and the chancellor to find out whether or not they could elevate ECHO to a point where the other schools saw ECHO as a platform they could utilize for their education and training requirements and where they could engage more subject matter experts in addressing the current public health issues they collaborated on. An important step in this direction was research gathered from patients associated with primary care doctors who attended the Endo ECHO program. This study, conducted by health science outreach researchers at Rutgers University, showed a positive impact on patient health outcomes. Full results of this study were expected to be published in 2022.

Dodsworth-Rugani shared that the future was likely to include more competition. She anticipated increased competition in the virtual education space. When ECHO was first getting started, it was novel, especially the use of Zoom, but “that’s a day gone by.” Competition could increase among ECHO hubs. New ECHO hubs were opening in New Jersey but there wasn’t coordination among them. Dodsworth-Rugani had heard about a new ECHO hub in the region, but no one from the new hub or the ECHO Institute told her about it. The ECHO space was a bit of a “free-for-all in terms of trying to maintain your space.”   

Showing results could help make ECHO more fundable and ease competitive tensions. To this end, a future vision of the Rutgers Project ECHO was to have more evaluation support and to move away from solely qualitative data analysis and moving toward more data-driven quantitative evaluation.

The Endo ECHO’s future was “going to depend on funding.” Losing this ECHO would “create a hole in our community.” They were reaching out to the state, pharmaceutical companies, and other foundations for funding opportunities, since they had data to show that ECHO worked to change hospitalization costs in this population. They were considering whether or not “payers within the state would take an interest in ECHO support.” Beyond funding, they were concerned about keeping the work “fresh” and the balance between “delving deeper into complex ENDO issues” with the “general needs of a primary care–focused audience.” 

The Community Health Worker ECHO wanted to run another series that built on the momentum and the community created during the initial offering. The pandemic brought together multiple stakeholders where “all needed information. You had a piece of information that I didn’t have, but if I gave you mine and you gave me yours, we would both be stronger, and we would be better able to serve our communities and be resilient.” Continuing with the CHW ECHO was viewed as a means to prepare for the future “because if it’s not the pandemic, it will be something else tomorrow.” The program needed Department of Health support to continue. Subsequent to our interview, Asare, and her department—in collaboration with multiple organizations across the state, including the Rutgers Project ECHO and the School of Public Health, four local junior colleges, Seton Hall, and others—secured funding for the program’s continuation.

Louis Amorosa, MD
Rutgers Robert Wood Johnson Medical School
Robert Wood Johnson University Hospital

Lisa Asare, MPH
Assistant Commission in the Division of Family Health Services
NJ Department of Health

Mary Bridgeman, PharmD
Internal Medicine Clinical Pharmacist
Robert Wood Johnson University Hospital

Kathy Dodsworth-Rugani, PhD
Executive Director, Rutgers Project ECHO and Telehealth
Rutgers Robert Wood Johnson Medical School

Ashley Figueroa
Program Administrator, Rutgers Project ECHO
Rutgers Robert Wood Johnson Medical School

Amy Fisher
Program Administrator, Rutgers Project ECHO
Rutgers Robert Wood Johnson Medical School

Meg Fisher, MD
NJ Department of Health, Office of the Commissioner

Raquel Jeffers
Former Program Officer, The Nicholson Foundation

Beiler, J. & Larson, R.S. (2022). Project ECHO at the Robert Wood Johnson Medical School, Rutgers University Implementation Profile. Diffusion Associates. https://www.diffusionassociates.com/echo.