Reflecting on the life of the Opening Pathways grant

In a two year grant project with hundreds of conversations, it’s hard to reflect articulately and summarize the most impactful moments and key things we’ve learned in a single post. Thankfully, we’ve been documenting in this blog along the way, which helped us track progress and helps us now in seeing the meandering paths that this grant has taken. I say meandering, because one of the things I included in my final grant report to RWJF was advice to other grantees to ‘build in extra time for other important work that will arise throughout the life of the grant, and have flexibility in your proposed work to be able to address emergent work’. Like our other reports to RWJF, you can read the full text of our final report at the bottom of the post, but I wanted to use this post to also reflect more broadly about the life of this grant, project, and team.

One of the frameworks we’ve used in reflecting on this project has been an analogy. Paul has described that when he’s been discussing the Opening Pathways work inside RWJF, he’s asked: What happens when a new ‘lifeform’ is introduced into an existing - and closed - ecosystem? What changes? Who changes? What are the ripple effects?

Logistically speaking

Positioning me as a patient PI as the new ‘lifeform’ in the ecosystem of traditional healthcare research was challenging from the start. For example, what ‘shape’ does the lifeform take? For me, that question was largely a practical one: How do I get the money to do the work? Do I create an organization to receive the grant? How much headache would that be? Can I even do that? Or, is there another way using another institution to receive the money, and then I can subcontract to get the money, while the institution handles the paperwork? We tried the latter method for this grant, with ASU kindly serving as the institution that ‘received’ the grant, because originally both of my co-PI’s (Erik Johnston and Eric Hekler) and the rest of the team members were at ASU.

How did that work? It did allow us to get the grant and start the project: RWJF gave ASU the money, but as I’ve mentioned, there’s been a lot of paperwork headaches throughout the entire lifecycle of the grant with me and other subcontractors getting paid, getting the financial reports submitted back to RWJF, etc. It’s been challenging to feel and hold responsibility for the grant as PI, but not have any actual power to push ASU along any more quickly. Thankfully, Erik Johnston has been championining internally at ASU and trying to make it work to the best of his ability, but one of his reflections has been noting the difference between ‘allowing’ such a project to happen and fully supporting and encouraging such a project, with such a structure, to happen. This has been an issue all the way to the end of the Opening Pathways grant.

As a result, one idea that has emerged is that of using a more traditional fiscal sponsor organization for future patient-driven projects. Instead of having to create your own organization, you would run a grant through a fiscal sponsor organization. They’re already designed to be a pass-through institution, so theoretically there would be less bureaucracy and more support for the PI using this method. I have another emerging project in this space, in which we are experimenting with using a fiscal sponsor as the grantee institution, and while we have not received a decision on funding yet, the application process has been smoother and easier. This may be partially related to my increased level of experience in applying for grants, but either way so far it’s been a good experience, and I do think it’s worth exploring as another alternative model. I plan to continue exploring it myself and sharing my experiences.

Thankfully, the structure of this grant and the funding mechanism was probably the hardest and most time consuming part of the grant.

Unfortunately, despite my attempts to give feedback to ASU about the process of being a community PI on a grant through their organization, I don’t think it’s gone anywhere, so I don’t feel like we’ve affected change at ASU as a direct result of my work there. However, I do think the best impact for ASU has been having a co-PI, Erik Johnston, at ASU who was and has been impacted by this work. He has written future grants with some of these learnings in mind with regards to building equitable relationships in the communities they propose to partner with in the future. I hope this ripple effect at the individual level of PI’s adjusting their grant work impacts the ecosystem player that is ASU and similar organizations in the long run, but that’s a longer tail effect than I was hoping to see as a result of our work.

How did the ecosystem react?

Overall, my conclusion is that I don’t think the ecosystem really noticed overall or cares that there’s a new ‘lifeform’ in action.

Like with OpenAPS, n=1 is not that meaningful (to anyone other than the ‘one’); people ask, “So, what?” and “Does it work for others?”. Thankfully, like OpenAPS, n=1 is becoming noticeably (n=1)*many with regards to patients leading research. Many patients are now becoming PI’s or Co-PI’s with grants from RWJF, PCORI, and other funding institutions. I can confidently say that funding patients as PI’s does work, is meaningful, and creates incredible results. But for the ecosystem to really adapt, we need many more of us ‘lifeforms’ to start altering the ecosystem. I don’t think our single lifeform (grant) is enough to really answer the question of what happens when we are added to a closed ecosystem. That’s one of the reasons we created the patient pathways site to help connect and encourage patients; and that’s also why we created the partner toolkit to help support and bring more partners for patients into this new way of working.

Those sites were heavily influenced throughout this project’s evolution, and potential impact on the ecosystem, by my growing realization that the most important work we were doing was not the final ‘product’ of the grant, as I had originally thought, but the way in which we were working, and the many, many ‘hard’ meta-conversations we were having about power dynamics in the traditional ecosystem. Paul and I both reflected on the power dynamics specifically between the funder (RWJF/Paul) and the grantee (me/Opening Pathways team) in two blog posts, where we each asked questions and answered them, then had a chance to respond to each other’s posts further. Here is my post; and here is Paul’s, if you missed them previously. In terms of ripple effects, Paul’s post was posted internally at RWJF and I did not see any direct follow up discussion that may have happened there. But I am hoping that some of the discussions Paul started internally at RWJF as a result of our conversations, and these posts in particular, have their own ripple effect and yield more discussions at RWJF as they (hopefully) consider funding other non-traditional and patient PI’s to lead subsequent work in similar fields that align with their vision.

How I’ve personally evolved

One specific reflection on my individual role as a lifeform in this ecosystem is that how other people perceive me is noticeably different than before. I am perceived and treated differently as a “RWJF-funded researcher.” It is apparently being used as a proxy for legitimacy and in place of more traditional credentials (e.g. a role at a traditional university). I have mixed feelings about this, because I think credentials and their proxies should be used far less often, vs. direct evaluation of actual work or contributions. But it has been a noticeable effect, more so than the effect overall of our project on the ecosystem. After discussing this periodically throughout the grant, Paul commented that the ecosystem likes clearly defined roles and individuals - and does not do well adapting or dealing with fluid identities or roles. This may also be a generational thing, as it is more generally in society today, where the current established and in-power generation is less accepting of fluidity but younger/newer generations are more accepting and able to deal with the resulting impacts of such fluidity. As a new lifeform, the current ecosystem was more accepting and open to me once I had a clearly defined role as a “grant-funded researcher”.

Overall, I do feel like I have evolved and learned a lot in my first-time PI role, and my comfort level with “being a researcher” has definitely grown. I’m incredibly thankful for the team’s support as I evolved in this role and in this project: everyone supported my growth and learning without fail, and adapted to support me as I evolved, which took time and effort that I greatly appreciate. This was probably less noticeable to the ecosystem, but important for my future work.

How our team evolved

I also noticed other members of the team growing in ways they probably did not expect. At the beginning of the project, I badgered everyone on every phone call about writing blog posts to actually capture the amazing team conversations that we were having. That transitioned into many team conversations about writing, and I was thrilled to see each team member writing more and more throughout the life of the project, and also improving their writing skills and writing more frequently in other projects and channels as a result.

Throughout our journey, this amazing team was key to the successful completion of this grant-funded project. I have been trying over time to figure out what those elements are that made it so easy to work with the team, and asked John to write this post a few months ago to try to capture some of the elements. He articulated that there was some pre-existing team culture that played a role, which I previously had not been aware of, and which I do think definitely helped. But I also think the humility with which Eric, Erik, and John (and Paul as well) approached all of our interactions and conversations also allowed for a meaningful exploration of the power dynamics between patients and traditional ecosystem lifeforms. This became apparent when I asked them to share their narratives of this work. As you can see there, and from Eric’s previous post reflecting on how we asked him to participate at the Convening, they have all been phenomenally humble, open, and receptive to thinking differently in this work.

As PI, I have also tried to do the same, including being clear about where I feel my limitations and concerns are, and asking them to be explicit throughout the lifecycle of this grant about what happens typically on their other grant-funded projects, and why, so that we could make conscious and thoughtful decisions about how this grant might be different, and why. These conversations were incredibly helpful not only for developing our team relationship and the project’s work, but also in fueling future collaboration and future work in this space. This has continued until the very end of our grant: at our last wrap-up call for this project, I specifically asked for each of us to make a list of “offers” of support for the future, as well as any “asks”, and to be explicit as possible. This fueled a valuable conversation about not closing off the relationships we have developed and want to continue in the future just because this grant is coming to an end. I’m very cognizant of the fact that in the traditional academic world, these relationships and collaborations continue naturally after grants end. But in my role outside of the academic world, it’s not something I felt I could take for granted. I was glad we had an explicit discussion around each person their offering future support in specific ways, and that everyone agreed and volunteered to stay connected in the future. So while the grant is now officially over, I can confidently say that we’re not done with this work - this is just the beginning for the next chapter of this evolving ecosystem and opening more pathways for research, discovery, and innovation in health and healthcare.

---

(And here’s the full report of our final RWJF report that was submitted)

1. What was the goal of your project? Do you have measures of your performance?

The project met its goals:

• We enabled submissions and requests to work with our data science team, and commenced work on several projects, which resulted in one major scientific submission, acceptance, and presentation of a poster at an international scientific/medical conference (see Lewis DM, Leibrand S, Street TJ & Phatak SS. Detecting Insulin Sensitivity Changes for Individuals with Type 1 Diabetes. 2018;67(suppl 1). Available at: https://doi.org/10.2337%2Fdb18-79-LB). 
  • However, we originally theorized success as answering one dozen research questions. We did not receive that many submissions; and many did not follow up after expressing original interest. We did, however, successfully engage those who continued to work with us. 
  • As a result, we have also developed new hypotheses for the most efficient way to fund an initiative to encourage data science around patient research questions, by providing mini-grants to community members with the data science and/or technical experience to partner with the patients with data science questions, along with a specified timeline, to test in the future.
• We hosted a convening for traditional and non-traditional researchers, innovators, and advocates, carefully designing the meeting to ensure equitable participation.
  • This meeting is continually talked about by participants as the most impactful meeting they attended for the year. 
  • The conversation from the meeting resulted in the design of the patient toolkit.
  • The conversations from the meeting also resulted in the idea for the humility audit/readiness quiz, for which we extended the grant for 6 months in order to complete. 
    • The quiz is already successfully being used by individuals to assess potential partners for their work.
    • The quiz is also being proposed to be used by an entire organization in its work to balance power dynamics between the organization and community partners. 
• We developed the toolkits as full open source websites (two, instead of the one originally proposed) for patients https://patient.openingpathways.org/ and partners (https://partner.openingpathways.org/). 
• We created 32 blog posts (http://OpeningPathways.org/blog) from various members of the team reflecting on the work and the meta-themes of this work.

2. Do you have any stories that capture the impact of this project?

Another patient has been funded as a PI on a grant project as a result of this project.

Additionally, the humility audit was originally used within our team. It was so powerful that we developed it into an external tool (http://partner.OpeningPathways.org/quiz), and plan to share it externally and encourage other people and organizations to use it. However, before the tool was even finalized and publicized, another organization (ASU) has included it within a proposal for another grant around community-driven work as a mechanism for balancing power dynamics with community partners for future projects. 

In addition, the concept of community-driven design is increasingly becoming a foundational part of how the UC San Diego Design Lab is advancing the use of design for helping people. Within this, there is a strong grounding, based in part on the lived experience Dr. Hekler had in this project, on seeking to counteract confident ignorance and emotional blindness towards true meaningful equitable participation, contribution, and benefit from design. 

3. Did RWJF assist or hinder your project in any way?

RWJF assisted significantly via our project officer, Paul Tarini. His practice of scheduling monthly calls with the PI and Co-PIs allowed for rich, meta-conversations about power dynamics and other impacts of our work, and allowed us to explore new aspects of this work that we would not otherwise have been likely to see.

This practice has been so helpful that a co-PI (Johnston) has specifically put this request into future grant proposals to build space and time for developing these types of relationships and conversations into other grants that he works on in the future. It’s also a practice the other PI’s (Lewis, Hekler) plan to continue as well.

4. If the project encountered internal or external challenges, how did they affect the project and how did you address them? Was there something RWJF could have done to assist you?

Most of the challenges have arisen from entrenched power dynamics in healthcare and research. RWJF has been a good ally (and in particular, our project officer Paul Tarini has done a great job) in helping us address the power dynamics as they influence this work and understand the ripple effects outside of this work.

However, we do not think these challenges have been eliminated from others doing this work, and Is something RWJF and all project officers should keep in mind moving forward when funding this type of work with non-traditional or patient PIs.

The other challenges existed as a result of the way the grant was set up at an institution with a patient PI who does not sit inside that organization. RWJF did what it could to encourage individuals at the institution to try to smooth processes; however, it is something that RWJF and other project officers should also be aware of and try to address proactively in future projects funded with non-traditional, new, or patient PIs who sit outside the main institution hosting the grant.

5. Has your organization received funding from other foundations, corporations or government bodies for the project RWJF has been supporting?

No.

6. When considering the design and implementation of this project, what lessons did you learn that might help other grantees implement similar work in this field?

• Balancing power dynamics in the team is important, and should be addressed from the start and throughout the life of grants. 
• Build in extra time for other important work that will arise throughout the life of the grant, and have flexibility in your proposed work to be able to address emergent work.

Paul and (PI Lewis) exchanged other thoughts and feedback regarding the funder-grantee relationship in written form from each of our perspectives in parallel blog posts. Please see http://openingpathways.org/reflections-funder-grantee-relationship and http://openingpathways.org/reflections-funder-grantee-relationship-funder-perspective .

PI Lewis has also shared advice on our project site (see this post) for others doing patient-driven work from non-traditional perspectives, and via this post about some first-time PI learnings.

7. What impact do you think the project has had to date?

Provide evidence for all statements (e.g., publication in major journals, citations of the project in literature, major press coverage, adoption of the model by other organizations).

The ‘humility audit’/’readiness quiz’ is being proposed to be adopted by ASU for use in a William T. Grant application, to be used to balance the power dynamic between ASU and community members and organization for joint research and projects moving forward.

The team also developed a commentary on advancing a small data paradigm based in part of this project; particularly the “flipped paradigm” of helping individuals first as a strategy for advancing science.

The patient (https://patient.openingpathways.org/) site is already being used by patients to find inspiration and others like them doing similar work, and the partner toolkit (https://partner.openingpathways.org/) is also being used by patient-partner duo’s to strengthen partner relationships and vet potential partnerships.

(8 & 9 not included because they have personal contact information for follow up)

10. What are post-grant plans for the project, if it does not conclude with the grant?

We plan to continue to share & promote use of the readiness quiz for partners; for more partners to share their narratives; and for more patients to share their pathways. The sites are all open source, so we hope to see more people take and build upon the content and tools developed.

11. With a perspective on the entire project, what were its most effective communications and advocacy approaches, its key publications, and its national/regional communications activities?

Most effective communications have been keynote presentations by the PI that reference this work. For example, PI Lewis gave a presentation in June at the mHealth Training Institute in June 2019, discussing the Opening Pathways work, and was invited to give a similar presentation at ASCO (the major scientific cancer meeting) for cancer researchers & providers in 2020.

Similarly, PI Lewis’s Twitter activity announcing the grant, during the grant, and launching the final products has been highly effective. See this Twitter thread regarding the project announcement that received 19,000 impressions, this thread launching the patient & partner sites, this thread highlighting the initial patient stories shared, and this thread highlighting the partner narratives as examples.

Key publications include the Opening Pathways blog itself for distributing our work widely among community partners who share in similar work, and doing so throughout the entire life of the grant and not only at key announcement points.