I’m Lindsey, and this is Debbie. How can we get more Veterans better without adding hours in our day? To get Veterans better, we have to account for the number of hours available in the day. Yet because we don’t have the right information about how we manage our time, we can pursue unrealistic treatment plans or improvement strategies that violate the laws of physics. Modeling to Learn encourages us to account for clinician time differently, to make our plans feasible locally. Most VA data systems offer snapshots of cross-sectional data, but this kind of information doesn’t help us account for the dynamics of evidence-based care over time. While many VA data systems are available, most display either quarterly lags from a high-level view or an in the weeds patient by patient view. Modeling to Learn helps to bridge these two views. Modeling to Learn introduces trends over time based on the clinic selections carefully reviewed by the staff on site. Trends help us zoom out to understand how we’re doing for most of the Veterans we serve. Modeling to Learn also introduces time-based definitions of care in the units of patients per week, appointments per week, or episodes of care per week. We use a week because over the last several years clinicians told us this is how they think in the clinic. There often is no typical day and a lot has happened by the end of a month. Defining care by week is more beneficial. High quality care in the clinic is always constrained by the available staffing capacity and the hours in the day to see patients, the number of weeks between visits, and the weeks of engagement over time. But do you have time to run these equations in your head or calculate them from the other dashboards you use? As an alternative, Modeling to Learn puts these components of care together to understand their interdependence over time. How can Modeling to Learn consultation help? Watch that video to find out.
+How can we get more Veterans better without adding hours in our day? To get Veterans better, we have to account for the number of hours available in the day. Yet because we don’t have the right information about how we manage our time, we can pursue unrealistic treatment plans or improvement strategies that violate the laws of physics. Modeling to Learn encourages us to account for clinician time differently, to make our plans feasible locally. Most VA data systems offer snapshots of cross-sectional data, but this kind of information doesn’t help us account for the dynamics of evidence-based care over time. While many VA data systems are available, most display either quarterly lags from a high-level view or an in the weeds patient by patient view. Modeling to Learn helps to bridge these two views. Modeling to Learn introduces trends over time based on the clinic selections carefully reviewed by the staff on site. Trends help us zoom out to understand how we’re doing for most of the Veterans we serve. Modeling to Learn also introduces time-based definitions of care in the units of patients per week, appointments per week, or episodes of care per week. We use a week because over the last several years clinicians told us this is how they think in the clinic. There often is no typical day and a lot has happened by the end of a month. Defining care by week is more beneficial. High quality care in the clinic is always constrained by the available staffing capacity and the hours in the day to see patients, the number of weeks between visits, and the weeks of engagement over time. But do you have time to run these equations in your head or calculate them from the other dashboards you use? As an alternative, Modeling to Learn puts these components of care together to understand their interdependence over time. How can Modeling to Learn consultation help? Watch that video to find out.
Hi, I’m Lindsey and this is David. How does Modeling to Learn benefit substance use disorder, or SUD programs? When Modeling to Learn began almost 10 years ago, as a partnership among patients, providers and policy makers across VA, we worked iteratively to define the dynamics of common care problems for the primary reasons Veterans seek addiction and mental health care. We found that if we focused on improving timely, high-quality evidence-based psychotherapy and pharmacotherapy for alcohol use disorder, depression, opioid use disorder and PTSD, we could support VA staff in meeting around 80% of Veterans needs for behavioral healthcare. We work with frontline teams and partners from the VA Academic Detailing Program and the Psychotropic Drug Safety Initiative or PDSI and developed the Medication Management Module of Modeling to Learn. This module helps manage the dynamics of evidence-based pharmacotherapies for alcohol use disorder and opioid use disorder based on their effectiveness in reducing craving, relapse, and overdose. You can find the Medication Management module in Modeling to Learn data mtl.how/data and Simulation User Interface mtl.how/sim. The Modeling to Learn Medication Management module supports improvement in the AlcTop or Alcohol Topiramate indicator from the Mental Health Information System or MIS and the SUD 16 indicator from the Strategic Analytics for Improvement in Learning or SAIL. How does Modeling to Learn help? Prescribers describe the need to better visualize how to locally optimize their medication management appointment supply, including top of license care from staff with and without a Drug Enforcement Agency X-waiver for OUD medication. Modeling to Learn helps a team assess this appointment supply, new patient start rate, no show or missed appointment rate, and the clinically appropriate return-to-clinic visit interval needed for a therapeutic response to alcohol use disorder and opioid use disorder medications. The Medication Management module helps teams and sites find the optimal number of patients who can be engaged in AUD and OUD therapies over time and helps them distinguish the flow of their patients who require antidepressants or other medication needs. Given that community needs and team staffing is dynamic and can change over time, these resources can empower teams to find local improvements that otherwise may be hard to find. Another common module for addressing Veterans SUD needs for group and other therapies is the Team Care module, which helps teams assess and optimize their overall mix of local multidisciplinary services as a function of staffing and patient needs. We are evaluating how Modeling to Learn improves Veterans who start and complete a therapeutic course of medication for AUD and OUD with research funding from the VA and the National Institute on Drug Abuse or NIDA. What about other common SUD comorbidities and presenting concerns? For example, how does Modeling to Learn benefit PTSD, clinical teams or PCTs? Watch that video to find out.
+How does Modeling to Learn benefit substance use disorder, or SUD programs? When Modeling to Learn began almost 10 years ago, as a partnership among patients, providers and policy makers across VA, we worked iteratively to define the dynamics of common care problems for the primary reasons Veterans seek addiction and mental health care. We found that if we focused on improving timely, high-quality evidence-based psychotherapy and pharmacotherapy for alcohol use disorder, depression, opioid use disorder and PTSD, we could support VA staff in meeting around 80% of Veterans needs for behavioral healthcare. We work with frontline teams and partners from the VA Academic Detailing Program and the Psychotropic Drug Safety Initiative or PDSI and developed the Medication Management Module of Modeling to Learn. This module helps manage the dynamics of evidence-based pharmacotherapies for alcohol use disorder and opioid use disorder based on their effectiveness in reducing craving, relapse, and overdose. You can find the Medication Management module in Modeling to Learn data mtl.how/data and Simulation User Interface mtl.how/sim. The Modeling to Learn Medication Management module supports improvement in the AlcTop or Alcohol Topiramate indicator from the Mental Health Information System or MIS and the SUD 16 indicator from the Strategic Analytics for Improvement in Learning or SAIL. How does Modeling to Learn help? Prescribers describe the need to better visualize how to locally optimize their medication management appointment supply, including top of license care from staff with and without a Drug Enforcement Agency X-waiver for OUD medication. Modeling to Learn helps a team assess this appointment supply, new patient start rate, no show or missed appointment rate, and the clinically appropriate return-to-clinic visit interval needed for a therapeutic response to alcohol use disorder and opioid use disorder medications. The Medication Management module helps teams and sites find the optimal number of patients who can be engaged in AUD and OUD therapies over time and helps them distinguish the flow of their patients who require antidepressants or other medication needs. Given that community needs and team staffing is dynamic and can change over time, these resources can empower teams to find local improvements that otherwise may be hard to find. Another common module for addressing Veterans SUD needs for group and other therapies is the Team Care module, which helps teams assess and optimize their overall mix of local multidisciplinary services as a function of staffing and patient needs. We are evaluating how Modeling to Learn improves Veterans who start and complete a therapeutic course of medication for AUD and OUD with research funding from the VA and the National Institute on Drug Abuse or NIDA. What about other common SUD comorbidities and presenting concerns? For example, how does Modeling to Learn benefit PTSD, clinical teams or PCTs? Watch that video to find out.
Hi, I’m Lindsey and this is Debbie. How does Modeling to Learn benefit PTSD clinical teams, or PCTs? Since I’m at the National Center for PTSD, this priority is very close to my heart. When Modeling to Learn began almost 10 years ago as a partnership among patients, providers and policy makers across VA, we worked iteratively to define the dynamics of common care problems for the primary reasons Veterans seek addiction and mental health care. We found that if we focused on improving timely, high-quality evidence-based psychotherapy and pharmacotherapy for alcohol use disorder, depression, opioid use disorder and PTSD, we could support VA staff in meeting around 80% of Veterans needs for behavioral healthcare. Modeling to Learn helps teams better evaluate the local dynamics of balancing the needs of new and existing patients for evidence-based psychotherapies, particularly cognitive processing therapy and prolonged exposure. Teams described the challenges of starting new patients who had never had their needs met without compromising care for Veterans already engaged in therapy. Modeling to Learn enables teams to see where their Veterans get stuck, to decide when to graduate patients, or to recognize when the weeks between return to clinic visits start to get too long. Have you been trying to improve PTSD 56 in the Strategic Analytics for Improvement and Learning, or SAIL? We worked with local PTSD clinical teams from the beginning and have supported many more teams over the years who have engaged with the evidence-based psychotherapy programs or the PTSD mentorship program to meet these needs. The Psychotherapy module of the Modeling to Learn data user interface, and simulation user interface helps teams to see how they’re doing for patients who are starting psychotherapy, how many flow through to complete a therapeutic dose, and whether Veterans are getting near weekly therapy consistent with the PTSD treatment evidence base. There’s also the ability to assess emerging approaches such as masked or intensive outpatient psychotherapy and other scenarios that are commonly considered and implemented in PCTs. You may also be wondering about the flow from primary care or from primary care mental health integration, or PCMHI, teams into general mental health and specialty mental health programs like PCTs. Stepped care up and down the mental health continuum of care can also be optimized to local needs using the Modeling to Learn Team Flow module. Speaking of the care continuum, you may be wondering How does Modeling to Learn benefit Behavioral Health Integration program or BHIP teams? Watch that video to find out.
+How does Modeling to Learn benefit PTSD clinical teams, or PCTs? Since I’m at the National Center for PTSD, this priority is very close to my heart. When Modeling to Learn began almost 10 years ago as a partnership among patients, providers and policy makers across VA, we worked iteratively to define the dynamics of common care problems for the primary reasons Veterans seek addiction and mental health care. We found that if we focused on improving timely, high-quality evidence-based psychotherapy and pharmacotherapy for alcohol use disorder, depression, opioid use disorder and PTSD, we could support VA staff in meeting around 80% of Veterans needs for behavioral healthcare. Modeling to Learn helps teams better evaluate the local dynamics of balancing the needs of new and existing patients for evidence-based psychotherapies, particularly cognitive processing therapy and prolonged exposure. Teams described the challenges of starting new patients who had never had their needs met without compromising care for Veterans already engaged in therapy. Modeling to Learn enables teams to see where their Veterans get stuck, to decide when to graduate patients, or to recognize when the weeks between return to clinic visits start to get too long. Have you been trying to improve PTSD 56 in the Strategic Analytics for Improvement and Learning, or SAIL? We worked with local PTSD clinical teams from the beginning and have supported many more teams over the years who have engaged with the evidence-based psychotherapy programs or the PTSD mentorship program to meet these needs. The Psychotherapy module of the Modeling to Learn data user interface, and simulation user interface helps teams to see how they’re doing for patients who are starting psychotherapy, how many flow through to complete a therapeutic dose, and whether Veterans are getting near weekly therapy consistent with the PTSD treatment evidence base. There’s also the ability to assess emerging approaches such as masked or intensive outpatient psychotherapy and other scenarios that are commonly considered and implemented in PCTs. You may also be wondering about the flow from primary care or from primary care mental health integration, or PCMHI, teams into general mental health and specialty mental health programs like PCTs. Stepped care up and down the mental health continuum of care can also be optimized to local needs using the Modeling to Learn Team Flow module. Speaking of the care continuum, you may be wondering How does Modeling to Learn benefit Behavioral Health Integration program or BHIP teams? Watch that video to find out.
Hi, I’m Lindsey and this is David. How does modeling to learn benefit Behavioral Health Integration Program or BHIP teams? What are the challenges with providing excellent team care? One challenge is the data sources to support team decision making jump from the individual patient and clinician counter all the way to the entire clinic or facility. It can mean that teams are flying blind to how they’re coordinating care for their Veterans. This can be especially hard when teams are short staffed or when staff, patient needs, and local policies continually change. It’s hard to know what improvements you might be able to find in your own team for coordinating evidence-based psychotherapies and pharmacotherapies over time. One way that Modeling to Learn can complement other BHIP resources is through the Data User Interface available at mtl.how/data. The Clinic Selection and Team Flow Selection tabs empower teams to carefully review the clinics that comprise their data views. Data are never all good or all bad. Modeling to Learn emphasizes transparency. Teams need to know what data they’re getting and its strengths and weaknesses for guiding a particular decision. The Clinic Selection tabs update every day to reflect all the Mental Health 500 series Stop Code, Clinic, or Grid updates that may be going on locally. Teams can save bookmarks that reflect their specific clinic selections, for example to include trainees or to select any of the clinics they have used within the last two years. This can be helpful when new BHIP teams are being formed or remapped. The team Patient Data tabs update daily too, so teams can view the patient diagnosis, encounter, health factor, measurement-based care, and high-risk flag information all in one place, and then the team can zoom out to filter and view these data as team trends. Coordinating care within a team is challenging. No single clinician can offer all the services a Veteran may need, so treatment plans must be coordinated in real time, over time, to support Veteran improvement. Clinicians may not have visibility on exactly how appointment supplies divvied up into intake evaluations, individual and group psychotherapy, medication management, or other adjunctive supportive therapies. What is particularly important about the Modeling to Learn simulation user interface at mtl.how/sim is the ability to assess not only the appointment supply, but even more important, the service proportions of patients that flow through to these services after intake or treatment plan review. One of the most common scenarios teams assess is how to respond to patient needs when they may not have the perfect staffing mix. Even when you know patient flow through care is divvied up into a mix of services, the dynamics over time are too much to understand in your head. The Modeling to Learn simulation user interface enables teams to quickly assess the dynamics of available appointment supply, service proportions, and the other two major factors that govern patient improvement, the return-to-clinic visit interval in weeks and the overall duration in care. We’ve often used Modeling to Learn to help teams find a way to ensure evidence-based care through the decisions that they make all day and find improvements in quality of care and quality of work life that they didn’t think was possible due to limited staffing. You might wonder How does Modeling to Learn support stepped care up to BHIP from primary care or primary care mental health integration, or from general mental health up or down into specialty mental health programs such as substance use disorder programs or PTSD clinical teams? Watch that video to find out.
+How does modeling to learn benefit Behavioral Health Integration Program or BHIP teams? What are the challenges with providing excellent team care? One challenge is the data sources to support team decision making jump from the individual patient and clinician counter all the way to the entire clinic or facility. It can mean that teams are flying blind to how they’re coordinating care for their Veterans. This can be especially hard when teams are short staffed or when staff, patient needs, and local policies continually change. It’s hard to know what improvements you might be able to find in your own team for coordinating evidence-based psychotherapies and pharmacotherapies over time. One way that Modeling to Learn can complement other BHIP resources is through the Data User Interface available at mtl.how/data. The Clinic Selection and Team Flow Selection tabs empower teams to carefully review the clinics that comprise their data views. Data are never all good or all bad. Modeling to Learn emphasizes transparency. Teams need to know what data they’re getting and its strengths and weaknesses for guiding a particular decision. The Clinic Selection tabs update every day to reflect all the Mental Health 500 series Stop Code, Clinic, or Grid updates that may be going on locally. Teams can save bookmarks that reflect their specific clinic selections, for example to include trainees or to select any of the clinics they have used within the last two years. This can be helpful when new BHIP teams are being formed or remapped. The team Patient Data tabs update daily too, so teams can view the patient diagnosis, encounter, health factor, measurement-based care, and high-risk flag information all in one place, and then the team can zoom out to filter and view these data as team trends. Coordinating care within a team is challenging. No single clinician can offer all the services a Veteran may need, so treatment plans must be coordinated in real time, over time, to support Veteran improvement. Clinicians may not have visibility on exactly how appointment supplies divvied up into intake evaluations, individual and group psychotherapy, medication management, or other adjunctive supportive therapies. What is particularly important about the Modeling to Learn simulation user interface at mtl.how/sim is the ability to assess not only the appointment supply, but even more important, the service proportions of patients that flow through to these services after intake or treatment plan review. One of the most common scenarios teams assess is how to respond to patient needs when they may not have the perfect staffing mix. Even when you know patient flow through care is divvied up into a mix of services, the dynamics over time are too much to understand in your head. The Modeling to Learn simulation user interface enables teams to quickly assess the dynamics of available appointment supply, service proportions, and the other two major factors that govern patient improvement, the return-to-clinic visit interval in weeks and the overall duration in care. We’ve often used Modeling to Learn to help teams find a way to ensure evidence-based care through the decisions that they make all day and find improvements in quality of care and quality of work life that they didn’t think was possible due to limited staffing. You might wonder How does Modeling to Learn support stepped care up to BHIP from primary care or primary care mental health integration, or from general mental health up or down into specialty mental health programs such as substance use disorder programs or PTSD clinical teams? Watch that video to find out.
Hi, I’m Lindsey and this is Debbie. How does Modeling to Learn support stepped care? We developed the measurement-based stepped care module of Modeling to Learn to help teams assess episodes of care when stepping patients up or down across the continuum of care. General mental health or behavioral health integration program, or BHIP teams, and specialty mental health teams such as substance use disorder, or SUD, programs and PTSD clinical teams, or PCTs, use the team flow selection tab in the Modeling to Learn Data User Interface to define flow from their care up or down to another team. Teams make these team flow selections at mtl.how/data. Based on years of testing the Data User Interface, we found that teams are most accurate in defining who they refer Veterans to after completing an episode of care within their team, rather than where they receive referrals from. And the teams also define the gap between visits that they believe is locally most reasonable for defining a new episode of care in each setting. The primary evaluations in the Team Flow module of the Modeling to Learn Simulation User Interface include assessing the flow through care of Veterans with high-risk flags as well as the proportions of high- and low-symptom patients as they flow through care to recovery and step down or discharge. We know that higher care quality improves recovery among Veterans, but how does that vary as a function of the total patients a team is serving? Team Flow can be used to assess the total manageable patients in a team and the impact of the patient load on care quality. Teams can also account for use of community care, which reduces some of the patients actively managed in the team. But as teams know, it does not reduce this care management to zero as Veterans using community care may still trigger an emergency care response or coordination of other services within a team. Perhaps most helpful is the view at a glance of where all Veterans are accumulating, waiting to step up or down across the continuum of care. This can help leadership and teams understand where there may need to be improved referral flows or service agreements between these settings. Teams can evaluate the flows through care when implementing measurement-based care to reduce the time it takes to detect when Veterans may be getting better or worse, which can also help improve agreements or decisions about when transitions across service settings are warranted. Would you like to know more about using measurement-based care to improve the time to detect patient improvement or risk? Watch that video to find out.
+How does Modeling to Learn support stepped care? We developed the measurement-based stepped care module of Modeling to Learn to help teams assess episodes of care when stepping patients up or down across the continuum of care. General mental health or behavioral health integration program, or BHIP teams, and specialty mental health teams such as substance use disorder, or SUD, programs and PTSD clinical teams, or PCTs, use the team flow selection tab in the Modeling to Learn Data User Interface to define flow from their care up or down to another team. Teams make these team flow selections at mtl.how/data. Based on years of testing the Data User Interface, we found that teams are most accurate in defining who they refer Veterans to after completing an episode of care within their team, rather than where they receive referrals from. And the teams also define the gap between visits that they believe is locally most reasonable for defining a new episode of care in each setting. The primary evaluations in the Team Flow module of the Modeling to Learn Simulation User Interface include assessing the flow through care of Veterans with high-risk flags as well as the proportions of high- and low-symptom patients as they flow through care to recovery and step down or discharge. We know that higher care quality improves recovery among Veterans, but how does that vary as a function of the total patients a team is serving? Team Flow can be used to assess the total manageable patients in a team and the impact of the patient load on care quality. Teams can also account for use of community care, which reduces some of the patients actively managed in the team. But as teams know, it does not reduce this care management to zero as Veterans using community care may still trigger an emergency care response or coordination of other services within a team. Perhaps most helpful is the view at a glance of where all Veterans are accumulating, waiting to step up or down across the continuum of care. This can help leadership and teams understand where there may need to be improved referral flows or service agreements between these settings. Teams can evaluate the flows through care when implementing measurement-based care to reduce the time it takes to detect when Veterans may be getting better or worse, which can also help improve agreements or decisions about when transitions across service settings are warranted. Would you like to know more about using measurement-based care to improve the time to detect patient improvement or risk? Watch that video to find out.
Hi, I’m Lindsey and this is David. Why use measurement-based care to improve the time to detect patient improvement or risk? To improve care quality in the flow of patients through care to recovery, the major value of measurement-based care is ensuring Veterans are getting appropriate care and are responding to the care they receive. Clinicians are likely familiar with the emphasis on measurement-based care, but from the perspective of care flow, what is powerful is reducing the time it takes for a team to detect whether a patient is getting better or worse. In the Modeling to Learn Data User Interface at mtl.how/data, the dataMeas tab provides tables of all the patient measures recorded at the VA Corporate Data Warehouse, whether they were captured via an evidence-based psychotherapy template, mental health assistant, or other approach. These include measures such as the GAD-7 for anxiety, PHQ-2 or PHQ-9 for depression, PCL-5 for PTSD, and the Brief Addiction Monitor or BAM. As a clinician, I like to use the patient-level views to sort scores from high to low to identify patients who may need to step up to a higher level of care or might benefit from referral to a specialty substance use disorder or PTSD service. The lowest score may indicate Veterans who can graduate from outpatient mental health care and step back down to primary care, which is an amazing day for the Veteran, their loved ones, and their treatment team. The vizMeas tab helps teams see trends in these measures to confirm when their efforts to implement measurement-based care are paying off. Teams can toggle to see what proportion of services include measurement-based care by comparing the vizMeas and vizEnc views. The Modeling to Learn Simulation User Interface at mtl.how/sim also empowers teams to assess the impacts of improved implementation of measurement-based care and how it can drive improvements in care quality. Do you want to learn more about the key drivers of higher care quality to improve recovery and prevent suicide? Watch that video to find out.
+Why use measurement-based care to improve the time to detect patient improvement or risk? To improve care quality in the flow of patients through care to recovery, the major value of measurement-based care is ensuring Veterans are getting appropriate care and are responding to the care they receive. Clinicians are likely familiar with the emphasis on measurement-based care, but from the perspective of care flow, what is powerful is reducing the time it takes for a team to detect whether a patient is getting better or worse. In the Modeling to Learn Data User Interface at mtl.how/data, the dataMeas tab provides tables of all the patient measures recorded at the VA Corporate Data Warehouse, whether they were captured via an evidence-based psychotherapy template, mental health assistant, or other approach. These include measures such as the GAD-7 for anxiety, PHQ-2 or PHQ-9 for depression, PCL-5 for PTSD, and the Brief Addiction Monitor or BAM. As a clinician, I like to use the patient-level views to sort scores from high to low to identify patients who may need to step up to a higher level of care or might benefit from referral to a specialty substance use disorder or PTSD service. The lowest score may indicate Veterans who can graduate from outpatient mental health care and step back down to primary care, which is an amazing day for the Veteran, their loved ones, and their treatment team. The vizMeas tab helps teams see trends in these measures to confirm when their efforts to implement measurement-based care are paying off. Teams can toggle to see what proportion of services include measurement-based care by comparing the vizMeas and vizEnc views. The Modeling to Learn Simulation User Interface at mtl.how/sim also empowers teams to assess the impacts of improved implementation of measurement-based care and how it can drive improvements in care quality. Do you want to learn more about the key drivers of higher care quality to improve recovery and prevent suicide? Watch that video to find out.
Hi, I’m Lindsey and this is David. If you’re watching this video, you want to know about strategic analytics improvement in learning, or SAIL metrics, with Modeling to Learn. Modeling to Learn was developed to complement existing VA priorities. One important benchmark is SAIL. Some of the more frustrating aspects of SAIL are the lag in data to support daily decision-making in the clinic and emphasis on high-level facility views and, of course, how challenging it can be to improve sale metrics over time, especially when you’re short-staffed. In fact, when SAIL metrics change over time, either in a positive or negative direction, it is difficult to know what’s driving the change. Facility leadership and clinical teams may not know how to interpret these changes or what improvement strategy to try next. Modeling to Learn supports two primary dimensions of the SAIL mental health quality domain—care coverage and care continuity. But importantly, Modeling to Learn recognizes that these two dimensions may work against each other in a dynamic tension over time. If you’re expanding population coverage measures, such as PSY 32 for depression, PSY 38, or PTSD 56 for PTSD, or SUD 16 for opioid use disorder, then it may be more challenging to ensure continuity of care measures such as MDD43h, MDD47h, and PSY 33 for depression and PSY 39 for PTSD. Of course, the reverse is also true. If you’re doing great with care continuity on those measures, then it may be challenging to expand population coverage at the same time. I’m fond of calling this the physics of our care quality problems. What I mean is that we cannot change the laws of physics, such as the law of conservation, which states that matter and energy in a physical process cannot be created or destroyed, but can only be transformed. In Modeling to Learn, staff time is neither created nor destroyed. This means that when data is exported from the MTL Data User Interface and uploaded to the MTL Simulation User Interface, the scenarios a team or VA explores to find improvements in care quality will always account for or conserve the total local staff time available. This helps to avoid fixes that fail, such as something that improves population coverage in the short term but reduces care continuity in the long term. Have you ever felt like you were playing Whac-A-Mole® with these metrics, where you improve one indicator only to have a problem later somewhere else? Modeling to Learn can help. Let’s start with why Modeling to Learn Red is useful. And why does the Modeling to Learn Data User Interface provide new insights? Watch that video to find out.
+If you’re watching this video, you want to know about strategic analytics improvement in learning, or SAIL metrics, with Modeling to Learn. Modeling to Learn was developed to complement existing VA priorities. One important benchmark is SAIL. Some of the more frustrating aspects of SAIL are the lag in data to support daily decision-making in the clinic and emphasis on high-level facility views and, of course, how challenging it can be to improve sale metrics over time, especially when you’re short-staffed. In fact, when SAIL metrics change over time, either in a positive or negative direction, it is difficult to know what’s driving the change. Facility leadership and clinical teams may not know how to interpret these changes or what improvement strategy to try next. Modeling to Learn supports two primary dimensions of the SAIL mental health quality domain—care coverage and care continuity. But importantly, Modeling to Learn recognizes that these two dimensions may work against each other in a dynamic tension over time. If you’re expanding population coverage measures, such as PSY 32 for depression, PSY 38, or PTSD 56 for PTSD, or SUD 16 for opioid use disorder, then it may be more challenging to ensure continuity of care measures such as MDD43h, MDD47h, and PSY 33 for depression and PSY 39 for PTSD. Of course, the reverse is also true. If you’re doing great with care continuity on those measures, then it may be challenging to expand population coverage at the same time. I’m fond of calling this the physics of our care quality problems. What I mean is that we cannot change the laws of physics, such as the law of conservation, which states that matter and energy in a physical process cannot be created or destroyed, but can only be transformed. In Modeling to Learn, staff time is neither created nor destroyed. This means that when data is exported from the MTL Data User Interface and uploaded to the MTL Simulation User Interface, the scenarios a team or VA explores to find improvements in care quality will always account for or conserve the total local staff time available. This helps to avoid fixes that fail, such as something that improves population coverage in the short term but reduces care continuity in the long term. Have you ever felt like you were playing Whac-A-Mole® with these metrics, where you improve one indicator only to have a problem later somewhere else? Modeling to Learn can help. Let’s start with why Modeling to Learn Red is useful. And why does the Modeling to Learn Data User Interface provide new insights? Watch that video to find out.
Hi, I’m Lindsey and this is David. How do five key variables drive care quality? Modeling to Learn emphasizes the dynamics of care over time, which can be accurately simplified to the key time-based variables that drive care quality. But be careful, the important principle is that these variables operate together over time to define an episode of care. That means care quality cannot be improved without understanding how these variables influence one another. If you want to see what Lindsey’s talking about, navigate to the Modeling to Learn Data User Interface at mtl.how/data and review each care problem—care coordination, medication management, psychotherapy, team care, and team flow. Our partners across VA describe how challenging it is to review data in one information system and then in another and end up unsure how to reconcile them, especially when you think that they indicate a different course of action. Folks are extremely busy and any new data resources must really add value to be worth learning. Let’s see if these Modeling to Learn variables meet the commonsense test of value added. Well, what do you think the five time-based variables are that make up an evidence-based episode of care? All outpatient care is defined by whether you can get an appointment when you need help. We focus on this all the time in VA. But then, and this is critical, you must be able to be seen again to complete a therapeutic course of care adequate to meet your need. Clinicians told us that a week was the way they think clinically. So, in Modeling to Learn, teams make their clinic selections to obtain an estimate of their local new patient start rate in patients per week and their appointment supply in appointments per week. Then we define evidence-based engagement as the new patient wait time in weeks, time between visits and weeks, and the engagement duration over time, again in weeks. This is a simplified definition of an evidence-based episode of care that is accurate for time. Why is it wise to focus on the dynamics of care over time? That’s why Modeling to Learn Blue is useful. So watch that video to find out.
+How do five key variables drive care quality? Modeling to Learn emphasizes the dynamics of care over time, which can be accurately simplified to the key time-based variables that drive care quality. But be careful, the important principle is that these variables operate together over time to define an episode of care. That means care quality cannot be improved without understanding how these variables influence one another. If you want to see what Lindsey’s talking about, navigate to the Modeling to Learn Data User Interface at mtl.how/data and review each care problem—care coordination, medication management, psychotherapy, team care, and team flow. Our partners across VA describe how challenging it is to review data in one information system and then in another and end up unsure how to reconcile them, especially when you think that they indicate a different course of action. Folks are extremely busy and any new data resources must really add value to be worth learning. Let’s see if these Modeling to Learn variables meet the commonsense test of value added. Well, what do you think the five time-based variables are that make up an evidence-based episode of care? All outpatient care is defined by whether you can get an appointment when you need help. We focus on this all the time in VA. But then, and this is critical, you must be able to be seen again to complete a therapeutic course of care adequate to meet your need. Clinicians told us that a week was the way they think clinically. So, in Modeling to Learn, teams make their clinic selections to obtain an estimate of their local new patient start rate in patients per week and their appointment supply in appointments per week. Then we define evidence-based engagement as the new patient wait time in weeks, time between visits and weeks, and the engagement duration over time, again in weeks. This is a simplified definition of an evidence-based episode of care that is accurate for time. Why is it wise to focus on the dynamics of care over time? That’s why Modeling to Learn Blue is useful. So watch that video to find out.
Hi, I’m Lindsey and this is Debbie. How does Modeling to Learn improve medication management? Medication management requires identifying a patient need, starting a medication, and evaluating for a therapeutic response over time. Some of the challenges and effective medication management over time include the realities that not all clinicians can prescribe. And of course, it requires follow up care to ensure that a patient is benefiting from their medication. So, when should the patient be seen again? One thing that can be so frustrating to prescribers is that a scheduler tells them when a patient can be seen again, rather than them telling the scheduler when the patient should be seen based on top of licensed care. The frustration is that medication management return-to-clinic visits may not occur at the appropriate interval to balance patient needs and risks. The VA has guidelines for some patient cohorts such as SAIL, MDD43h, and MDD47h, which define high quality continuity of follow up care as a valuation once every 12 weeks for Veterans diagnosed with depression who are using antidepressants. Other needs may reflect a different follow-up interval. For example, many addiction psychiatrists have described the appropriate medication management return-to-clinic visit interval for buprenorphine or methadone for opioid use disorder as every four weeks. These differences in medication needs, and the evidence-based pharmacotherapy standards that go with them, make it hard for prescribers to evaluate at a patient population level in a real time, especially when you’re completing another medication management encounter every 20 minutes. And of course, depending on the community at any given time, the staffing mix may change and the size or volume of patient needs can change too. Some SUD teams may have a high volume of patients receiving evidence-based pharmacotherapy for alcohol use disorder, whereas the modal presenting concern in a BHIP may be more likely to be depression. And the decisions the prescribers are making using current rules of thumb may be boxing them in to a pattern of care that makes it even harder to be responsive to crises or reduce future wait times. The Medication Management module of Modeling to Learn helps a team evaluate the trade-offs among all these factors as compared to the status quo over the last two years in the clinic. Comparing to the base case of no new decisions, the team can evaluate changes in their X-waivered MOUD appointment supply or evaluate and tailor the return-to-clinic visit interval by patient cohort. The goal is to help prescribers see what daily medication management decisions are adding up to over time and find feasible and effective local rules of thumb for clinically appropriate return-to-clinic orders in weeks that reduce wait times and ensure Veterans are getting better. For coordinating medication management in a multidisciplinary team such as a BHIP or SUD program, the Team Care module of Modeling to Learn can also help. If you want to know how Modeling to Learn helps to improve psychotherapy, watch that video to find out.
+How does Modeling to Learn improve medication management? Medication management requires identifying a patient need, starting a medication, and evaluating for a therapeutic response over time. Some of the challenges and effective medication management over time include the realities that not all clinicians can prescribe. And of course, it requires follow up care to ensure that a patient is benefiting from their medication. So, when should the patient be seen again? One thing that can be so frustrating to prescribers is that a scheduler tells them when a patient can be seen again, rather than them telling the scheduler when the patient should be seen based on top of licensed care. The frustration is that medication management return-to-clinic visits may not occur at the appropriate interval to balance patient needs and risks. The VA has guidelines for some patient cohorts such as SAIL, MDD43h, and MDD47h, which define high quality continuity of follow up care as a valuation once every 12 weeks for Veterans diagnosed with depression who are using antidepressants. Other needs may reflect a different follow-up interval. For example, many addiction psychiatrists have described the appropriate medication management return-to-clinic visit interval for buprenorphine or methadone for opioid use disorder as every four weeks. These differences in medication needs, and the evidence-based pharmacotherapy standards that go with them, make it hard for prescribers to evaluate at a patient population level in a real time, especially when you’re completing another medication management encounter every 20 minutes. And of course, depending on the community at any given time, the staffing mix may change and the size or volume of patient needs can change too. Some SUD teams may have a high volume of patients receiving evidence-based pharmacotherapy for alcohol use disorder, whereas the modal presenting concern in a BHIP may be more likely to be depression. And the decisions the prescribers are making using current rules of thumb may be boxing them in to a pattern of care that makes it even harder to be responsive to crises or reduce future wait times. The Medication Management module of Modeling to Learn helps a team evaluate the trade-offs among all these factors as compared to the status quo over the last two years in the clinic. Comparing to the base case of no new decisions, the team can evaluate changes in their X-waivered MOUD appointment supply or evaluate and tailor the return-to-clinic visit interval by patient cohort. The goal is to help prescribers see what daily medication management decisions are adding up to over time and find feasible and effective local rules of thumb for clinically appropriate return-to-clinic orders in weeks that reduce wait times and ensure Veterans are getting better. For coordinating medication management in a multidisciplinary team such as a BHIP or SUD program, the Team Care module of Modeling to Learn can also help. If you want to know how Modeling to Learn helps to improve psychotherapy, watch that video to find out.
Hi, I’m Lindsey and this is David. How does Modeling to Learn help improve psychotherapy? VA led the nation in dissemination and training for evidence-based psychotherapy. Many clinicians are ready to offer Veterans the highest quality care available. So, what makes it hard to get Veterans through an evidence-based episode of care when they start? What I’ve learned from working with clinical psychologists like Lindsey and other VA therapists from around the country is that the dynamics of psychotherapy in the clinic are challenging for the same reasons most care problems persist over time. Clinical teams typically do not have good visibility of where all their Veterans are in care. You may know that you plan to do CPT session six today with an individual Veteran, but how many Veterans are currently in the middle of therapy now? How many appointments are on the books? How many weeks are there before they return to clinic for follow-up care and how many graduate when they complete a course of an evidence-based psychotherapy? And how many drop out of therapy after one session or early in care but return to the team later? Evidence-based psychotherapy is typically defined by getting near weekly psychotherapy and having clinically meaningful improvement, which usually starts around session 8 to 12. Many VA data systems provide useful counts and proportions of evidence-based psychotherapy templates and the SAIL continuity of care metrics to find quality based on the number of sessions Veterans get within a specific time period. But the SAIL metric itself is a quarterly snapshot, and we know Veterans don’t all start therapy at the beginning of a new quarter. In the Modeling to Learn Data User Interface, a one-year patient cohort is identified that includes all Veterans seen from 18 months to six months ago based on the clinic selections made with the Data User Interface at mtl.how/data. Then, for the Veterans seen within those 12 months, we look back to see when those individual Veterans began therapy and how they were engaged over time. Once this psychotherapy cohort is defined and uploaded to the Simulation User Interface at mtl.how/sim, we can look at the stocks and see where all the Veterans have accumulated, where they are in care in a typical week, and how many patients per week are progressing through key therapy milestones. Finally, at a local team level, psychotherapists can see how they are doing for most of the Veterans they serve. And even better, they can run experiments to see what the impact will be of making new decisions over time. You might wonder What if we keep making the same care decisions? Will things get better or worse? Watch that video to find out.
+How does Modeling to Learn help improve psychotherapy? VA led the nation in dissemination and training for evidence-based psychotherapy. Many clinicians are ready to offer Veterans the highest quality care available. So, what makes it hard to get Veterans through an evidence-based episode of care when they start? What I’ve learned from working with clinical psychologists like Lindsey and other VA therapists from around the country is that the dynamics of psychotherapy in the clinic are challenging for the same reasons most care problems persist over time. Clinical teams typically do not have good visibility of where all their Veterans are in care. You may know that you plan to do CPT session six today with an individual Veteran, but how many Veterans are currently in the middle of therapy now? How many appointments are on the books? How many weeks are there before they return to clinic for follow-up care and how many graduate when they complete a course of an evidence-based psychotherapy? And how many drop out of therapy after one session or early in care but return to the team later? Evidence-based psychotherapy is typically defined by getting near weekly psychotherapy and having clinically meaningful improvement, which usually starts around session 8 to 12. Many VA data systems provide useful counts and proportions of evidence-based psychotherapy templates and the SAIL continuity of care metrics to find quality based on the number of sessions Veterans get within a specific time period. But the SAIL metric itself is a quarterly snapshot, and we know Veterans don’t all start therapy at the beginning of a new quarter. In the Modeling to Learn Data User Interface, a one-year patient cohort is identified that includes all Veterans seen from 18 months to six months ago based on the clinic selections made with the Data User Interface at mtl.how/data. Then, for the Veterans seen within those 12 months, we look back to see when those individual Veterans began therapy and how they were engaged over time. Once this psychotherapy cohort is defined and uploaded to the Simulation User Interface at mtl.how/sim, we can look at the stocks and see where all the Veterans have accumulated, where they are in care in a typical week, and how many patients per week are progressing through key therapy milestones. Finally, at a local team level, psychotherapists can see how they are doing for most of the Veterans they serve. And even better, they can run experiments to see what the impact will be of making new decisions over time. You might wonder What if we keep making the same care decisions? Will things get better or worse? Watch that video to find out.
Hi, I’m Lindsey and this is David. How does an appointment backlog extend the weeks between visits? At first, this sounds like a simple system story that many clinicians describe as we work together to co-develop Modeling to Learn. But they also told us when an appointment backlog extends the week between visits, a long interval between visits effects care too. That’s right. As the number of appointments for a specific service grows on the books, then the clinic has no choice but to increase the number of weeks or push out the time between visits to get everyone seen. We call this the return-to-clinic visit interval. This quantifies the return-to-clinic order in a unit of time—one week—which is critical for finding realistic local improvements in our partnerships with sites. Okay, so the balancing feedback principle operating here is that if you have a clinic calendar filled up with appointments, it’s harder for new patients to start care and it becomes harder for all existing patients to be seen with appropriate therapeutic follow-up standards for psychotherapy and pharmacotherapy. Lindsey, you and Debbie also talked about balancing feedback in the video, What if we keep making the same care decisions, will things get better or worse? For an explanation of how start delays affect treatment decisions over time, watch that video. As I said at the beginning of this video, it’s not just that a growing appointment backlog forces teams and clinics to extend the number of weeks between visits to get Veterans seen. It’s that long return-to-clinic visit intervals impact treatment decisions and care quality. None of these effects occur in isolation. All these components of care interact over time. It’s why system scientists will often point out there are no side effects and systems, there are only effects. Some we want and some we don’t. As clinicians identify Veterans who might benefit from an evidence-based course of psychotherapy for depression or PTSD, they may find that many patients are waiting to start, or they know that those 90833 or 90834 psychotherapy encounters are occurring 12 weeks apart—or both. Clinicians recognize that they cannot see patients in a way that is consistent with an evidence-based episode of care and will have to start finding alternatives. Modeling to Learn can help. What a relief that hundreds of clinicians, managers, data experts, quality improvement leads, and patients have already worked with us over the last decade to develop scalable Modeling to Learn resources for clinical and improvement teams to use today. The Modeling to Learn Simulation User Interface accounts for all these feedback effects so the team can upload their data, review what they see, and get immediate feedback about what the most feasible alternatives are to ensure quality of care for patients and quality of work life for providers and local staff. Based on this discussion, you may wonder How can we better balance the needs of new and existing patients? Should we prioritize new patient start rate or weeks between visits for quality? Watch that video to find out.
+How does an appointment backlog extend the weeks between visits? At first, this sounds like a simple system story that many clinicians describe as we work together to co-develop Modeling to Learn. But they also told us when an appointment backlog extends the week between visits, a long interval between visits effects care too. That’s right. As the number of appointments for a specific service grows on the books, then the clinic has no choice but to increase the number of weeks or push out the time between visits to get everyone seen. We call this the return-to-clinic visit interval. This quantifies the return-to-clinic order in a unit of time—one week—which is critical for finding realistic local improvements in our partnerships with sites. Okay, so the balancing feedback principle operating here is that if you have a clinic calendar filled up with appointments, it’s harder for new patients to start care and it becomes harder for all existing patients to be seen with appropriate therapeutic follow-up standards for psychotherapy and pharmacotherapy. Lindsey, you and Debbie also talked about balancing feedback in the video, What if we keep making the same care decisions, will things get better or worse? For an explanation of how start delays affect treatment decisions over time, watch that video. As I said at the beginning of this video, it’s not just that a growing appointment backlog forces teams and clinics to extend the number of weeks between visits to get Veterans seen. It’s that long return-to-clinic visit intervals impact treatment decisions and care quality. None of these effects occur in isolation. All these components of care interact over time. It’s why system scientists will often point out there are no side effects and systems, there are only effects. Some we want and some we don’t. As clinicians identify Veterans who might benefit from an evidence-based course of psychotherapy for depression or PTSD, they may find that many patients are waiting to start, or they know that those 90833 or 90834 psychotherapy encounters are occurring 12 weeks apart—or both. Clinicians recognize that they cannot see patients in a way that is consistent with an evidence-based episode of care and will have to start finding alternatives. Modeling to Learn can help. What a relief that hundreds of clinicians, managers, data experts, quality improvement leads, and patients have already worked with us over the last decade to develop scalable Modeling to Learn resources for clinical and improvement teams to use today. The Modeling to Learn Simulation User Interface accounts for all these feedback effects so the team can upload their data, review what they see, and get immediate feedback about what the most feasible alternatives are to ensure quality of care for patients and quality of work life for providers and local staff. Based on this discussion, you may wonder How can we better balance the needs of new and existing patients? Should we prioritize new patient start rate or weeks between visits for quality? Watch that video to find out.
Hi, I’m Lindsey and this is Debbie. How can we better balance the needs of new and existing patients? Should we prioritize new patient start rate or weeks between visits? These trade-offs are challenging for clinicians when you know there is a whole community of patients who need help. When teams are struggling with the limits of their available time in the day to see patients, it can feel like a clinical, ethical, and even moral quandary about how to best balance patients’ needs for services with the staff resources available. Of course, staff resources are always changing. We know many teams that are excelling in providing the highest quality addiction and mental health care available. But the behavioral health workforce shortage in the US is much bigger than VA. And VAs and teams need the right tools to make sure Veterans get the care they need for recovery. Challenges balancing the new patient start rate and the weeks between visits for existing patients apply systems thinking insights that we’ve talked about in other MTL videos, such as the physics of conserving staff time in order to ensure a clinically beneficial, realistic approach to care decisions and quality improvement that meets VA quality standards and meets Veterans needs for evidence-based episodes of care. The Modeling to Learn Blue Simulation User Interface available at mtl.how/sim enables a site or team to evaluate these two balancing system stories as a function of their data for the last two years exported from Modeling to Learn Red Data User Interface at mtl.how/data. As we talked about in the How does an appointment backlog extend the weeks between visits? video and What if we keep making the same care decisions, will things get better or worse? video, balancing feedbacks occur in any system that has a goal, including our healthcare and clinical systems. Balancing feedbacks occur in systems that have constraints of resources, staff, and time. And as a result, the trends that occur over time tend to reset to a status quo or oscillate around the status quo. A brief Modeling to Learn consult uses the site or team reviewed local data plus simulation to efficiently find local improvements that account for all these balancing trade-offs. We aim to help clinical and improvement teams find a couple empowering clinical heuristics, or rules of thumb, that are more effective for ensuring evidence-based care to get more Veterans better. When we partner with a VA or a team, we aim to find the lightest lift we can, like increase groups by 10% or adjust the return-to-clinic order for three weeks for these presenting concerns. And we often find something small that has a big payoff for Veterans. Do you want to be empowered to leverage the feedback, flow, and volume of your local care system? Watch that video to find out.
+How can we better balance the needs of new and existing patients? Should we prioritize new patient start rate or weeks between visits? These trade-offs are challenging for clinicians when you know there is a whole community of patients who need help. When teams are struggling with the limits of their available time in the day to see patients, it can feel like a clinical, ethical, and even moral quandary about how to best balance patients’ needs for services with the staff resources available. Of course, staff resources are always changing. We know many teams that are excelling in providing the highest quality addiction and mental health care available. But the behavioral health workforce shortage in the US is much bigger than VA. And VAs and teams need the right tools to make sure Veterans get the care they need for recovery. Challenges balancing the new patient start rate and the weeks between visits for existing patients apply systems thinking insights that we’ve talked about in other MTL videos, such as the physics of conserving staff time in order to ensure a clinically beneficial, realistic approach to care decisions and quality improvement that meets VA quality standards and meets Veterans needs for evidence-based episodes of care. The Modeling to Learn Blue Simulation User Interface available at mtl.how/sim enables a site or team to evaluate these two balancing system stories as a function of their data for the last two years exported from Modeling to Learn Red Data User Interface at mtl.how/data. As we talked about in the How does an appointment backlog extend the weeks between visits? video and What if we keep making the same care decisions, will things get better or worse? video, balancing feedbacks occur in any system that has a goal, including our healthcare and clinical systems. Balancing feedbacks occur in systems that have constraints of resources, staff, and time. And as a result, the trends that occur over time tend to reset to a status quo or oscillate around the status quo. A brief Modeling to Learn consult uses the site or team reviewed local data plus simulation to efficiently find local improvements that account for all these balancing trade-offs. We aim to help clinical and improvement teams find a couple empowering clinical heuristics, or rules of thumb, that are more effective for ensuring evidence-based care to get more Veterans better. When we partner with a VA or a team, we aim to find the lightest lift we can, like increase groups by 10% or adjust the return-to-clinic order for three weeks for these presenting concerns. And we often find something small that has a big payoff for Veterans. Do you want to be empowered to leverage the feedback, flow, and volume of your local care system? Watch that video to find out.
Hi, I’m Lindsey and this is David. Do you want to be empowered to leverage the feedback, flow, and volume of your local care system? In any system, leverage describes the places where a small shift in one thing can produce big changes in everything. For a big benefit for Veterans in their care teams, the primary places to focus are where the greatest volume is, the fastest flower rates are, and of course to account for feedback dynamics over time. When we partner with the VA or team, we aim to find the lightest lift we can, like finding for one BHIP that increasing group psychotherapy by 10% eliminates wait times and gets five times the number of Veterans near weekly therapy. Or, adjusting the return-to-clinic order by three weeks for some presenting concerns can provide the same medication management supply as hiring another prescriber. We can often find small care decisions that when implemented frequently over time, have a big payoff for Veterans because they add to more than the sum of the parts in the interdependent system over time. The last thing we want to do is make big changes that are difficult for Veterans and clinical teams that produce limited benefit over time. In my experience over many years now, finding something clinically powerful that leverages the dynamics of the system can be a huge boost to morale, productivity, and staff empowerment. You might wonder How can you find powerful insights that you are sure will work locally? We do this by reviewing the Modeling to Learn Simulation User Interface stocks to see where all the Veterans are accumulating the most. We review the rates to see where the floodwaters are rising fastest and require action. Or where things are trending in the right direction and our attention could be better focused elsewhere. Modeling to Learn consultation offers clinical and improvement teams real-time efficient support. We can do this quickly because we’ve partnered with patients, providers, policy makers, data experts, evaluators, and quality improvement consultants from the frontline to VA Central Office for nearly 10 years now to define the primary drivers of care problems and care quality using system dynamics. Modeling to Learn consultation involves empowering clinicians, managers, improvers, data leads, and all of VA to leverage the system for getting more Veterans better. How can Modeling to Learn consultation help? Watch that video to find out.
+Do you want to be empowered to leverage the feedback, flow, and volume of your local care system? In any system, leverage describes the places where a small shift in one thing can produce big changes in everything. For a big benefit for Veterans in their care teams, the primary places to focus are where the greatest volume is, the fastest flower rates are, and of course to account for feedback dynamics over time. When we partner with the VA or team, we aim to find the lightest lift we can, like finding for one BHIP that increasing group psychotherapy by 10% eliminates wait times and gets five times the number of Veterans near weekly therapy. Or, adjusting the return-to-clinic order by three weeks for some presenting concerns can provide the same medication management supply as hiring another prescriber. We can often find small care decisions that when implemented frequently over time, have a big payoff for Veterans because they add to more than the sum of the parts in the interdependent system over time. The last thing we want to do is make big changes that are difficult for Veterans and clinical teams that produce limited benefit over time. In my experience over many years now, finding something clinically powerful that leverages the dynamics of the system can be a huge boost to morale, productivity, and staff empowerment. You might wonder How can you find powerful insights that you are sure will work locally? We do this by reviewing the Modeling to Learn Simulation User Interface stocks to see where all the Veterans are accumulating the most. We review the rates to see where the floodwaters are rising fastest and require action. Or where things are trending in the right direction and our attention could be better focused elsewhere. Modeling to Learn consultation offers clinical and improvement teams real-time efficient support. We can do this quickly because we’ve partnered with patients, providers, policy makers, data experts, evaluators, and quality improvement consultants from the frontline to VA Central Office for nearly 10 years now to define the primary drivers of care problems and care quality using system dynamics. Modeling to Learn consultation involves empowering clinicians, managers, improvers, data leads, and all of VA to leverage the system for getting more Veterans better. How can Modeling to Learn consultation help? Watch that video to find out.
Hi, I’m Lindsey and this is Debbie. What are key drivers of higher care quality to improve recovery and prevent suicide? A central idea of Modeling to Learn resources is that delivering consistent, timely, high-quality care is challenging for teams because we don’t accurately account for the primary dynamics driving care quality over time. You might be thinking Care dynamics, what does that mean? We literally mean that due to time pressures and cognitive limits, we don’t effectively understand relationships among key care decisions that we make every day over time, especially decisions made in the aggregate across a clinical team. For example, we may try to break down problems by looking at referrals or wait times, but we don’t look at how clinical teams’ knowledge of wait times influence referrals to a specific service or team. In the Modeling to Learn Data User Interface, we define care dynamics in terms of rates over time, like new patients per week, so that we can understand the flow of Veterans through care. As another example in terms of care quality, recovery and suicide prevention, the Modeling to Learn Team Flow module looks not only at the number of patients served by a team or site, or the number of patients with a high-risk flag for suicide, but also the time it takes to unflag high-risk patients in a team and the typical time to improve within the team over the last two years. The Modeling to Learn Simulation User Interface uses an interactive structure fed by local team data from the Data User Interface. The Modeling to Learn Simulation User Interface visually depicts accumulation of patients in desirable or undesirable states of care, like waiting to start a new service. These states are defined by their inflows and outflows. A lot of clinical teams and mental health leadership wonder whether they can get any new insights when they are already doing everything that they can to support the Veterans in their care. Would you like to know how to improve SAIL with Modeling to Learn? Watch that video to find out.
+What are key drivers of higher care quality to improve recovery and prevent suicide? A central idea of Modeling to Learn resources is that delivering consistent, timely, high-quality care is challenging for teams because we don’t accurately account for the primary dynamics driving care quality over time. You might be thinking Care dynamics, what does that mean? We literally mean that due to time pressures and cognitive limits, we don’t effectively understand relationships among key care decisions that we make every day over time, especially decisions made in the aggregate across a clinical team. For example, we may try to break down problems by looking at referrals or wait times, but we don’t look at how clinical teams’ knowledge of wait times influence referrals to a specific service or team. In the Modeling to Learn Data User Interface, we define care dynamics in terms of rates over time, like new patients per week, so that we can understand the flow of Veterans through care. As another example in terms of care quality, recovery and suicide prevention, the Modeling to Learn Team Flow module looks not only at the number of patients served by a team or site, or the number of patients with a high-risk flag for suicide, but also the time it takes to unflag high-risk patients in a team and the typical time to improve within the team over the last two years. The Modeling to Learn Simulation User Interface uses an interactive structure fed by local team data from the Data User Interface. The Modeling to Learn Simulation User Interface visually depicts accumulation of patients in desirable or undesirable states of care, like waiting to start a new service. These states are defined by their inflows and outflows. A lot of clinical teams and mental health leadership wonder whether they can get any new insights when they are already doing everything that they can to support the Veterans in their care. Would you like to know how to improve SAIL with Modeling to Learn? Watch that video to find out.
](https://bcove.video/4bIhtF6)
-I'm Lindsey, and this is Debbie. How can we get more Veterans better without adding hours in our day? To get Veterans better, we have to account for the number of hours available in the day. Yet because we don't have the right information about how we manage our time, we can pursue unrealistic treatment plans or improvement strategies that violate the laws of physics. _Modeling to Learn_ encourages us to account for clinician time differently, to make our plans feasible locally. Most VA data systems offer snapshots of cross-sectional data, but this kind of information doesn't help us account for the dynamics of evidence-based care over time. While many VA data systems are available, most display either quarterly lags from a high-level view or an in the weeds patient by patient view. _Modeling to Learn_ helps to bridge these two views. _Modeling to Learn_ introduces trends over time based on the clinic selections carefully reviewed by the staff on site. Trends help us zoom out to understand how we're doing for most of the Veterans we serve. _Modeling to Learn_ also introduces time-based definitions of care in the units of patients per week, appointments per week, or episodes of care per week. We use a week because over the last several years clinicians told us this is how they think in the clinic. There often is no typical day and a lot has happened by the end of a month. Defining care by week is more beneficial. High quality care in the clinic is always constrained by the available staffing capacity and the hours in the day to see patients, the number of weeks between visits, and the weeks of engagement over time. But do you have time to run these equations in your head or calculate them from the other dashboards you use? As an alternative, _Modeling to Learn_ puts these components of care together to understand their interdependence over time. How can _Modeling to Learn_ consultation help? Watch that video to find out.
+How can we get more Veterans better without adding hours in our day? To get Veterans better, we have to account for the number of hours available in the day. Yet because we don't have the right information about how we manage our time, we can pursue unrealistic treatment plans or improvement strategies that violate the laws of physics. _Modeling to Learn_ encourages us to account for clinician time differently, to make our plans feasible locally. Most VA data systems offer snapshots of cross-sectional data, but this kind of information doesn't help us account for the dynamics of evidence-based care over time. While many VA data systems are available, most display either quarterly lags from a high-level view or an in the weeds patient by patient view. _Modeling to Learn_ helps to bridge these two views. _Modeling to Learn_ introduces trends over time based on the clinic selections carefully reviewed by the staff on site. Trends help us zoom out to understand how we're doing for most of the Veterans we serve. _Modeling to Learn_ also introduces time-based definitions of care in the units of patients per week, appointments per week, or episodes of care per week. We use a week because over the last several years clinicians told us this is how they think in the clinic. There often is no typical day and a lot has happened by the end of a month. Defining care by week is more beneficial. High quality care in the clinic is always constrained by the available staffing capacity and the hours in the day to see patients, the number of weeks between visits, and the weeks of engagement over time. But do you have time to run these equations in your head or calculate them from the other dashboards you use? As an alternative, _Modeling to Learn_ puts these components of care together to understand their interdependence over time. How can _Modeling to Learn_ consultation help? Watch that video to find out.
## How does _Modeling to Learn_ benefit Substance Use Disorder or SUD programs?
@@ -16,7 +16,7 @@ I'm Lindsey, and this is Debbie. How can we get more Veterans better without add
[](https://bcove.video/46dSKHt)
-Hi, I'm Lindsey and this is David. How does _Modeling to Learn_ benefit substance use disorder, or SUD programs? When _Modeling to Learn_ began almost 10 years ago, as a partnership among patients, providers and policy makers across VA, we worked iteratively to define the dynamics of common care problems for the primary reasons Veterans seek addiction and mental health care. We found that if we focused on improving timely, high-quality evidence-based psychotherapy and pharmacotherapy for alcohol use disorder, depression, opioid use disorder and PTSD, we could support VA staff in meeting around 80% of Veterans needs for behavioral healthcare. We work with frontline teams and partners from the VA Academic Detailing Program and the Psychotropic Drug Safety Initiative or PDSI and developed the Medication Management Module of _Modeling to Learn_. This module helps manage the dynamics of evidence-based pharmacotherapies for alcohol use disorder and opioid use disorder based on their effectiveness in reducing craving, relapse, and overdose. You can find the Medication Management module in _Modeling to Learn_ data [mtl.how/data](https://app.powerbigov.us/groups/me/apps/b9686a29-6857-46c9-bdf9-043ca2b29138/reports/05dd8dbd-313f-4993-b406-6feea2fdb060/ReportSection?ctid=e95f1b23-abaf-45ee-821d-b7ab251ab3bf) and Simulation User Interface [mtl.how/sim](https://forio.com/app/va/va-psd-sim/login.html). The _Modeling to Learn_ Medication Management module supports improvement in the AlcTop or Alcohol Topiramate indicator from the Mental Health Information System or MIS and the SUD 16 indicator from the Strategic Analytics for Improvement in Learning or SAIL. How does _Modeling to Learn_ help? Prescribers describe the need to better visualize how to locally optimize their medication management appointment supply, including top of license care from staff with and without a Drug Enforcement Agency X-waiver for OUD medication. _Modeling to Learn_ helps a team assess this appointment supply, new patient start rate, no show or missed appointment rate, and the clinically appropriate return-to-clinic visit interval needed for a therapeutic response to alcohol use disorder and opioid use disorder medications. The Medication Management module helps teams and sites find the optimal number of patients who can be engaged in AUD and OUD therapies over time and helps them distinguish the flow of their patients who require antidepressants or other medication needs. Given that community needs and team staffing is dynamic and can change over time, these resources can empower teams to find local improvements that otherwise may be hard to find. Another common module for addressing Veterans SUD needs for group and other therapies is the Team Care module, which helps teams assess and optimize their overall mix of local multidisciplinary services as a function of staffing and patient needs. We are evaluating how _Modeling to Learn_ improves Veterans who start and complete a therapeutic course of medication for AUD and OUD with research funding from the VA and the National Institute on Drug Abuse or NIDA. What about other common SUD comorbidities and presenting concerns? For example, how does _Modeling to Learn_ benefit PTSD, clinical teams or PCTs? Watch that video to find out.
+How does _Modeling to Learn_ benefit substance use disorder, or SUD programs? When _Modeling to Learn_ began almost 10 years ago, as a partnership among patients, providers and policy makers across VA, we worked iteratively to define the dynamics of common care problems for the primary reasons Veterans seek addiction and mental health care. We found that if we focused on improving timely, high-quality evidence-based psychotherapy and pharmacotherapy for alcohol use disorder, depression, opioid use disorder and PTSD, we could support VA staff in meeting around 80% of Veterans needs for behavioral healthcare. We work with frontline teams and partners from the VA Academic Detailing Program and the Psychotropic Drug Safety Initiative or PDSI and developed the Medication Management Module of _Modeling to Learn_. This module helps manage the dynamics of evidence-based pharmacotherapies for alcohol use disorder and opioid use disorder based on their effectiveness in reducing craving, relapse, and overdose. You can find the Medication Management module in _Modeling to Learn_ data [mtl.how/data](https://app.powerbigov.us/groups/me/apps/b9686a29-6857-46c9-bdf9-043ca2b29138/reports/05dd8dbd-313f-4993-b406-6feea2fdb060/ReportSection?ctid=e95f1b23-abaf-45ee-821d-b7ab251ab3bf) and Simulation User Interface [mtl.how/sim](https://forio.com/app/va/va-psd-sim/login.html). The _Modeling to Learn_ Medication Management module supports improvement in the AlcTop or Alcohol Topiramate indicator from the Mental Health Information System or MIS and the SUD 16 indicator from the Strategic Analytics for Improvement in Learning or SAIL. How does _Modeling to Learn_ help? Prescribers describe the need to better visualize how to locally optimize their medication management appointment supply, including top of license care from staff with and without a Drug Enforcement Agency X-waiver for OUD medication. _Modeling to Learn_ helps a team assess this appointment supply, new patient start rate, no show or missed appointment rate, and the clinically appropriate return-to-clinic visit interval needed for a therapeutic response to alcohol use disorder and opioid use disorder medications. The Medication Management module helps teams and sites find the optimal number of patients who can be engaged in AUD and OUD therapies over time and helps them distinguish the flow of their patients who require antidepressants or other medication needs. Given that community needs and team staffing is dynamic and can change over time, these resources can empower teams to find local improvements that otherwise may be hard to find. Another common module for addressing Veterans SUD needs for group and other therapies is the Team Care module, which helps teams assess and optimize their overall mix of local multidisciplinary services as a function of staffing and patient needs. We are evaluating how _Modeling to Learn_ improves Veterans who start and complete a therapeutic course of medication for AUD and OUD with research funding from the VA and the National Institute on Drug Abuse or NIDA. What about other common SUD comorbidities and presenting concerns? For example, how does _Modeling to Learn_ benefit PTSD, clinical teams or PCTs? Watch that video to find out.
## How does _Modeling to Learn_ benefit PTSD Clinical Teams or PCTs?
@@ -24,7 +24,7 @@ Hi, I'm Lindsey and this is David. How does _Modeling to Learn_ benefit substanc
[](https://bcove.video/3zNR6An)
-Hi, I'm Lindsey and this is Debbie. How does _Modeling to Learn_ benefit PTSD clinical teams, or PCTs? Since I'm at the National Center for PTSD, this priority is very close to my heart. When _Modeling to Learn_ began almost 10 years ago as a partnership among patients, providers and policy makers across VA, we worked iteratively to define the dynamics of common care problems for the primary reasons Veterans seek addiction and mental health care. We found that if we focused on improving timely, high-quality evidence-based psychotherapy and pharmacotherapy for alcohol use disorder, depression, opioid use disorder and PTSD, we could support VA staff in meeting around 80% of Veterans needs for behavioral healthcare. _Modeling to Learn_ helps teams better evaluate the local dynamics of balancing the needs of new and existing patients for evidence-based psychotherapies, particularly cognitive processing therapy and prolonged exposure. Teams described the challenges of starting new patients who had never had their needs met without compromising care for Veterans already engaged in therapy. _Modeling to Learn_ enables teams to see where their Veterans get stuck, to decide when to graduate patients, or to recognize when the weeks between return to clinic visits start to get too long. Have you been trying to improve PTSD 56 in the Strategic Analytics for Improvement and Learning, or SAIL? We worked with local PTSD clinical teams from the beginning and have supported many more teams over the years who have engaged with the evidence-based psychotherapy programs or the PTSD mentorship program to meet these needs. The Psychotherapy module of the _Modeling to Learn_ data user interface, and simulation user interface helps teams to see how they're doing for patients who are starting psychotherapy, how many flow through to complete a therapeutic dose, and whether Veterans are getting near weekly therapy consistent with the PTSD treatment evidence base. There's also the ability to assess emerging approaches such as masked or intensive outpatient psychotherapy and other scenarios that are commonly considered and implemented in PCTs. You may also be wondering about the flow from primary care or from primary care mental health integration, or PCMHI, teams into general mental health and specialty mental health programs like PCTs. Stepped care up and down the mental health continuum of care can also be optimized to local needs using the _Modeling to Learn_ Team Flow module. Speaking of the care continuum, you may be wondering _How does _Modeling to Learn_ benefit Behavioral Health Integration program or BHIP teams?_ Watch that video to find out.
+How does _Modeling to Learn_ benefit PTSD clinical teams, or PCTs? Since I'm at the National Center for PTSD, this priority is very close to my heart. When _Modeling to Learn_ began almost 10 years ago as a partnership among patients, providers and policy makers across VA, we worked iteratively to define the dynamics of common care problems for the primary reasons Veterans seek addiction and mental health care. We found that if we focused on improving timely, high-quality evidence-based psychotherapy and pharmacotherapy for alcohol use disorder, depression, opioid use disorder and PTSD, we could support VA staff in meeting around 80% of Veterans needs for behavioral healthcare. _Modeling to Learn_ helps teams better evaluate the local dynamics of balancing the needs of new and existing patients for evidence-based psychotherapies, particularly cognitive processing therapy and prolonged exposure. Teams described the challenges of starting new patients who had never had their needs met without compromising care for Veterans already engaged in therapy. _Modeling to Learn_ enables teams to see where their Veterans get stuck, to decide when to graduate patients, or to recognize when the weeks between return to clinic visits start to get too long. Have you been trying to improve PTSD 56 in the Strategic Analytics for Improvement and Learning, or SAIL? We worked with local PTSD clinical teams from the beginning and have supported many more teams over the years who have engaged with the evidence-based psychotherapy programs or the PTSD mentorship program to meet these needs. The Psychotherapy module of the _Modeling to Learn_ data user interface, and simulation user interface helps teams to see how they're doing for patients who are starting psychotherapy, how many flow through to complete a therapeutic dose, and whether Veterans are getting near weekly therapy consistent with the PTSD treatment evidence base. There's also the ability to assess emerging approaches such as masked or intensive outpatient psychotherapy and other scenarios that are commonly considered and implemented in PCTs. You may also be wondering about the flow from primary care or from primary care mental health integration, or PCMHI, teams into general mental health and specialty mental health programs like PCTs. Stepped care up and down the mental health continuum of care can also be optimized to local needs using the _Modeling to Learn_ Team Flow module. Speaking of the care continuum, you may be wondering _How does _Modeling to Learn_ benefit Behavioral Health Integration program or BHIP teams?_ Watch that video to find out.
## Can _Modeling to Learn_ benefit Behavioral Health Integration Program or BHIP teams?
@@ -32,7 +32,7 @@ Hi, I'm Lindsey and this is Debbie. How does _Modeling to Learn_ benefit PTSD cl
[](https://bcove.video/4bQfDSH)
-Hi, I'm Lindsey and this is David. How does modeling to learn benefit Behavioral Health Integration Program or BHIP teams? What are the challenges with providing excellent team care? One challenge is the data sources to support team decision making jump from the individual patient and clinician counter all the way to the entire clinic or facility. It can mean that teams are flying blind to how they're coordinating care for their Veterans. This can be especially hard when teams are short staffed or when staff, patient needs, and local policies continually change. It's hard to know what improvements you might be able to find in your own team for coordinating evidence-based psychotherapies and pharmacotherapies over time. One way that _Modeling to Learn_ can complement other BHIP resources is through the Data User Interface available at [mtl.how/data](https://app.powerbigov.us/groups/me/apps/b9686a29-6857-46c9-bdf9-043ca2b29138/reports/05dd8dbd-313f-4993-b406-6feea2fdb060/ReportSection?ctid=e95f1b23-abaf-45ee-821d-b7ab251ab3bf). The Clinic Selection and Team Flow Selection tabs empower teams to carefully review the clinics that comprise their data views. Data are never all good or all bad. _Modeling to Learn_ emphasizes transparency. Teams need to know what data they're getting and its strengths and weaknesses for guiding a particular decision. The Clinic Selection tabs update every day to reflect all the Mental Health 500 series Stop Code, Clinic, or Grid updates that may be going on locally. Teams can save bookmarks that reflect their specific clinic selections, for example to include trainees or to select any of the clinics they have used within the last two years. This can be helpful when new BHIP teams are being formed or remapped. The team Patient Data tabs update daily too, so teams can view the patient diagnosis, encounter, health factor, measurement-based care, and high-risk flag information all in one place, and then the team can zoom out to filter and view these data as team trends. Coordinating care within a team is challenging. No single clinician can offer all the services a Veteran may need, so treatment plans must be coordinated in real time, over time, to support Veteran improvement. Clinicians may not have visibility on exactly how appointment supplies divvied up into intake evaluations, individual and group psychotherapy, medication management, or other adjunctive supportive therapies. What is particularly important about the _Modeling to Learn_ simulation user interface at [mtl.how/sim](https://forio.com/app/va/va-psd-sim/login.html) is the ability to assess not only the appointment supply, but even more important, the service proportions of patients that flow through to these services after intake or treatment plan review. One of the most common scenarios teams assess is how to respond to patient needs when they may not have the perfect staffing mix. Even when you know patient flow through care is divvied up into a mix of services, the dynamics over time are too much to understand in your head. The _Modeling to Learn_ simulation user interface enables teams to quickly assess the dynamics of available appointment supply, service proportions, and the other two major factors that govern patient improvement, the return-to-clinic visit interval in weeks and the overall duration in care. We've often used _Modeling to Learn_ to help teams find a way to ensure evidence-based care through the decisions that they make all day and find improvements in quality of care and quality of work life that they didn't think was possible due to limited staffing. You might wonder _How does _Modeling to Learn_ support stepped care up to BHIP from primary care or primary care mental health integration, or from general mental health up or down into specialty mental health programs such as substance use disorder programs or PTSD clinical teams?_ Watch that video to find out.
+How does modeling to learn benefit Behavioral Health Integration Program or BHIP teams? What are the challenges with providing excellent team care? One challenge is the data sources to support team decision making jump from the individual patient and clinician counter all the way to the entire clinic or facility. It can mean that teams are flying blind to how they're coordinating care for their Veterans. This can be especially hard when teams are short staffed or when staff, patient needs, and local policies continually change. It's hard to know what improvements you might be able to find in your own team for coordinating evidence-based psychotherapies and pharmacotherapies over time. One way that _Modeling to Learn_ can complement other BHIP resources is through the Data User Interface available at [mtl.how/data](https://app.powerbigov.us/groups/me/apps/b9686a29-6857-46c9-bdf9-043ca2b29138/reports/05dd8dbd-313f-4993-b406-6feea2fdb060/ReportSection?ctid=e95f1b23-abaf-45ee-821d-b7ab251ab3bf). The Clinic Selection and Team Flow Selection tabs empower teams to carefully review the clinics that comprise their data views. Data are never all good or all bad. _Modeling to Learn_ emphasizes transparency. Teams need to know what data they're getting and its strengths and weaknesses for guiding a particular decision. The Clinic Selection tabs update every day to reflect all the Mental Health 500 series Stop Code, Clinic, or Grid updates that may be going on locally. Teams can save bookmarks that reflect their specific clinic selections, for example to include trainees or to select any of the clinics they have used within the last two years. This can be helpful when new BHIP teams are being formed or remapped. The team Patient Data tabs update daily too, so teams can view the patient diagnosis, encounter, health factor, measurement-based care, and high-risk flag information all in one place, and then the team can zoom out to filter and view these data as team trends. Coordinating care within a team is challenging. No single clinician can offer all the services a Veteran may need, so treatment plans must be coordinated in real time, over time, to support Veteran improvement. Clinicians may not have visibility on exactly how appointment supplies divvied up into intake evaluations, individual and group psychotherapy, medication management, or other adjunctive supportive therapies. What is particularly important about the _Modeling to Learn_ simulation user interface at [mtl.how/sim](https://forio.com/app/va/va-psd-sim/login.html) is the ability to assess not only the appointment supply, but even more important, the service proportions of patients that flow through to these services after intake or treatment plan review. One of the most common scenarios teams assess is how to respond to patient needs when they may not have the perfect staffing mix. Even when you know patient flow through care is divvied up into a mix of services, the dynamics over time are too much to understand in your head. The _Modeling to Learn_ simulation user interface enables teams to quickly assess the dynamics of available appointment supply, service proportions, and the other two major factors that govern patient improvement, the return-to-clinic visit interval in weeks and the overall duration in care. We've often used _Modeling to Learn_ to help teams find a way to ensure evidence-based care through the decisions that they make all day and find improvements in quality of care and quality of work life that they didn't think was possible due to limited staffing. You might wonder _How does _Modeling to Learn_ support stepped care up to BHIP from primary care or primary care mental health integration, or from general mental health up or down into specialty mental health programs such as substance use disorder programs or PTSD clinical teams?_ Watch that video to find out.
## How does _Modeling to Learn_ support stepped care?
@@ -40,7 +40,7 @@ Hi, I'm Lindsey and this is David. How does modeling to learn benefit Behavioral
[](https://bcove.video/4d6F2Ja)
-Hi, I'm Lindsey and this is Debbie. How does _Modeling to Learn_ support stepped care? We developed the measurement-based stepped care module of _Modeling to Learn_ to help teams assess episodes of care when stepping patients up or down across the continuum of care. General mental health or behavioral health integration program, or BHIP teams, and specialty mental health teams such as substance use disorder, or SUD, programs and PTSD clinical teams, or PCTs, use the team flow selection tab in the _Modeling to Learn_ Data User Interface to define flow from their care up or down to another team. Teams make these team flow selections at [mtl.how/data](https://app.powerbigov.us/groups/me/apps/b9686a29-6857-46c9-bdf9-043ca2b29138/reports/05dd8dbd-313f-4993-b406-6feea2fdb060/ReportSection?ctid=e95f1b23-abaf-45ee-821d-b7ab251ab3bf). Based on years of testing the Data User Interface, we found that teams are most accurate in defining who they refer Veterans to after completing an episode of care within their team, rather than where they receive referrals from. And the teams also define the gap between visits that they believe is locally most reasonable for defining a new episode of care in each setting. The primary evaluations in the Team Flow module of the _Modeling to Learn_ Simulation User Interface include assessing the flow through care of Veterans with high-risk flags as well as the proportions of high- and low-symptom patients as they flow through care to recovery and step down or discharge. We know that higher care quality improves recovery among Veterans, but how does that vary as a function of the total patients a team is serving? Team Flow can be used to assess the total manageable patients in a team and the impact of the patient load on care quality. Teams can also account for use of community care, which reduces some of the patients actively managed in the team. But as teams know, it does not reduce this care management to zero as Veterans using community care may still trigger an emergency care response or coordination of other services within a team. Perhaps most helpful is the view at a glance of where all Veterans are accumulating, waiting to step up or down across the continuum of care. This can help leadership and teams understand where there may need to be improved referral flows or service agreements between these settings. Teams can evaluate the flows through care when implementing measurement-based care to reduce the time it takes to detect when Veterans may be getting better or worse, which can also help improve agreements or decisions about when transitions across service settings are warranted. Would you like to know more about using measurement-based care to improve the time to detect patient improvement or risk? Watch that video to find out.
+How does _Modeling to Learn_ support stepped care? We developed the measurement-based stepped care module of _Modeling to Learn_ to help teams assess episodes of care when stepping patients up or down across the continuum of care. General mental health or behavioral health integration program, or BHIP teams, and specialty mental health teams such as substance use disorder, or SUD, programs and PTSD clinical teams, or PCTs, use the team flow selection tab in the _Modeling to Learn_ Data User Interface to define flow from their care up or down to another team. Teams make these team flow selections at [mtl.how/data](https://app.powerbigov.us/groups/me/apps/b9686a29-6857-46c9-bdf9-043ca2b29138/reports/05dd8dbd-313f-4993-b406-6feea2fdb060/ReportSection?ctid=e95f1b23-abaf-45ee-821d-b7ab251ab3bf). Based on years of testing the Data User Interface, we found that teams are most accurate in defining who they refer Veterans to after completing an episode of care within their team, rather than where they receive referrals from. And the teams also define the gap between visits that they believe is locally most reasonable for defining a new episode of care in each setting. The primary evaluations in the Team Flow module of the _Modeling to Learn_ Simulation User Interface include assessing the flow through care of Veterans with high-risk flags as well as the proportions of high- and low-symptom patients as they flow through care to recovery and step down or discharge. We know that higher care quality improves recovery among Veterans, but how does that vary as a function of the total patients a team is serving? Team Flow can be used to assess the total manageable patients in a team and the impact of the patient load on care quality. Teams can also account for use of community care, which reduces some of the patients actively managed in the team. But as teams know, it does not reduce this care management to zero as Veterans using community care may still trigger an emergency care response or coordination of other services within a team. Perhaps most helpful is the view at a glance of where all Veterans are accumulating, waiting to step up or down across the continuum of care. This can help leadership and teams understand where there may need to be improved referral flows or service agreements between these settings. Teams can evaluate the flows through care when implementing measurement-based care to reduce the time it takes to detect when Veterans may be getting better or worse, which can also help improve agreements or decisions about when transitions across service settings are warranted. Would you like to know more about using measurement-based care to improve the time to detect patient improvement or risk? Watch that video to find out.
## How does measurement-based care detect patient improvement or risk?
@@ -48,7 +48,7 @@ Hi, I'm Lindsey and this is Debbie. How does _Modeling to Learn_ support stepped
[](https://bcove.video/3yl8FqY)
-Hi, I'm Lindsey and this is David. Why use measurement-based care to improve the time to detect patient improvement or risk? To improve care quality in the flow of patients through care to recovery, the major value of measurement-based care is ensuring Veterans are getting appropriate care and are responding to the care they receive. Clinicians are likely familiar with the emphasis on measurement-based care, but from the perspective of care flow, what is powerful is reducing the time it takes for a team to detect whether a patient is getting better or worse. In the _Modeling to Learn_ Data User Interface at [mtl.how/data](https://app.powerbigov.us/groups/me/apps/b9686a29-6857-46c9-bdf9-043ca2b29138/reports/05dd8dbd-313f-4993-b406-6feea2fdb060/ReportSection?ctid=e95f1b23-abaf-45ee-821d-b7ab251ab3bf), the dataMeas tab provides tables of all the patient measures recorded at the VA Corporate Data Warehouse, whether they were captured via an evidence-based psychotherapy template, mental health assistant, or other approach. These include measures such as the GAD-7 for anxiety, PHQ-2 or PHQ-9 for depression, PCL-5 for PTSD, and the Brief Addiction Monitor or BAM. As a clinician, I like to use the patient-level views to sort scores from high to low to identify patients who may need to step up to a higher level of care or might benefit from referral to a specialty substance use disorder or PTSD service. The lowest score may indicate Veterans who can graduate from outpatient mental health care and step back down to primary care, which is an amazing day for the Veteran, their loved ones, and their treatment team. The vizMeas tab helps teams see trends in these measures to confirm when their efforts to implement measurement-based care are paying off. Teams can toggle to see what proportion of services include measurement-based care by comparing the vizMeas and vizEnc views. The _Modeling to Learn_ Simulation User Interface at [mtl.how/sim](https://forio.com/app/va/va-psd-sim/login.html) also empowers teams to assess the impacts of improved implementation of measurement-based care and how it can drive improvements in care quality. Do you want to learn more about the key drivers of higher care quality to improve recovery and prevent suicide? Watch that video to find out.
+Why use measurement-based care to improve the time to detect patient improvement or risk? To improve care quality in the flow of patients through care to recovery, the major value of measurement-based care is ensuring Veterans are getting appropriate care and are responding to the care they receive. Clinicians are likely familiar with the emphasis on measurement-based care, but from the perspective of care flow, what is powerful is reducing the time it takes for a team to detect whether a patient is getting better or worse. In the _Modeling to Learn_ Data User Interface at [mtl.how/data](https://app.powerbigov.us/groups/me/apps/b9686a29-6857-46c9-bdf9-043ca2b29138/reports/05dd8dbd-313f-4993-b406-6feea2fdb060/ReportSection?ctid=e95f1b23-abaf-45ee-821d-b7ab251ab3bf), the dataMeas tab provides tables of all the patient measures recorded at the VA Corporate Data Warehouse, whether they were captured via an evidence-based psychotherapy template, mental health assistant, or other approach. These include measures such as the GAD-7 for anxiety, PHQ-2 or PHQ-9 for depression, PCL-5 for PTSD, and the Brief Addiction Monitor or BAM. As a clinician, I like to use the patient-level views to sort scores from high to low to identify patients who may need to step up to a higher level of care or might benefit from referral to a specialty substance use disorder or PTSD service. The lowest score may indicate Veterans who can graduate from outpatient mental health care and step back down to primary care, which is an amazing day for the Veteran, their loved ones, and their treatment team. The vizMeas tab helps teams see trends in these measures to confirm when their efforts to implement measurement-based care are paying off. Teams can toggle to see what proportion of services include measurement-based care by comparing the vizMeas and vizEnc views. The _Modeling to Learn_ Simulation User Interface at [mtl.how/sim](https://forio.com/app/va/va-psd-sim/login.html) also empowers teams to assess the impacts of improved implementation of measurement-based care and how it can drive improvements in care quality. Do you want to learn more about the key drivers of higher care quality to improve recovery and prevent suicide? Watch that video to find out.
## How can we improve SAIL with _Modeling to Learn_?
@@ -56,7 +56,7 @@ Hi, I'm Lindsey and this is David. Why use measurement-based care to improve the
[](https://bcove.video/4dmPVX5)
-Hi, I'm Lindsey and this is David. If you're watching this video, you want to know about strategic analytics improvement in learning, or SAIL metrics, with _Modeling to Learn_. _Modeling to Learn_ was developed to complement existing VA priorities. One important benchmark is SAIL. Some of the more frustrating aspects of SAIL are the lag in data to support daily decision-making in the clinic and emphasis on high-level facility views and, of course, how challenging it can be to improve sale metrics over time, especially when you're short-staffed. In fact, when SAIL metrics change over time, either in a positive or negative direction, it is difficult to know what's driving the change. Facility leadership and clinical teams may not know how to interpret these changes or what improvement strategy to try next. _Modeling to Learn_ supports two primary dimensions of the SAIL mental health quality domain—care coverage and care continuity. But importantly, _Modeling to Learn_ recognizes that these two dimensions may work against each other in a dynamic tension over time. If you're expanding population coverage measures, such as PSY 32 for depression, PSY 38, or PTSD 56 for PTSD, or SUD 16 for opioid use disorder, then it may be more challenging to ensure continuity of care measures such as MDD43h, MDD47h, and PSY 33 for depression and PSY 39 for PTSD. Of course, the reverse is also true. If you're doing great with care continuity on those measures, then it may be challenging to expand population coverage at the same time. I'm fond of calling this the physics of our care quality problems. What I mean is that we cannot change the laws of physics, such as the law of conservation, which states that matter and energy in a physical process cannot be created or destroyed, but can only be transformed. In _Modeling to Learn_, staff time is neither created nor destroyed. This means that when data is exported from the _MTL_ Data User Interface and uploaded to the _MTL_ Simulation User Interface, the scenarios a team or VA explores to find improvements in care quality will always account for or conserve the total local staff time available. This helps to avoid fixes that fail, such as something that improves population coverage in the short term but reduces care continuity in the long term. Have you ever felt like you were playing Whac-A-Mole® with these metrics, where you improve one indicator only to have a problem later somewhere else? _Modeling to Learn_ can help. Let's start with why _Modeling to Learn_ Red is useful. And why does the _Modeling to Learn_ Data User Interface provide new insights? Watch that video to find out.
+If you're watching this video, you want to know about strategic analytics improvement in learning, or SAIL metrics, with _Modeling to Learn_. _Modeling to Learn_ was developed to complement existing VA priorities. One important benchmark is SAIL. Some of the more frustrating aspects of SAIL are the lag in data to support daily decision-making in the clinic and emphasis on high-level facility views and, of course, how challenging it can be to improve sale metrics over time, especially when you're short-staffed. In fact, when SAIL metrics change over time, either in a positive or negative direction, it is difficult to know what's driving the change. Facility leadership and clinical teams may not know how to interpret these changes or what improvement strategy to try next. _Modeling to Learn_ supports two primary dimensions of the SAIL mental health quality domain—care coverage and care continuity. But importantly, _Modeling to Learn_ recognizes that these two dimensions may work against each other in a dynamic tension over time. If you're expanding population coverage measures, such as PSY 32 for depression, PSY 38, or PTSD 56 for PTSD, or SUD 16 for opioid use disorder, then it may be more challenging to ensure continuity of care measures such as MDD43h, MDD47h, and PSY 33 for depression and PSY 39 for PTSD. Of course, the reverse is also true. If you're doing great with care continuity on those measures, then it may be challenging to expand population coverage at the same time. I'm fond of calling this the physics of our care quality problems. What I mean is that we cannot change the laws of physics, such as the law of conservation, which states that matter and energy in a physical process cannot be created or destroyed, but can only be transformed. In _Modeling to Learn_, staff time is neither created nor destroyed. This means that when data is exported from the _MTL_ Data User Interface and uploaded to the _MTL_ Simulation User Interface, the scenarios a team or VA explores to find improvements in care quality will always account for or conserve the total local staff time available. This helps to avoid fixes that fail, such as something that improves population coverage in the short term but reduces care continuity in the long term. Have you ever felt like you were playing Whac-A-Mole® with these metrics, where you improve one indicator only to have a problem later somewhere else? _Modeling to Learn_ can help. Let's start with why _Modeling to Learn_ Red is useful. And why does the _Modeling to Learn_ Data User Interface provide new insights? Watch that video to find out.
## How do five key variables drive care quality over time?
@@ -64,7 +64,7 @@ Hi, I'm Lindsey and this is David. If you're watching this video, you want to kn
[](https://bcove.video/3A3sBiG)
-Hi, I'm Lindsey and this is David. How do five key variables drive care quality? _Modeling to Learn_ emphasizes the dynamics of care over time, which can be accurately simplified to the key time-based variables that drive care quality. But be careful, the important principle is that these variables operate together over time to define an episode of care. That means care quality cannot be improved without understanding how these variables influence one another. If you want to see what Lindsey's talking about, navigate to the _Modeling to Learn_ Data User Interface at [mtl.how/data](https://app.powerbigov.us/groups/me/apps/b9686a29-6857-46c9-bdf9-043ca2b29138/reports/05dd8dbd-313f-4993-b406-6feea2fdb060/ReportSection?ctid=e95f1b23-abaf-45ee-821d-b7ab251ab3bf) and review each care problem—care coordination, medication management, psychotherapy, team care, and team flow. Our partners across VA describe how challenging it is to review data in one information system and then in another and end up unsure how to reconcile them, especially when you think that they indicate a different course of action. Folks are extremely busy and any new data resources must really add value to be worth learning. Let's see if these _Modeling to Learn_ variables meet the commonsense test of value added. Well, what do you think the five time-based variables are that make up an evidence-based episode of care? All outpatient care is defined by whether you can get an appointment when you need help. We focus on this all the time in VA. But then, and this is critical, you must be able to be seen again to complete a therapeutic course of care adequate to meet your need. Clinicians told us that a week was the way they think clinically. So, in _Modeling to Learn_, teams make their clinic selections to obtain an estimate of their local new patient start rate in patients per week and their appointment supply in appointments per week. Then we define evidence-based engagement as the new patient wait time in weeks, time between visits and weeks, and the engagement duration over time, again in weeks. This is a simplified definition of an evidence-based episode of care that is accurate for time. Why is it wise to focus on the dynamics of care over time? That's why _Modeling to Learn Blue_ is useful. So watch that video to find out.
+How do five key variables drive care quality? _Modeling to Learn_ emphasizes the dynamics of care over time, which can be accurately simplified to the key time-based variables that drive care quality. But be careful, the important principle is that these variables operate together over time to define an episode of care. That means care quality cannot be improved without understanding how these variables influence one another. If you want to see what Lindsey's talking about, navigate to the _Modeling to Learn_ Data User Interface at [mtl.how/data](https://app.powerbigov.us/groups/me/apps/b9686a29-6857-46c9-bdf9-043ca2b29138/reports/05dd8dbd-313f-4993-b406-6feea2fdb060/ReportSection?ctid=e95f1b23-abaf-45ee-821d-b7ab251ab3bf) and review each care problem—care coordination, medication management, psychotherapy, team care, and team flow. Our partners across VA describe how challenging it is to review data in one information system and then in another and end up unsure how to reconcile them, especially when you think that they indicate a different course of action. Folks are extremely busy and any new data resources must really add value to be worth learning. Let's see if these _Modeling to Learn_ variables meet the commonsense test of value added. Well, what do you think the five time-based variables are that make up an evidence-based episode of care? All outpatient care is defined by whether you can get an appointment when you need help. We focus on this all the time in VA. But then, and this is critical, you must be able to be seen again to complete a therapeutic course of care adequate to meet your need. Clinicians told us that a week was the way they think clinically. So, in _Modeling to Learn_, teams make their clinic selections to obtain an estimate of their local new patient start rate in patients per week and their appointment supply in appointments per week. Then we define evidence-based engagement as the new patient wait time in weeks, time between visits and weeks, and the engagement duration over time, again in weeks. This is a simplified definition of an evidence-based episode of care that is accurate for time. Why is it wise to focus on the dynamics of care over time? That's why _Modeling to Learn Blue_ is useful. So watch that video to find out.
## How does _Modeling to Learn_ help improve medication management?
@@ -72,7 +72,7 @@ Hi, I'm Lindsey and this is David. How do five key variables drive care quality?
[](https://bcove.video/3To3mP4)
-Hi, I'm Lindsey and this is Debbie. How does _Modeling to Learn_ improve medication management? Medication management requires identifying a patient need, starting a medication, and evaluating for a therapeutic response over time. Some of the challenges and effective medication management over time include the realities that not all clinicians can prescribe. And of course, it requires follow up care to ensure that a patient is benefiting from their medication. So, when should the patient be seen again? One thing that can be so frustrating to prescribers is that a scheduler tells them when a patient can be seen again, rather than them telling the scheduler when the patient should be seen based on top of licensed care. The frustration is that medication management return-to-clinic visits may not occur at the appropriate interval to balance patient needs and risks. The VA has guidelines for some patient cohorts such as SAIL, MDD43h, and MDD47h, which define high quality continuity of follow up care as a valuation once every 12 weeks for Veterans diagnosed with depression who are using antidepressants. Other needs may reflect a different follow-up interval. For example, many addiction psychiatrists have described the appropriate medication management return-to-clinic visit interval for buprenorphine or methadone for opioid use disorder as every four weeks. These differences in medication needs, and the evidence-based pharmacotherapy standards that go with them, make it hard for prescribers to evaluate at a patient population level in a real time, especially when you're completing another medication management encounter every 20 minutes. And of course, depending on the community at any given time, the staffing mix may change and the size or volume of patient needs can change too. Some SUD teams may have a high volume of patients receiving evidence-based pharmacotherapy for alcohol use disorder, whereas the modal presenting concern in a BHIP may be more likely to be depression. And the decisions the prescribers are making using current rules of thumb may be boxing them in to a pattern of care that makes it even harder to be responsive to crises or reduce future wait times. The Medication Management module of _Modeling to Learn_ helps a team evaluate the trade-offs among all these factors as compared to the status quo over the last two years in the clinic. Comparing to the base case of no new decisions, the team can evaluate changes in their X-waivered MOUD appointment supply or evaluate and tailor the return-to-clinic visit interval by patient cohort. The goal is to help prescribers see what daily medication management decisions are adding up to over time and find feasible and effective local rules of thumb for clinically appropriate return-to-clinic orders in weeks that reduce wait times and ensure Veterans are getting better. For coordinating medication management in a multidisciplinary team such as a BHIP or SUD program, the Team Care module of _Modeling to Learn_ can also help. If you want to know how _Modeling to Learn_ helps to improve psychotherapy, watch that video to find out.
+How does _Modeling to Learn_ improve medication management? Medication management requires identifying a patient need, starting a medication, and evaluating for a therapeutic response over time. Some of the challenges and effective medication management over time include the realities that not all clinicians can prescribe. And of course, it requires follow up care to ensure that a patient is benefiting from their medication. So, when should the patient be seen again? One thing that can be so frustrating to prescribers is that a scheduler tells them when a patient can be seen again, rather than them telling the scheduler when the patient should be seen based on top of licensed care. The frustration is that medication management return-to-clinic visits may not occur at the appropriate interval to balance patient needs and risks. The VA has guidelines for some patient cohorts such as SAIL, MDD43h, and MDD47h, which define high quality continuity of follow up care as a valuation once every 12 weeks for Veterans diagnosed with depression who are using antidepressants. Other needs may reflect a different follow-up interval. For example, many addiction psychiatrists have described the appropriate medication management return-to-clinic visit interval for buprenorphine or methadone for opioid use disorder as every four weeks. These differences in medication needs, and the evidence-based pharmacotherapy standards that go with them, make it hard for prescribers to evaluate at a patient population level in a real time, especially when you're completing another medication management encounter every 20 minutes. And of course, depending on the community at any given time, the staffing mix may change and the size or volume of patient needs can change too. Some SUD teams may have a high volume of patients receiving evidence-based pharmacotherapy for alcohol use disorder, whereas the modal presenting concern in a BHIP may be more likely to be depression. And the decisions the prescribers are making using current rules of thumb may be boxing them in to a pattern of care that makes it even harder to be responsive to crises or reduce future wait times. The Medication Management module of _Modeling to Learn_ helps a team evaluate the trade-offs among all these factors as compared to the status quo over the last two years in the clinic. Comparing to the base case of no new decisions, the team can evaluate changes in their X-waivered MOUD appointment supply or evaluate and tailor the return-to-clinic visit interval by patient cohort. The goal is to help prescribers see what daily medication management decisions are adding up to over time and find feasible and effective local rules of thumb for clinically appropriate return-to-clinic orders in weeks that reduce wait times and ensure Veterans are getting better. For coordinating medication management in a multidisciplinary team such as a BHIP or SUD program, the Team Care module of _Modeling to Learn_ can also help. If you want to know how _Modeling to Learn_ helps to improve psychotherapy, watch that video to find out.
## How does _Modeling_ to Learn help improve psychotherapy?
@@ -80,7 +80,7 @@ Hi, I'm Lindsey and this is Debbie. How does _Modeling to Learn_ improve medicat
[](https://bcove.video/4gjSdbG)
-Hi, I'm Lindsey and this is David. How does _Modeling to Learn_ help improve psychotherapy? VA led the nation in dissemination and training for evidence-based psychotherapy. Many clinicians are ready to offer Veterans the highest quality care available. So, what makes it hard to get Veterans through an evidence-based episode of care when they start? What I've learned from working with clinical psychologists like Lindsey and other VA therapists from around the country is that the dynamics of psychotherapy in the clinic are challenging for the same reasons most care problems persist over time. Clinical teams typically do not have good visibility of where all their Veterans are in care. You may know that you plan to do CPT session six today with an individual Veteran, but how many Veterans are currently in the middle of therapy now? How many appointments are on the books? How many weeks are there before they return to clinic for follow-up care and how many graduate when they complete a course of an evidence-based psychotherapy? And how many drop out of therapy after one session or early in care but return to the team later? Evidence-based psychotherapy is typically defined by getting near weekly psychotherapy and having clinically meaningful improvement, which usually starts around session 8 to 12. Many VA data systems provide useful counts and proportions of evidence-based psychotherapy templates and the SAIL continuity of care metrics to find quality based on the number of sessions Veterans get within a specific time period. But the SAIL metric itself is a quarterly snapshot, and we know Veterans don't all start therapy at the beginning of a new quarter. In the _Modeling to Learn_ Data User Interface, a one-year patient cohort is identified that includes all Veterans seen from 18 months to six months ago based on the clinic selections made with the Data User Interface at [mtl.how/data](https://app.powerbigov.us/groups/me/apps/b9686a29-6857-46c9-bdf9-043ca2b29138/reports/05dd8dbd-313f-4993-b406-6feea2fdb060/ReportSection?ctid=e95f1b23-abaf-45ee-821d-b7ab251ab3bf). Then, for the Veterans seen within those 12 months, we look back to see when those individual Veterans began therapy and how they were engaged over time. Once this psychotherapy cohort is defined and uploaded to the Simulation User Interface at [mtl.how/sim](https://forio.com/app/va/va-psd-sim/login.html), we can look at the stocks and see where all the Veterans have accumulated, where they are in care in a typical week, and how many patients per week are progressing through key therapy milestones. Finally, at a local team level, psychotherapists can see how they are doing for most of the Veterans they serve. And even better, they can run experiments to see what the impact will be of making new decisions over time. You might wonder _What if we keep making the same care decisions? Will things get better or worse?_ Watch that video to find out.
+How does _Modeling to Learn_ help improve psychotherapy? VA led the nation in dissemination and training for evidence-based psychotherapy. Many clinicians are ready to offer Veterans the highest quality care available. So, what makes it hard to get Veterans through an evidence-based episode of care when they start? What I've learned from working with clinical psychologists like Lindsey and other VA therapists from around the country is that the dynamics of psychotherapy in the clinic are challenging for the same reasons most care problems persist over time. Clinical teams typically do not have good visibility of where all their Veterans are in care. You may know that you plan to do CPT session six today with an individual Veteran, but how many Veterans are currently in the middle of therapy now? How many appointments are on the books? How many weeks are there before they return to clinic for follow-up care and how many graduate when they complete a course of an evidence-based psychotherapy? And how many drop out of therapy after one session or early in care but return to the team later? Evidence-based psychotherapy is typically defined by getting near weekly psychotherapy and having clinically meaningful improvement, which usually starts around session 8 to 12. Many VA data systems provide useful counts and proportions of evidence-based psychotherapy templates and the SAIL continuity of care metrics to find quality based on the number of sessions Veterans get within a specific time period. But the SAIL metric itself is a quarterly snapshot, and we know Veterans don't all start therapy at the beginning of a new quarter. In the _Modeling to Learn_ Data User Interface, a one-year patient cohort is identified that includes all Veterans seen from 18 months to six months ago based on the clinic selections made with the Data User Interface at [mtl.how/data](https://app.powerbigov.us/groups/me/apps/b9686a29-6857-46c9-bdf9-043ca2b29138/reports/05dd8dbd-313f-4993-b406-6feea2fdb060/ReportSection?ctid=e95f1b23-abaf-45ee-821d-b7ab251ab3bf). Then, for the Veterans seen within those 12 months, we look back to see when those individual Veterans began therapy and how they were engaged over time. Once this psychotherapy cohort is defined and uploaded to the Simulation User Interface at [mtl.how/sim](https://forio.com/app/va/va-psd-sim/login.html), we can look at the stocks and see where all the Veterans have accumulated, where they are in care in a typical week, and how many patients per week are progressing through key therapy milestones. Finally, at a local team level, psychotherapists can see how they are doing for most of the Veterans they serve. And even better, they can run experiments to see what the impact will be of making new decisions over time. You might wonder _What if we keep making the same care decisions? Will things get better or worse?_ Watch that video to find out.
## How does our appointment backlog extend weeks between visits?
@@ -88,7 +88,7 @@ Hi, I'm Lindsey and this is David. How does _Modeling to Learn_ help improve psy
[](https://bcove.video/4d3jVGG)
-Hi, I'm Lindsey and this is David. How does an appointment backlog extend the weeks between visits? At first, this sounds like a simple system story that many clinicians describe as we work together to co-develop _Modeling to Learn_. But they also told us when an appointment backlog extends the week between visits, a long interval between visits effects care too. That's right. As the number of appointments for a specific service grows on the books, then the clinic has no choice but to increase the number of weeks or push out the time between visits to get everyone seen. We call this the return-to-clinic visit interval. This quantifies the return-to-clinic order in a unit of time—one week—which is critical for finding realistic local improvements in our partnerships with sites. Okay, so the balancing feedback principle operating here is that if you have a clinic calendar filled up with appointments, it's harder for new patients to start care _and_ it becomes harder for all existing patients to be seen with appropriate therapeutic follow-up standards for psychotherapy and pharmacotherapy. Lindsey, you and Debbie also talked about balancing feedback in the video, _What if we keep making the same care decisions, will things get better or worse?_ For an explanation of how start delays affect treatment decisions over time, watch that video. As I said at the beginning of this video, it's not just that a growing appointment backlog forces teams and clinics to extend the number of weeks between visits to get Veterans seen. It's that long return-to-clinic visit intervals impact treatment decisions and care quality. None of these effects occur in isolation. All these components of care interact over time. It's why system scientists will often point out there are no side effects and systems, there are only effects. Some we want and some we don't. As clinicians identify Veterans who might benefit from an evidence-based course of psychotherapy for depression or PTSD, they may find that many patients are waiting to start, or they know that those 90833 or 90834 psychotherapy encounters are occurring 12 weeks apart—or both. Clinicians recognize that they cannot see patients in a way that is consistent with an evidence-based episode of care and will have to start finding alternatives. _Modeling to Learn_ can help. What a relief that hundreds of clinicians, managers, data experts, quality improvement leads, and patients have already worked with us over the last decade to develop scalable _Modeling to Learn_ resources for clinical and improvement teams to use today. The _Modeling to Learn_ Simulation User Interface accounts for all these feedback effects so the team can upload their data, review what they see, and get immediate feedback about what the most feasible alternatives are to ensure quality of care for patients and quality of work life for providers and local staff. Based on this discussion, you may wonder _How can we better balance the needs of new and existing patients? Should we prioritize new patient start rate or weeks between visits for quality?_ Watch that video to find out.
+How does an appointment backlog extend the weeks between visits? At first, this sounds like a simple system story that many clinicians describe as we work together to co-develop _Modeling to Learn_. But they also told us when an appointment backlog extends the week between visits, a long interval between visits effects care too. That's right. As the number of appointments for a specific service grows on the books, then the clinic has no choice but to increase the number of weeks or push out the time between visits to get everyone seen. We call this the return-to-clinic visit interval. This quantifies the return-to-clinic order in a unit of time—one week—which is critical for finding realistic local improvements in our partnerships with sites. Okay, so the balancing feedback principle operating here is that if you have a clinic calendar filled up with appointments, it's harder for new patients to start care _and_ it becomes harder for all existing patients to be seen with appropriate therapeutic follow-up standards for psychotherapy and pharmacotherapy. Lindsey, you and Debbie also talked about balancing feedback in the video, _What if we keep making the same care decisions, will things get better or worse?_ For an explanation of how start delays affect treatment decisions over time, watch that video. As I said at the beginning of this video, it's not just that a growing appointment backlog forces teams and clinics to extend the number of weeks between visits to get Veterans seen. It's that long return-to-clinic visit intervals impact treatment decisions and care quality. None of these effects occur in isolation. All these components of care interact over time. It's why system scientists will often point out there are no side effects and systems, there are only effects. Some we want and some we don't. As clinicians identify Veterans who might benefit from an evidence-based course of psychotherapy for depression or PTSD, they may find that many patients are waiting to start, or they know that those 90833 or 90834 psychotherapy encounters are occurring 12 weeks apart—or both. Clinicians recognize that they cannot see patients in a way that is consistent with an evidence-based episode of care and will have to start finding alternatives. _Modeling to Learn_ can help. What a relief that hundreds of clinicians, managers, data experts, quality improvement leads, and patients have already worked with us over the last decade to develop scalable _Modeling to Learn_ resources for clinical and improvement teams to use today. The _Modeling to Learn_ Simulation User Interface accounts for all these feedback effects so the team can upload their data, review what they see, and get immediate feedback about what the most feasible alternatives are to ensure quality of care for patients and quality of work life for providers and local staff. Based on this discussion, you may wonder _How can we better balance the needs of new and existing patients? Should we prioritize new patient start rate or weeks between visits for quality?_ Watch that video to find out.
## How can we better balance needs of new and existing patients?
@@ -96,7 +96,7 @@ Hi, I'm Lindsey and this is David. How does an appointment backlog extend the we
[](https://bcove.video/3yaXy41)
-Hi, I'm Lindsey and this is Debbie. How can we better balance the needs of new and existing patients? Should we prioritize new patient start rate or weeks between visits? These trade-offs are challenging for clinicians when you know there is a whole community of patients who need help. When teams are struggling with the limits of their available time in the day to see patients, it can feel like a clinical, ethical, and even moral quandary about how to best balance patients’ needs for services with the staff resources available. Of course, staff resources are always changing. We know many teams that are excelling in providing the highest quality addiction and mental health care available. But the behavioral health workforce shortage in the US is much bigger than VA. And VAs and teams need the right tools to make sure Veterans get the care they need for recovery. Challenges balancing the new patient start rate and the weeks between visits for existing patients apply systems thinking insights that we've talked about in other _MTL_ videos, such as the physics of conserving staff time in order to ensure a clinically beneficial, realistic approach to care decisions and quality improvement that meets VA quality standards _and_ meets Veterans needs for evidence-based episodes of care. The _Modeling to Learn Blue_ Simulation User Interface available at [mtl.how/sim](https://forio.com/app/va/va-psd-sim/login.html) enables a site or team to evaluate these two balancing system stories as a function of their data for the last two years exported from _Modeling to Learn Red_ Data User Interface at [mtl.how/data](https://app.powerbigov.us/groups/me/apps/b9686a29-6857-46c9-bdf9-043ca2b29138/reports/05dd8dbd-313f-4993-b406-6feea2fdb060/ReportSection?ctid=e95f1b23-abaf-45ee-821d-b7ab251ab3bf). As we talked about in the _How does an appointment backlog extend the weeks between visits?_ video and _What if we keep making the same care decisions, will things get better or worse?_ video, balancing feedbacks occur in any system that has a goal, including our healthcare and clinical systems. Balancing feedbacks occur in systems that have constraints of resources, staff, and time. And as a result, the trends that occur over time tend to reset to a status quo or oscillate around the status quo. A brief _Modeling to Learn_ consult uses the site or team reviewed local data plus simulation to efficiently find local improvements that account for all these balancing trade-offs. We aim to help clinical and improvement teams find a couple empowering clinical heuristics, or rules of thumb, that are more effective for ensuring evidence-based care to get more Veterans better. When we partner with a VA or a team, we aim to find the lightest lift we can, like increase groups by 10% or adjust the return-to-clinic order for three weeks for these presenting concerns. And we often find something small that has a big payoff for Veterans. Do you want to be empowered to leverage the feedback, flow, and volume of your local care system? Watch that video to find out.
+How can we better balance the needs of new and existing patients? Should we prioritize new patient start rate or weeks between visits? These trade-offs are challenging for clinicians when you know there is a whole community of patients who need help. When teams are struggling with the limits of their available time in the day to see patients, it can feel like a clinical, ethical, and even moral quandary about how to best balance patients’ needs for services with the staff resources available. Of course, staff resources are always changing. We know many teams that are excelling in providing the highest quality addiction and mental health care available. But the behavioral health workforce shortage in the US is much bigger than VA. And VAs and teams need the right tools to make sure Veterans get the care they need for recovery. Challenges balancing the new patient start rate and the weeks between visits for existing patients apply systems thinking insights that we've talked about in other _MTL_ videos, such as the physics of conserving staff time in order to ensure a clinically beneficial, realistic approach to care decisions and quality improvement that meets VA quality standards _and_ meets Veterans needs for evidence-based episodes of care. The _Modeling to Learn Blue_ Simulation User Interface available at [mtl.how/sim](https://forio.com/app/va/va-psd-sim/login.html) enables a site or team to evaluate these two balancing system stories as a function of their data for the last two years exported from _Modeling to Learn Red_ Data User Interface at [mtl.how/data](https://app.powerbigov.us/groups/me/apps/b9686a29-6857-46c9-bdf9-043ca2b29138/reports/05dd8dbd-313f-4993-b406-6feea2fdb060/ReportSection?ctid=e95f1b23-abaf-45ee-821d-b7ab251ab3bf). As we talked about in the _How does an appointment backlog extend the weeks between visits?_ video and _What if we keep making the same care decisions, will things get better or worse?_ video, balancing feedbacks occur in any system that has a goal, including our healthcare and clinical systems. Balancing feedbacks occur in systems that have constraints of resources, staff, and time. And as a result, the trends that occur over time tend to reset to a status quo or oscillate around the status quo. A brief _Modeling to Learn_ consult uses the site or team reviewed local data plus simulation to efficiently find local improvements that account for all these balancing trade-offs. We aim to help clinical and improvement teams find a couple empowering clinical heuristics, or rules of thumb, that are more effective for ensuring evidence-based care to get more Veterans better. When we partner with a VA or a team, we aim to find the lightest lift we can, like increase groups by 10% or adjust the return-to-clinic order for three weeks for these presenting concerns. And we often find something small that has a big payoff for Veterans. Do you want to be empowered to leverage the feedback, flow, and volume of your local care system? Watch that video to find out.
## How can we leverage the feedback, rates, and volume of our local care system?
@@ -104,7 +104,7 @@ Hi, I'm Lindsey and this is Debbie. How can we better balance the needs of new a
[](https://bcove.video/4fonk5v)
-Hi, I'm Lindsey and this is David. Do you want to be empowered to leverage the feedback, flow, and volume of your local care system? In any system, leverage describes the places where a small shift in one thing can produce big changes in everything. For a big benefit for Veterans in their care teams, the primary places to focus are where the greatest volume is, the fastest flower rates are, and of course to account for feedback dynamics over time. When we partner with the VA or team, we aim to find the lightest lift we can, like finding for one BHIP that increasing group psychotherapy by 10% eliminates wait times and gets five times the number of Veterans near weekly therapy. Or, adjusting the return-to-clinic order by three weeks for some presenting concerns can provide the same medication management supply as hiring another prescriber. We can often find small care decisions that when implemented frequently over time, have a big payoff for Veterans because they add to more than the sum of the parts in the interdependent system over time. The last thing we want to do is make big changes that are difficult for Veterans and clinical teams that produce limited benefit over time. In my experience over many years now, finding something clinically powerful that leverages the dynamics of the system can be a huge boost to morale, productivity, and staff empowerment. You might wonder _How can you find powerful insights that you are sure will work locally?_ We do this by reviewing the _Modeling to Learn_ Simulation User Interface stocks to see where all the Veterans are accumulating the most. We review the rates to see where the floodwaters are rising fastest and require action. Or where things are trending in the right direction and our attention could be better focused elsewhere. _Modeling to Learn_ consultation offers clinical and improvement teams real-time efficient support. We can do this quickly because we've partnered with patients, providers, policy makers, data experts, evaluators, and quality improvement consultants from the frontline to VA Central Office for nearly 10 years now to define the primary drivers of care problems and care quality using system dynamics. _Modeling to Learn_ consultation involves empowering clinicians, managers, improvers, data leads, and all of VA to leverage the system for getting more Veterans better. How can _Modeling to Learn_ consultation help? Watch that video to find out.
+Do you want to be empowered to leverage the feedback, flow, and volume of your local care system? In any system, leverage describes the places where a small shift in one thing can produce big changes in everything. For a big benefit for Veterans in their care teams, the primary places to focus are where the greatest volume is, the fastest flower rates are, and of course to account for feedback dynamics over time. When we partner with the VA or team, we aim to find the lightest lift we can, like finding for one BHIP that increasing group psychotherapy by 10% eliminates wait times and gets five times the number of Veterans near weekly therapy. Or, adjusting the return-to-clinic order by three weeks for some presenting concerns can provide the same medication management supply as hiring another prescriber. We can often find small care decisions that when implemented frequently over time, have a big payoff for Veterans because they add to more than the sum of the parts in the interdependent system over time. The last thing we want to do is make big changes that are difficult for Veterans and clinical teams that produce limited benefit over time. In my experience over many years now, finding something clinically powerful that leverages the dynamics of the system can be a huge boost to morale, productivity, and staff empowerment. You might wonder _How can you find powerful insights that you are sure will work locally?_ We do this by reviewing the _Modeling to Learn_ Simulation User Interface stocks to see where all the Veterans are accumulating the most. We review the rates to see where the floodwaters are rising fastest and require action. Or where things are trending in the right direction and our attention could be better focused elsewhere. _Modeling to Learn_ consultation offers clinical and improvement teams real-time efficient support. We can do this quickly because we've partnered with patients, providers, policy makers, data experts, evaluators, and quality improvement consultants from the frontline to VA Central Office for nearly 10 years now to define the primary drivers of care problems and care quality using system dynamics. _Modeling to Learn_ consultation involves empowering clinicians, managers, improvers, data leads, and all of VA to leverage the system for getting more Veterans better. How can _Modeling to Learn_ consultation help? Watch that video to find out.
## What key drivers of higher care quality improve recovery and prevent suicide?
@@ -112,4 +112,4 @@ Hi, I'm Lindsey and this is David. Do you want to be empowered to leverage the f
[](https://bcove.video/4c1YgOQ)
-Hi, I'm Lindsey and this is Debbie. What are key drivers of higher care quality to improve recovery and prevent suicide? A central idea of _Modeling to Learn_ resources is that delivering consistent, timely, high-quality care is challenging for teams because we don't accurately account for the primary dynamics driving care quality over time. You might be thinking _Care dynamics, what does that mean?_ We literally mean that due to time pressures and cognitive limits, we don't effectively understand relationships among key care decisions that we make every day over time, especially decisions made in the aggregate across a clinical team. For example, we may try to break down problems by looking at referrals or wait times, but we don't look at how clinical teams’ knowledge of wait times influence referrals to a specific service or team. In the _Modeling to Learn_ Data User Interface, we define care dynamics in terms of rates over time, like new patients per week, so that we can understand the flow of Veterans through care. As another example in terms of care quality, recovery and suicide prevention, the _Modeling to Learn_ Team Flow module looks not only at the number of patients served by a team or site, or the number of patients with a high-risk flag for suicide, but also the time it takes to unflag high-risk patients in a team and the typical time to improve within the team over the last two years. The _Modeling to Learn_ Simulation User Interface uses an interactive structure fed by local team data from the Data User Interface. The _Modeling to Learn_ Simulation User Interface visually depicts accumulation of patients in desirable or undesirable states of care, like waiting to start a new service. These states are defined by their inflows and outflows. A lot of clinical teams and mental health leadership wonder whether they can get any new insights when they are already doing everything that they can to support the Veterans in their care. Would you like to know how to improve SAIL with _Modeling to Learn_? Watch that video to find out.
+What are key drivers of higher care quality to improve recovery and prevent suicide? A central idea of _Modeling to Learn_ resources is that delivering consistent, timely, high-quality care is challenging for teams because we don't accurately account for the primary dynamics driving care quality over time. You might be thinking _Care dynamics, what does that mean?_ We literally mean that due to time pressures and cognitive limits, we don't effectively understand relationships among key care decisions that we make every day over time, especially decisions made in the aggregate across a clinical team. For example, we may try to break down problems by looking at referrals or wait times, but we don't look at how clinical teams’ knowledge of wait times influence referrals to a specific service or team. In the _Modeling to Learn_ Data User Interface, we define care dynamics in terms of rates over time, like new patients per week, so that we can understand the flow of Veterans through care. As another example in terms of care quality, recovery and suicide prevention, the _Modeling to Learn_ Team Flow module looks not only at the number of patients served by a team or site, or the number of patients with a high-risk flag for suicide, but also the time it takes to unflag high-risk patients in a team and the typical time to improve within the team over the last two years. The _Modeling to Learn_ Simulation User Interface uses an interactive structure fed by local team data from the Data User Interface. The _Modeling to Learn_ Simulation User Interface visually depicts accumulation of patients in desirable or undesirable states of care, like waiting to start a new service. These states are defined by their inflows and outflows. A lot of clinical teams and mental health leadership wonder whether they can get any new insights when they are already doing everything that they can to support the Veterans in their care. Would you like to know how to improve SAIL with _Modeling to Learn_? Watch that video to find out.
diff --git a/docs/3-technical-documentation.html b/docs/3-technical-documentation.html
index 8f6fb375..129be4f1 100644
--- a/docs/3-technical-documentation.html
+++ b/docs/3-technical-documentation.html
@@ -23,7 +23,7 @@
-
+
diff --git a/docs/404.html b/docs/404.html
index 06139b4c..39991169 100644
--- a/docs/404.html
+++ b/docs/404.html
@@ -23,7 +23,7 @@
-
+
diff --git a/docs/index.html b/docs/index.html
index 7d9d74b6..dfef52f0 100644
--- a/docs/index.html
+++ b/docs/index.html
@@ -23,7 +23,7 @@
-
+
@@ -154,7 +154,7 @@ 2024-10-22
+2024-11-07
Hi, I’m Lindsey and this is Debbie. You might wonder What is Modeling to Learn, why use it, and how might it help? We often don’t understand what is driving our care problems over time. Modeling to Learn helps to upgrade the decisions you make all day, every day, because the decisions made most frequently will be the most powerful for improving our care. We’re introducing Modeling to Learn, which has been developed and evaluated in the VA for over 9 years. Modeling to Learn is a way to upgrade care decisions by understanding the dynamics of common care problems over time. Are you very, very busy? Modeling to Learn includes step by step guides with gifs showing each click. These guides are available at mtl.how. Are you unsure what data to use, how it’s defined, and are you wishing it was current enough to guide your care decisions? Modeling to Learn includes a transparent, real-time locally defined data user interface available at mtl.how/data, where you specify the clinics that define your data views. Are you unsure what is realistic for improving care locally, given your patients’ needs and your existing staff and resources? Modeling to Learn also includes a simulation modeling interface available at mtl.how/sim, so you can try out new decisions before you implement them in the clinic. We all want to meet Veterans needs for timely, high-quality care. So, what gets in our way? Watch that video to find out.
+You might wonder What is Modeling to Learn, why use it, and how might it help? We often don’t understand what is driving our care problems over time. Modeling to Learn helps to upgrade the decisions you make all day, every day, because the decisions made most frequently will be the most powerful for improving our care. We’re introducing Modeling to Learn, which has been developed and evaluated in the VA for over 9 years. Modeling to Learn is a way to upgrade care decisions by understanding the dynamics of common care problems over time. Are you very, very busy? Modeling to Learn includes step by step guides with gifs showing each click. These guides are available at mtl.how. Are you unsure what data to use, how it’s defined, and are you wishing it was current enough to guide your care decisions? Modeling to Learn includes a transparent, real-time locally defined data user interface available at mtl.how/data, where you specify the clinics that define your data views. Are you unsure what is realistic for improving care locally, given your patients’ needs and your existing staff and resources? Modeling to Learn also includes a simulation modeling interface available at mtl.how/sim, so you can try out new decisions before you implement them in the clinic. We all want to meet Veterans needs for timely, high-quality care. So, what gets in our way? Watch that video to find out.
Hi, I’m Lindsey and this is David. Modeling to Learn improves visibility and provides new insights into how common care problems persist over time. How? Modeling to Learn is based on over 60 years of scholarship known as participatory system dynamics. For this reason, we call ourselves Team PSD for Team Participatory System Dynamics. The Data User Interface and Simulation User Interface comprise two versions of Modeling to Learn. The data-only version is known as Modeling to Learn Red; Modeling to Learn Blue ads participatory learning from simulation. Team PSD supports MTL Red and MTL Blue and is carefully evaluating how each works to support VA in meeting Veterans needs. Learning from simulation can help us to place a better initial bet on what is likely to work locally by evaluating alternative decisions via simulation before we implement them in the real world. MTL Red tells us where we’ve been over the last two years based on the clinic selections made to produce the patient data reports and team trends or visualizations. Many staff report that viewing the Data UI real-time patient data tabs or the team trends is efficient and encouraging. The data tabs help with clinical decision making. The visualizations of team trends provide leading indicators that improvement efforts are paying off, which can be validating. Why is Modeling to Learn useful when we have critical staffing and hiring needs? Watch that video to find out.
+Modeling to Learn improves visibility and provides new insights into how common care problems persist over time. How? Modeling to Learn is based on over 60 years of scholarship known as participatory system dynamics. For this reason, we call ourselves Team PSD for Team Participatory System Dynamics. The Data User Interface and Simulation User Interface comprise two versions of Modeling to Learn. The data-only version is known as Modeling to Learn Red; Modeling to Learn Blue ads participatory learning from simulation. Team PSD supports MTL Red and MTL Blue and is carefully evaluating how each works to support VA in meeting Veterans needs. Learning from simulation can help us to place a better initial bet on what is likely to work locally by evaluating alternative decisions via simulation before we implement them in the real world. MTL Red tells us where we’ve been over the last two years based on the clinic selections made to produce the patient data reports and team trends or visualizations. Many staff report that viewing the Data UI real-time patient data tabs or the team trends is efficient and encouraging. The data tabs help with clinical decision making. The visualizations of team trends provide leading indicators that improvement efforts are paying off, which can be validating. Why is Modeling to Learn useful when we have critical staffing and hiring needs? Watch that video to find out.
Hi, I’m Lindsey and this is David. We all want to meet Veterans’ needs for timely, high-quality care. So, what gets in our way? Modeling to Learn is a way to upgrade care decisions by understanding the dynamics of common care problems over time. Demanding clinical days mean we don’t often have bandwidth to gain new insights, particularly in our understanding of interdependent clinic dynamics over time. We’re often flying blind to the impacts that our daily decisions have on the overall community that relies on us. As clinicians, we look at the patient in front of us and decide when we think they should be seen again. But have you ever been told you cannot see a patient as soon as you would like due to the constraints of the clinic? For example, an evidence-based course of care requires starting treatment without delay and keeping the Veteran engaged in a therapeutic dose of care over time to meet their needs. If we only emphasize Veterans starting care, but the time between visits extends way out, this interferes with evidence-based care and Veterans getting better. How can we get more Veterans better without adding hours in our day? Watch that video to find out.
+We all want to meet Veterans’ needs for timely, high-quality care. So, what gets in our way? Modeling to Learn is a way to upgrade care decisions by understanding the dynamics of common care problems over time. Demanding clinical days mean we don’t often have bandwidth to gain new insights, particularly in our understanding of interdependent clinic dynamics over time. We’re often flying blind to the impacts that our daily decisions have on the overall community that relies on us. As clinicians, we look at the patient in front of us and decide when we think they should be seen again. But have you ever been told you cannot see a patient as soon as you would like due to the constraints of the clinic? For example, an evidence-based course of care requires starting treatment without delay and keeping the Veteran engaged in a therapeutic dose of care over time to meet their needs. If we only emphasize Veterans starting care, but the time between visits extends way out, this interferes with evidence-based care and Veterans getting better. How can we get more Veterans better without adding hours in our day? Watch that video to find out.
Hi, I’m Lindsey and this is Debbie. What if we keep making the same care decisions? Will things get better or worse? This is almost a trick question. If we keep making the same care decisions, more than likely things will stay the same in the clinic. Very rarely do clinical teams not have any local improvements that they would like to make. So, what do we mean by staying the same? We know that things in the clinic are not static but are always dynamic and always changing. We’ll describe how start delays affect treatment decisions to explain what we mean. The Modeling to Learn Simulation User Interface at mtl.how/sim is designed to help a team see what is likely to happen over the next two years if they continue to make the same daily clinical decisions that were made over the past two years. We call this the base case of no new decisions. Any number of feasible alternative decisions a team or site could make can be compared against this base case to find the best options for improving care over time. Because feedback dynamics produce non-linear trends or system behaviors, this means even the base case run can be surprising for some teams. The base case may show even more undesirable likely futures which teams would need to make new decisions to avoid. An example is the way start delays affect treatment decisions through balancing feedback. As the number of patients waiting to start a specific service grows, clinicians must adjust and try to find another way to meet their Veterans’ needs. Over time, as the clinicians learn about a local delay, it starts to affect what they think the most ethical, clinically appropriate treatment for the Veteran will be. An analogy from daily life will help. If there’s a traffic jam outside your VA, you may try to find another route home. As more drivers choose alternative routes, traffic returns to normal. Of course, when drivers don’t know about the jam, it will get worse over time. And whether drivers choose alternative routes or not—with balancing feedback like a traffic jam, where more cars are getting on the highway than are getting off—cars will build up and slow down the rate of traffic for everyone, whether we like it or not. We have these kinds of experiences in our clinics, too. Balancing feedback can be a blessing or curse. A blessing in that the balancing feedback causes the clinical system to return to its status quo. A curse if the status quo is undesirable for quality of patient care or provider quality of work life. When you start Modeling to Learn, it is surprising to see how powerful feedback is in driving care dynamics in any clinical care setting—VA or non-VA, VAMC or CBOC, urban and rural, BHIP, or PCT. General system insights, like the link between balancing feedbacks and nonlinear system behaviors, can be understood quickly from Modeling to Learn. You can understand exactly how many patients will be waiting and when. And then discover options for addressing the number of patients waiting for specific services with your existing staff mix. If you’d like specific recommendations for your local site or team based on these dynamics, request a Modeling to Learn consult or use the guides and videos at mtl.how. Clinical oscillations can really whipsaw clinicians and patients around, which is stressful and can increase potential clinical risks. If the number of patients waiting to start grows for a long time, it’s overwhelming. But it can also be exhausting when things go up and down quickly and clinics feel like they’re chasing their tails trying to find improvements. Especially given these dynamics can vary service by service, even within the same team. If you want to understand this principle, go to mtl.how/sim and check out the Team Care module. Or are you interested in another common balancing care problem, how the appointment backlog extends the weeks between visits? Watch that video to find out.
+What if we keep making the same care decisions? Will things get better or worse? This is almost a trick question. If we keep making the same care decisions, more than likely things will stay the same in the clinic. Very rarely do clinical teams not have any local improvements that they would like to make. So, what do we mean by staying the same? We know that things in the clinic are not static but are always dynamic and always changing. We’ll describe how start delays affect treatment decisions to explain what we mean. The Modeling to Learn Simulation User Interface at mtl.how/sim is designed to help a team see what is likely to happen over the next two years if they continue to make the same daily clinical decisions that were made over the past two years. We call this the base case of no new decisions. Any number of feasible alternative decisions a team or site could make can be compared against this base case to find the best options for improving care over time. Because feedback dynamics produce non-linear trends or system behaviors, this means even the base case run can be surprising for some teams. The base case may show even more undesirable likely futures which teams would need to make new decisions to avoid. An example is the way start delays affect treatment decisions through balancing feedback. As the number of patients waiting to start a specific service grows, clinicians must adjust and try to find another way to meet their Veterans’ needs. Over time, as the clinicians learn about a local delay, it starts to affect what they think the most ethical, clinically appropriate treatment for the Veteran will be. An analogy from daily life will help. If there’s a traffic jam outside your VA, you may try to find another route home. As more drivers choose alternative routes, traffic returns to normal. Of course, when drivers don’t know about the jam, it will get worse over time. And whether drivers choose alternative routes or not—with balancing feedback like a traffic jam, where more cars are getting on the highway than are getting off—cars will build up and slow down the rate of traffic for everyone, whether we like it or not. We have these kinds of experiences in our clinics, too. Balancing feedback can be a blessing or curse. A blessing in that the balancing feedback causes the clinical system to return to its status quo. A curse if the status quo is undesirable for quality of patient care or provider quality of work life. When you start Modeling to Learn, it is surprising to see how powerful feedback is in driving care dynamics in any clinical care setting—VA or non-VA, VAMC or CBOC, urban and rural, BHIP, or PCT. General system insights, like the link between balancing feedbacks and nonlinear system behaviors, can be understood quickly from Modeling to Learn. You can understand exactly how many patients will be waiting and when. And then discover options for addressing the number of patients waiting for specific services with your existing staff mix. If you’d like specific recommendations for your local site or team based on these dynamics, request a Modeling to Learn consult or use the guides and videos at mtl.how. Clinical oscillations can really whipsaw clinicians and patients around, which is stressful and can increase potential clinical risks. If the number of patients waiting to start grows for a long time, it’s overwhelming. But it can also be exhausting when things go up and down quickly and clinics feel like they’re chasing their tails trying to find improvements. Especially given these dynamics can vary service by service, even within the same team. If you want to understand this principle, go to mtl.how/sim and check out the Team Care module. Or are you interested in another common balancing care problem, how the appointment backlog extends the weeks between visits? Watch that video to find out.
Hi, I’m Lindsey and this is Debbie. Leaders and clinicians often ask how can Modeling to Learn be useful when facing critical staffing and hiring needs? With limited staff coverage, the need to hire is priority number one, but even then there isn’t a magic wand that produces staff where they may not exist today. What could help now? From the beginning, Modeling to Learn prioritized evidence-based episodes of care within existing staff time. When Modeling to Learn launched nationally in the VA in March of 2020, it was to empower staff to find the highest yield local improvements without asking staff to do more with less. As a result, when staff coverage may not change quickly, Modeling to Learn will find options for improving quality of care for Veterans and quality of work life for providers. How? What does Debbie mean by high yield local improvements? Because evidence-based behavioral healthcare is delivered over time, there are many possibilities. Small decisions made all day, every day by the clinicians, when compounded over time, can be surprisingly powerful. Think of your savings account or your waistline. In Modeling to Learn, we look for the lightest clinical lift teams and VAs can make that have the biggest payoff for Veterans in terms of timely, high-quality care. We work hard to avoid big difficult changes with limited benefit. When working with the Modeling to Learn-read data user interface, clinical teams are often motivated when they see trends that reflect hard-won efforts to implement high quality episodes of care which may not show up in other data systems for some time. The Modeling to Learn Blue simulation user interface saves staff time because alternatives can quickly be assessed during a modeling consultation. Change is hard, but we no longer have to learn by trial and error, wearing out already burdened staff. Does that sound too good to be true? If so, you may be wondering about examples of Modeling to Learn use cases for the pain points you face in your specific team or program. As an example, how does Modeling to Learn benefit substance use disorder, or SUD programs? Watch that video to find out.
+Leaders and clinicians often ask how can Modeling to Learn be useful when facing critical staffing and hiring needs? With limited staff coverage, the need to hire is priority number one, but even then there isn’t a magic wand that produces staff where they may not exist today. What could help now? From the beginning, Modeling to Learn prioritized evidence-based episodes of care within existing staff time. When Modeling to Learn launched nationally in the VA in March of 2020, it was to empower staff to find the highest yield local improvements without asking staff to do more with less. As a result, when staff coverage may not change quickly, Modeling to Learn will find options for improving quality of care for Veterans and quality of work life for providers. How? What does Debbie mean by high yield local improvements? Because evidence-based behavioral healthcare is delivered over time, there are many possibilities. Small decisions made all day, every day by the clinicians, when compounded over time, can be surprisingly powerful. Think of your savings account or your waistline. In Modeling to Learn, we look for the lightest clinical lift teams and VAs can make that have the biggest payoff for Veterans in terms of timely, high-quality care. We work hard to avoid big difficult changes with limited benefit. When working with the Modeling to Learn-read data user interface, clinical teams are often motivated when they see trends that reflect hard-won efforts to implement high quality episodes of care which may not show up in other data systems for some time. The Modeling to Learn Blue simulation user interface saves staff time because alternatives can quickly be assessed during a modeling consultation. Change is hard, but we no longer have to learn by trial and error, wearing out already burdened staff. Does that sound too good to be true? If so, you may be wondering about examples of Modeling to Learn use cases for the pain points you face in your specific team or program. As an example, how does Modeling to Learn benefit substance use disorder, or SUD programs? Watch that video to find out.
Hi, I’m Lindsey and this is Debbie. Why is Modeling to Learn Red useful and how does the Modeling to Learn Data User Interface provide new insights? The primary value of MTL Red is its power to efficiently query the VA Corporate Data Warehouse directly, and we’ve come a long way over the years. Yes we have. When we first began, we were using an Excel workbook so that frontline teams could carefully review the clinic selections that define their Modeling to Learn team data sets. Fast forward to the present and now teams have real time data available to them from within the VA domain from any computer with PIV badge access. Since the Data UI includes PHI, if you go to mtl.how/data, you will be able to see the same data you have permissions to access in the electronic health record. But given that clinicians, managers, data leads, quality improvement staff, evaluators … well, basically everyone is so busy, no one has time to review another data dashboard unless it offers something of really high value that distinguishes it from other resources. When you request a Modeling to Learn Red consultation, we work with your team or VA to explore care data that can address your locally identified priority. Clinics or scheduling grids are often changing. For that reason, we wanted a user interface where clinic selections can include active or inactive clinics over the last two years. That way, the team can filter the information to find the most appropriate clinics to include in their data set to gain new insights. And in Team Flow, clinic selections can be used to evaluate transitions between an episode of care in one team and the start of another episode of care in a higher or lower intensity care setting. Modeling to Learn Red also enables zooming in to check on the care of an individual patient at the start of the clinical day or during case reviews at a team meeting. But with the MTL Red Data User Interface, you can also zoom out to view teen care trends, bringing patient-level care coordination and trend-level process improvement decisions together. And that’s where things start to get interesting. Based on the clinic selections, the next set of tabs to find local data values for common care problems, including care coordination, psychotherapy, medication management, team care, and team flow. Each tab features simple definitions of how data were estimated for the common care problem. Detailed definitions with technical specifications are also provided to allow valid comparison of these data to other VA dashboards. That said, a focus on data details could be frustrating and add limited value. Modeling to Learn emphasizes understanding system problems in care flow over time. These care flow problems can be defined accurately with just five key time-based variables that drive care quality. How do five key variables drive care quality? Watch that video to find out.
+Why is Modeling to Learn Red useful and how does the Modeling to Learn Data User Interface provide new insights? The primary value of MTL Red is its power to efficiently query the VA Corporate Data Warehouse directly, and we’ve come a long way over the years. Yes we have. When we first began, we were using an Excel workbook so that frontline teams could carefully review the clinic selections that define their Modeling to Learn team data sets. Fast forward to the present and now teams have real time data available to them from within the VA domain from any computer with PIV badge access. Since the Data UI includes PHI, if you go to mtl.how/data, you will be able to see the same data you have permissions to access in the electronic health record. But given that clinicians, managers, data leads, quality improvement staff, evaluators … well, basically everyone is so busy, no one has time to review another data dashboard unless it offers something of really high value that distinguishes it from other resources. When you request a Modeling to Learn Red consultation, we work with your team or VA to explore care data that can address your locally identified priority. Clinics or scheduling grids are often changing. For that reason, we wanted a user interface where clinic selections can include active or inactive clinics over the last two years. That way, the team can filter the information to find the most appropriate clinics to include in their data set to gain new insights. And in Team Flow, clinic selections can be used to evaluate transitions between an episode of care in one team and the start of another episode of care in a higher or lower intensity care setting. Modeling to Learn Red also enables zooming in to check on the care of an individual patient at the start of the clinical day or during case reviews at a team meeting. But with the MTL Red Data User Interface, you can also zoom out to view teen care trends, bringing patient-level care coordination and trend-level process improvement decisions together. And that’s where things start to get interesting. Based on the clinic selections, the next set of tabs to find local data values for common care problems, including care coordination, psychotherapy, medication management, team care, and team flow. Each tab features simple definitions of how data were estimated for the common care problem. Detailed definitions with technical specifications are also provided to allow valid comparison of these data to other VA dashboards. That said, a focus on data details could be frustrating and add limited value. Modeling to Learn emphasizes understanding system problems in care flow over time. These care flow problems can be defined accurately with just five key time-based variables that drive care quality. How do five key variables drive care quality? Watch that video to find out.
Hi, I’m Lindsey and this is Debbie. Why is Modeling to Learn Blue useful? Well, why is it wisest to focus on the dynamics of care over time? The short answer is that clinical and improvement teams cannot adjust one part of the care equation without everything else changing. In Modeling to Learn Blue, we zoom out to see how care variables are locked in relationship with one another over time. The key variables that define either a poor quality or high quality episode of care must be understood together. Building from MTL Red, you can export your local data set created in the Modeling to Learn Data User Interface at mtl.how/data. Then, if you navigate to mtl.how/sim, you can find the Modeling to Learn Blue Simulation User Interface, which is a dynamic and interactive way to understand why problems with care coordination, medication management, psychotherapy, team care, and team flow persist over time. The Simulation User Interface is a way to see how adjustments in one part of an episode of care explain subsequent impacts in the care system. For any of the common care problems, the simulation saves you time and energy by accounting for the local new patient start rate in patients per week and the local appointment supply and appointments per week. The simulation also keeps track of the local new patient wait time in weeks, time between visits and weeks, and the engagement duration over time, again in weeks. All are calculated for you automatically in the Data User Interface, but their interdependence is accounted for in the Simulation User Interface. The Modeling to Learn Blue Simulation User Interface empowers teams to avoid ineffective strategies because you very quickly learn to develop new insights that would be inefficient, if not impossible, to figure out in your head or by hand. Learning from simulation is designed to help upgrade local decision-making. Teams develop new rules of thumb and insights in which the dependent dynamics among these variables that define care are all taken into account. With a Modeling to Learn consult, we come alongside with partners mid stride in their daily clinical activities who may have limited insight into what is likely to happen over the near future if they keep making the same decisions every day. With Modeling to Learn Blue simulation learning, sites and teams can safely see the impact of new decisions while building new capacities for systems thinking. Why is applied systems thinking more likely to help us avoid costly mistakes? Watch that video to find out.
+Why is Modeling to Learn Blue useful? Well, why is it wisest to focus on the dynamics of care over time? The short answer is that clinical and improvement teams cannot adjust one part of the care equation without everything else changing. In Modeling to Learn Blue, we zoom out to see how care variables are locked in relationship with one another over time. The key variables that define either a poor quality or high quality episode of care must be understood together. Building from MTL Red, you can export your local data set created in the Modeling to Learn Data User Interface at mtl.how/data. Then, if you navigate to mtl.how/sim, you can find the Modeling to Learn Blue Simulation User Interface, which is a dynamic and interactive way to understand why problems with care coordination, medication management, psychotherapy, team care, and team flow persist over time. The Simulation User Interface is a way to see how adjustments in one part of an episode of care explain subsequent impacts in the care system. For any of the common care problems, the simulation saves you time and energy by accounting for the local new patient start rate in patients per week and the local appointment supply and appointments per week. The simulation also keeps track of the local new patient wait time in weeks, time between visits and weeks, and the engagement duration over time, again in weeks. All are calculated for you automatically in the Data User Interface, but their interdependence is accounted for in the Simulation User Interface. The Modeling to Learn Blue Simulation User Interface empowers teams to avoid ineffective strategies because you very quickly learn to develop new insights that would be inefficient, if not impossible, to figure out in your head or by hand. Learning from simulation is designed to help upgrade local decision-making. Teams develop new rules of thumb and insights in which the dependent dynamics among these variables that define care are all taken into account. With a Modeling to Learn consult, we come alongside with partners mid stride in their daily clinical activities who may have limited insight into what is likely to happen over the near future if they keep making the same decisions every day. With Modeling to Learn Blue simulation learning, sites and teams can safely see the impact of new decisions while building new capacities for systems thinking. Why is applied systems thinking more likely to help us avoid costly mistakes? Watch that video to find out.
Hi, I’m Lindsey and this is David. Why is applied systems thinking more likely to help us avoid costly mistakes? Unless we understand interdependent system effects over time, we’re very likely to choose ineffective strategies for our clinical work. And when chronic impairment, relapse, suicide, and overdose are the critical concerns for our patients, we need system insights about how to best organize our clinical practice. Applied systems thinking includes understanding complexity, feedback, and system behaviors over time. By complexity, we mean understanding the relationship or the interaction among two or more variables, such as wait times and the improvement rate. By feedback, we mean moving beyond simple cause and effect events to consider how some effects are reinforced and get stronger over time and other effects are reduced over time. To explain feedback using David’s example, it’s not just that if you have long wait times then the patient improvement rate will go down because patients aren’t starting care. It’s also that as the improvement rate goes up, then at a system level wait times will go down as Veterans get their needs met and graduate from care. Our systems are perpetually causing themselves over time through feedback. Causal feedbacks explain trends over time, called system behaviors. Given that feedback can be a reinforcing effect or a balancing effect over time, Modeling to Learn provides better insights about short- and long-term understanding of change over time. This can include decisions that may make things worse before they get better, or may make things better before they get worse. For example, longer wait times for new patients may reduce their improvement rate in the short term, but if this enables higher quality care engagement for them and existing patients over time, then more Veterans can get better and graduate from care and wait times will go down for the entire community of patients. It is through improving our systems thinking that Modeling to Learn really helps us move beyond making costly mistakes. If you’d like to learn more about how Modeling to Learn helps us improve the dynamics driving where Veterans get stuck or drop out of care, watch that video to find out.
+Why is applied systems thinking more likely to help us avoid costly mistakes? Unless we understand interdependent system effects over time, we’re very likely to choose ineffective strategies for our clinical work. And when chronic impairment, relapse, suicide, and overdose are the critical concerns for our patients, we need system insights about how to best organize our clinical practice. Applied systems thinking includes understanding complexity, feedback, and system behaviors over time. By complexity, we mean understanding the relationship or the interaction among two or more variables, such as wait times and the improvement rate. By feedback, we mean moving beyond simple cause and effect events to consider how some effects are reinforced and get stronger over time and other effects are reduced over time. To explain feedback using David’s example, it’s not just that if you have long wait times then the patient improvement rate will go down because patients aren’t starting care. It’s also that as the improvement rate goes up, then at a system level wait times will go down as Veterans get their needs met and graduate from care. Our systems are perpetually causing themselves over time through feedback. Causal feedbacks explain trends over time, called system behaviors. Given that feedback can be a reinforcing effect or a balancing effect over time, Modeling to Learn provides better insights about short- and long-term understanding of change over time. This can include decisions that may make things worse before they get better, or may make things better before they get worse. For example, longer wait times for new patients may reduce their improvement rate in the short term, but if this enables higher quality care engagement for them and existing patients over time, then more Veterans can get better and graduate from care and wait times will go down for the entire community of patients. It is through improving our systems thinking that Modeling to Learn really helps us move beyond making costly mistakes. If you’d like to learn more about how Modeling to Learn helps us improve the dynamics driving where Veterans get stuck or drop out of care, watch that video to find out.
Hi, I’m Lindsey and this is Debbie. Why is Modeling to Learn able to provide new insights about where Veterans get stuck or drop out of care? Because the Modeling to Learn Simulation User Interface depicts the states of care as stocks based on the flows in and out of that state. Take for example, patients waiting to start. We depict patients waiting to start as a stock, a rectangle with an accumulated volume indicator. The number of Veterans waiting to start in a typical week is determined by the difference between the number of Veterans that flow into that state and the number of Veterans who flow out of that state each week. The rate dials depict the flows in and out of stocks, and the primary rates in Modeling to Learn are patients per week, appointments per week, and episodes of care per week. The initial values for any team or site depicted in the Simulation User Interface are based on the last two years of data exported from the Data User Interface. The important idea is that in any dynamic system, things get to be the way they are over time. Things either stay the same or improve decision by decision over time. For example, in a clinic or site, the number of Veterans engaged in a given state of care, such as medication management or psychotherapy, reflects decisions that lead to accumulation in that state over time. The dynamic stock and flow diagram provides a way of understanding where Veterans get stuck. As clinicians know, often there is more than one way out of a state of care. With psychotherapy, you may flow from the first session to the second session, but you may drop out at any point in that flow. Why does Modeling to Learn emphasize flows through care over time? Watch that video to find out.
+Why is Modeling to Learn able to provide new insights about where Veterans get stuck or drop out of care? Because the Modeling to Learn Simulation User Interface depicts the states of care as stocks based on the flows in and out of that state. Take for example, patients waiting to start. We depict patients waiting to start as a stock, a rectangle with an accumulated volume indicator. The number of Veterans waiting to start in a typical week is determined by the difference between the number of Veterans that flow into that state and the number of Veterans who flow out of that state each week. The rate dials depict the flows in and out of stocks, and the primary rates in Modeling to Learn are patients per week, appointments per week, and episodes of care per week. The initial values for any team or site depicted in the Simulation User Interface are based on the last two years of data exported from the Data User Interface. The important idea is that in any dynamic system, things get to be the way they are over time. Things either stay the same or improve decision by decision over time. For example, in a clinic or site, the number of Veterans engaged in a given state of care, such as medication management or psychotherapy, reflects decisions that lead to accumulation in that state over time. The dynamic stock and flow diagram provides a way of understanding where Veterans get stuck. As clinicians know, often there is more than one way out of a state of care. With psychotherapy, you may flow from the first session to the second session, but you may drop out at any point in that flow. Why does Modeling to Learn emphasize flows through care over time? Watch that video to find out.
Hi, I’m Lindsey, and this is David. Why does Modeling to Learn emphasize flows through care over time? Our goal is to ensure Veterans flow through our addiction and mental health care services to recovery. A basic idea is that care occurs over time. Flows are powerful because they express a rate of change, which means they give us key information about where we will be in the future. Imagine your next trip in the car. We need the miles per hour to know when we will arrive at our destination. But knowing the number of miles to our destination is not enough. We must also know how fast we’re traveling to know when we will get there. In other words, miles per hour. The same is true in our clinical care. It’s not enough to know how many new medication management appointments we have on the books today. We need to know how quickly those appointments were added to the books or the booking rate, in appointments per week, as well as how quickly those appointments are completed or the completing rate, in appointments per week. When the inflow is greater than the outflow, the level of the appointment or patient stock will rise. When the outflow is greater than the inflow, then the level of the appointment or patient stock will fall. Opening the Modeling to Learn Simulation User Interface at mtl.how/sim, you can see where all the Veterans are accumulating in stocks. You can understand the inflows and outflows for each stock and which accumulations are likely to get better or worse over time. Many clinical teams talk about feeling inundated by the addiction and mental health needs of the communities that rely on them. They can feel like they’re drowning, providing the volume of care needed, especially when short staffed. We understand this kind of flooding from the natural world. To stay safe, it’s critical to know how fast the waters are rising and where. We understand this by understanding flows. You may relate to this feeling of being overwhelmed and want to know more about how Modeling to Learn can help. For example, how does Modeling to Learn help improve medication management? Watch that video to find out.
+Why does Modeling to Learn emphasize flows through care over time? Our goal is to ensure Veterans flow through our addiction and mental health care services to recovery. A basic idea is that care occurs over time. Flows are powerful because they express a rate of change, which means they give us key information about where we will be in the future. Imagine your next trip in the car. We need the miles per hour to know when we will arrive at our destination. But knowing the number of miles to our destination is not enough. We must also know how fast we’re traveling to know when we will get there. In other words, miles per hour. The same is true in our clinical care. It’s not enough to know how many new medication management appointments we have on the books today. We need to know how quickly those appointments were added to the books or the booking rate, in appointments per week, as well as how quickly those appointments are completed or the completing rate, in appointments per week. When the inflow is greater than the outflow, the level of the appointment or patient stock will rise. When the outflow is greater than the inflow, then the level of the appointment or patient stock will fall. Opening the Modeling to Learn Simulation User Interface at mtl.how/sim, you can see where all the Veterans are accumulating in stocks. You can understand the inflows and outflows for each stock and which accumulations are likely to get better or worse over time. Many clinical teams talk about feeling inundated by the addiction and mental health needs of the communities that rely on them. They can feel like they’re drowning, providing the volume of care needed, especially when short staffed. We understand this kind of flooding from the natural world. To stay safe, it’s critical to know how fast the waters are rising and where. We understand this by understanding flows. You may relate to this feeling of being overwhelmed and want to know more about how Modeling to Learn can help. For example, how does Modeling to Learn help improve medication management? Watch that video to find out.
](https://bcove.video/465KOIt)
-Hi, I'm Lindsey and this is Debbie. You might wonder _What is _Modeling to Learn_, why use it, and how might it help?_ We often don't understand what is driving our care problems over time. _Modeling to Learn_ helps to upgrade the decisions you make all day, every day, because the decisions made most frequently will be the most powerful for improving our care. We're introducing _Modeling to Learn_, which has been developed and evaluated in the VA for over 9 years. _Modeling to Learn_ is a way to upgrade care decisions by understanding the dynamics of common care problems over time. Are you very, very busy? _Modeling to Learn_ includes step by step guides with gifs showing each click. These guides are available at [mtl.how](https://github.com/lzim/mtl). Are you unsure what data to use, how it's defined, and are you wishing it was current enough to guide your care decisions? _Modeling to Learn_ includes a transparent, real-time locally defined data user interface available at [mtl.how/data](https://app.powerbigov.us/groups/me/apps/b9686a29-6857-46c9-bdf9-043ca2b29138/reports/05dd8dbd-313f-4993-b406-6feea2fdb060/ReportSection?ctid=e95f1b23-abaf-45ee-821d-b7ab251ab3bf), where you specify the clinics that define your data views. Are you unsure what is realistic for improving care locally, given your patients’ needs and your existing staff and resources? _Modeling to Learn_ also includes a simulation modeling interface available at [mtl.how/sim](https://forio.com/app/va/va-psd-sim/login.html), so you can try out new decisions before you implement them in the clinic. We all want to meet Veterans needs for timely, high-quality care. So, what gets in our way? Watch that video to find out.
+You might wonder _What is _Modeling to Learn_, why use it, and how might it help?_ We often don't understand what is driving our care problems over time. _Modeling to Learn_ helps to upgrade the decisions you make all day, every day, because the decisions made most frequently will be the most powerful for improving our care. We're introducing _Modeling to Learn_, which has been developed and evaluated in the VA for over 9 years. _Modeling to Learn_ is a way to upgrade care decisions by understanding the dynamics of common care problems over time. Are you very, very busy? _Modeling to Learn_ includes step by step guides with gifs showing each click. These guides are available at [mtl.how](https://github.com/lzim/mtl). Are you unsure what data to use, how it's defined, and are you wishing it was current enough to guide your care decisions? _Modeling to Learn_ includes a transparent, real-time locally defined data user interface available at [mtl.how/data](https://app.powerbigov.us/groups/me/apps/b9686a29-6857-46c9-bdf9-043ca2b29138/reports/05dd8dbd-313f-4993-b406-6feea2fdb060/ReportSection?ctid=e95f1b23-abaf-45ee-821d-b7ab251ab3bf), where you specify the clinics that define your data views. Are you unsure what is realistic for improving care locally, given your patients’ needs and your existing staff and resources? _Modeling to Learn_ also includes a simulation modeling interface available at [mtl.how/sim](https://forio.com/app/va/va-psd-sim/login.html), so you can try out new decisions before you implement them in the clinic. We all want to meet Veterans needs for timely, high-quality care. So, what gets in our way? Watch that video to find out.
## How can a _Modeling to Learn_ consult help?
@@ -25,7 +25,7 @@ Hi, I'm Lindsey and this is Debbie. You might wonder _What is _Modeling to Learn
[](https://bcove.video/3WuJ7kI)
-Hi, I'm Lindsey and this is David. _Modeling to Learn_ improves visibility and provides new insights into how common care problems persist over time. How? _Modeling to Learn_ is based on over 60 years of scholarship known as participatory system dynamics. For this reason, we call ourselves Team PSD for Team Participatory System Dynamics. The Data User Interface and Simulation User Interface comprise two versions of _Modeling to Learn_. The data-only version is known as _Modeling to Learn Red_; _Modeling to Learn Blue_ ads participatory learning from simulation. Team PSD supports _MTL Red_ and _MTL Blue_ and is carefully evaluating how each works to support VA in meeting Veterans needs. Learning from simulation can help us to place a better initial bet on what is likely to work locally by evaluating alternative decisions via simulation before we implement them in the real world. _MTL Red_ tells us where we've been over the last two years based on the clinic selections made to produce the patient data reports and team trends or visualizations. Many staff report that viewing the Data UI real-time patient data tabs or the team trends is efficient and encouraging. The data tabs help with clinical decision making. The visualizations of team trends provide leading indicators that improvement efforts are paying off, which can be validating. Why is _Modeling to Learn_ useful when we have critical staffing and hiring needs? Watch that video to find out.
+_Modeling to Learn_ improves visibility and provides new insights into how common care problems persist over time. How? _Modeling to Learn_ is based on over 60 years of scholarship known as participatory system dynamics. For this reason, we call ourselves Team PSD for Team Participatory System Dynamics. The Data User Interface and Simulation User Interface comprise two versions of _Modeling to Learn_. The data-only version is known as _Modeling to Learn Red_; _Modeling to Learn Blue_ ads participatory learning from simulation. Team PSD supports _MTL Red_ and _MTL Blue_ and is carefully evaluating how each works to support VA in meeting Veterans needs. Learning from simulation can help us to place a better initial bet on what is likely to work locally by evaluating alternative decisions via simulation before we implement them in the real world. _MTL Red_ tells us where we've been over the last two years based on the clinic selections made to produce the patient data reports and team trends or visualizations. Many staff report that viewing the Data UI real-time patient data tabs or the team trends is efficient and encouraging. The data tabs help with clinical decision making. The visualizations of team trends provide leading indicators that improvement efforts are paying off, which can be validating. Why is _Modeling to Learn_ useful when we have critical staffing and hiring needs? Watch that video to find out.
## What gets in the way of meeting patients' needs?
@@ -33,7 +33,7 @@ Hi, I'm Lindsey and this is David. _Modeling to Learn_ improves visibility and p
[](https://bcove.video/4d4TmRZ)
-Hi, I'm Lindsey and this is David. We all want to meet Veterans’ needs for timely, high-quality care. So, what gets in our way? _Modeling to Learn_ is a way to upgrade care decisions by understanding the dynamics of common care problems over time. Demanding clinical days mean we don't often have bandwidth to gain new insights, particularly in our understanding of interdependent clinic dynamics over time. We're often flying blind to the impacts that our daily decisions have on the overall community that relies on us. As clinicians, we look at the patient in front of us and decide when we think they should be seen again. But have you ever been told you cannot see a patient as soon as you would like due to the constraints of the clinic? For example, an evidence-based course of care requires starting treatment without delay and keeping the Veteran engaged in a therapeutic dose of care over time to meet their needs. If we only emphasize Veterans starting care, but the time between visits extends way out, this interferes with evidence-based care and Veterans getting better. How can we get more Veterans better without adding hours in our day? Watch that video to find out.
+We all want to meet Veterans’ needs for timely, high-quality care. So, what gets in our way? _Modeling to Learn_ is a way to upgrade care decisions by understanding the dynamics of common care problems over time. Demanding clinical days mean we don't often have bandwidth to gain new insights, particularly in our understanding of interdependent clinic dynamics over time. We're often flying blind to the impacts that our daily decisions have on the overall community that relies on us. As clinicians, we look at the patient in front of us and decide when we think they should be seen again. But have you ever been told you cannot see a patient as soon as you would like due to the constraints of the clinic? For example, an evidence-based course of care requires starting treatment without delay and keeping the Veteran engaged in a therapeutic dose of care over time to meet their needs. If we only emphasize Veterans starting care, but the time between visits extends way out, this interferes with evidence-based care and Veterans getting better. How can we get more Veterans better without adding hours in our day? Watch that video to find out.
## If we keep making the same care decisions, will things get better or worse?
@@ -41,7 +41,7 @@ Hi, I'm Lindsey and this is David. We all want to meet Veterans’ needs for tim
[](https://bcove.video/4cZpLJk)
-Hi, I'm Lindsey and this is Debbie. What if we keep making the same care decisions? Will things get better or worse? This is almost a trick question. If we keep making the same care decisions, more than likely things will stay the same in the clinic. Very rarely do clinical teams _not_ have any local improvements that they would like to make. So, what do we mean by staying the same? We know that things in the clinic are not static but are always dynamic and always changing. We'll describe how start delays affect treatment decisions to explain what we mean. The _Modeling to Learn_ Simulation User Interface at [mtl.how/sim](https://forio.com/app/va/va-psd-sim/login.html) is designed to help a team see what is likely to happen over the next two years if they continue to make the same daily clinical decisions that were made over the past two years. We call this the base case of no new decisions. Any number of feasible alternative decisions a team or site could make can be compared against this base case to find the best options for improving care over time. Because feedback dynamics produce non-linear trends or system behaviors, this means even the base case run can be surprising for some teams. The base case may show even more undesirable likely futures which teams would need to make new decisions to avoid. An example is the way start delays affect treatment decisions through balancing feedback. As the number of patients waiting to start a specific service grows, clinicians must adjust and try to find another way to meet their Veterans’ needs. Over time, as the clinicians learn about a local delay, it starts to affect what they think the most ethical, clinically appropriate treatment for the Veteran will be. An analogy from daily life will help. If there's a traffic jam outside your VA, you may try to find another route home. As more drivers choose alternative routes, traffic returns to normal. Of course, when drivers don't know about the jam, it will get worse over time. And whether drivers choose alternative routes or not—with balancing feedback like a traffic jam, where more cars are getting on the highway than are getting off—cars will build up and slow down the rate of traffic for everyone, whether we like it or not. We have these kinds of experiences in our clinics, too. Balancing feedback can be a blessing or curse. A blessing in that the balancing feedback causes the clinical system to return to its status quo. A curse if the status quo is undesirable for quality of patient care or provider quality of work life. When you start _Modeling to Learn_, it is surprising to see how powerful feedback is in driving care dynamics in any clinical care setting—VA or non-VA, VAMC or CBOC, urban and rural, BHIP, or PCT. General system insights, like the link between balancing feedbacks and nonlinear system behaviors, can be understood quickly from _Modeling to Learn_. You can understand _exactly_ how many patients will be waiting and when. And then discover options for addressing the number of patients waiting for specific services with your existing staff mix. If you'd like specific recommendations for your local site or team based on these dynamics, request a _Modeling to Learn_ consult or use the guides and videos at [mtl.how](https://github.com/lzim/mtl). Clinical oscillations can really whipsaw clinicians and patients around, which is stressful and can increase potential clinical risks. If the number of patients waiting to start grows for a long time, it's overwhelming. But it can also be exhausting when things go up and down quickly and clinics feel like they're chasing their tails trying to find improvements. Especially given these dynamics can vary service by service, even within the same team. If you want to understand this principle, go to [mtl.how/sim](https://forio.com/app/va/va-psd-sim/login.html) and check out the Team Care module. Or are you interested in another common balancing care problem, how the appointment backlog extends the weeks between visits? Watch that video to find out.
+What if we keep making the same care decisions? Will things get better or worse? This is almost a trick question. If we keep making the same care decisions, more than likely things will stay the same in the clinic. Very rarely do clinical teams _not_ have any local improvements that they would like to make. So, what do we mean by staying the same? We know that things in the clinic are not static but are always dynamic and always changing. We'll describe how start delays affect treatment decisions to explain what we mean. The _Modeling to Learn_ Simulation User Interface at [mtl.how/sim](https://forio.com/app/va/va-psd-sim/login.html) is designed to help a team see what is likely to happen over the next two years if they continue to make the same daily clinical decisions that were made over the past two years. We call this the base case of no new decisions. Any number of feasible alternative decisions a team or site could make can be compared against this base case to find the best options for improving care over time. Because feedback dynamics produce non-linear trends or system behaviors, this means even the base case run can be surprising for some teams. The base case may show even more undesirable likely futures which teams would need to make new decisions to avoid. An example is the way start delays affect treatment decisions through balancing feedback. As the number of patients waiting to start a specific service grows, clinicians must adjust and try to find another way to meet their Veterans’ needs. Over time, as the clinicians learn about a local delay, it starts to affect what they think the most ethical, clinically appropriate treatment for the Veteran will be. An analogy from daily life will help. If there's a traffic jam outside your VA, you may try to find another route home. As more drivers choose alternative routes, traffic returns to normal. Of course, when drivers don't know about the jam, it will get worse over time. And whether drivers choose alternative routes or not—with balancing feedback like a traffic jam, where more cars are getting on the highway than are getting off—cars will build up and slow down the rate of traffic for everyone, whether we like it or not. We have these kinds of experiences in our clinics, too. Balancing feedback can be a blessing or curse. A blessing in that the balancing feedback causes the clinical system to return to its status quo. A curse if the status quo is undesirable for quality of patient care or provider quality of work life. When you start _Modeling to Learn_, it is surprising to see how powerful feedback is in driving care dynamics in any clinical care setting—VA or non-VA, VAMC or CBOC, urban and rural, BHIP, or PCT. General system insights, like the link between balancing feedbacks and nonlinear system behaviors, can be understood quickly from _Modeling to Learn_. You can understand _exactly_ how many patients will be waiting and when. And then discover options for addressing the number of patients waiting for specific services with your existing staff mix. If you'd like specific recommendations for your local site or team based on these dynamics, request a _Modeling to Learn_ consult or use the guides and videos at [mtl.how](https://github.com/lzim/mtl). Clinical oscillations can really whipsaw clinicians and patients around, which is stressful and can increase potential clinical risks. If the number of patients waiting to start grows for a long time, it's overwhelming. But it can also be exhausting when things go up and down quickly and clinics feel like they're chasing their tails trying to find improvements. Especially given these dynamics can vary service by service, even within the same team. If you want to understand this principle, go to [mtl.how/sim](https://forio.com/app/va/va-psd-sim/login.html) and check out the Team Care module. Or are you interested in another common balancing care problem, how the appointment backlog extends the weeks between visits? Watch that video to find out.
## Why is _Modeling to Learn_ useful when we have critical staffing and hiring needs?
@@ -49,7 +49,7 @@ Hi, I'm Lindsey and this is Debbie. What if we keep making the same care decisio
[](https://bcove.video/4f7OfSL)
-Hi, I'm Lindsey and this is Debbie. Leaders and clinicians often ask how can _Modeling to Learn_ be useful when facing critical staffing and hiring needs? With limited staff coverage, the need to hire is priority number one, but even then there isn't a magic wand that produces staff where they may not exist today. What could help now? From the beginning, _Modeling to Learn_ prioritized evidence-based episodes of care within existing staff time. When _Modeling to Learn_ launched nationally in the VA in March of 2020, it was to empower staff to find the highest yield local improvements without asking staff to do more with less. As a result, when staff coverage may not change quickly, _Modeling to Learn_ will find options for improving quality of care for Veterans and quality of work life for providers. How? What does Debbie mean by high yield local improvements? Because evidence-based behavioral healthcare is delivered over time, there are many possibilities. Small decisions made all day, every day by the clinicians, when compounded over time, can be surprisingly powerful. Think of your savings account or your waistline. In _Modeling to Learn_, we look for the lightest clinical lift teams and VAs can make that have the biggest payoff for Veterans in terms of timely, high-quality care. We work hard to avoid big difficult changes with limited benefit. When working with the _Modeling to Learn_-read data user interface, clinical teams are often motivated when they see trends that reflect hard-won efforts to implement high quality episodes of care which may not show up in other data systems for some time. The _Modeling to Learn Blue_ simulation user interface saves staff time because alternatives can quickly be assessed during a modeling consultation. Change is hard, but we no longer have to learn by trial and error, wearing out already burdened staff. Does that sound too good to be true? If so, you may be wondering about examples of _Modeling to Learn_ use cases for the pain points you face in your specific team or program. As an example, how does _Modeling to Learn_ benefit substance use disorder, or SUD programs? Watch that video to find out.
+Leaders and clinicians often ask how can _Modeling to Learn_ be useful when facing critical staffing and hiring needs? With limited staff coverage, the need to hire is priority number one, but even then there isn't a magic wand that produces staff where they may not exist today. What could help now? From the beginning, _Modeling to Learn_ prioritized evidence-based episodes of care within existing staff time. When _Modeling to Learn_ launched nationally in the VA in March of 2020, it was to empower staff to find the highest yield local improvements without asking staff to do more with less. As a result, when staff coverage may not change quickly, _Modeling to Learn_ will find options for improving quality of care for Veterans and quality of work life for providers. How? What does Debbie mean by high yield local improvements? Because evidence-based behavioral healthcare is delivered over time, there are many possibilities. Small decisions made all day, every day by the clinicians, when compounded over time, can be surprisingly powerful. Think of your savings account or your waistline. In _Modeling to Learn_, we look for the lightest clinical lift teams and VAs can make that have the biggest payoff for Veterans in terms of timely, high-quality care. We work hard to avoid big difficult changes with limited benefit. When working with the _Modeling to Learn_-read data user interface, clinical teams are often motivated when they see trends that reflect hard-won efforts to implement high quality episodes of care which may not show up in other data systems for some time. The _Modeling to Learn Blue_ simulation user interface saves staff time because alternatives can quickly be assessed during a modeling consultation. Change is hard, but we no longer have to learn by trial and error, wearing out already burdened staff. Does that sound too good to be true? If so, you may be wondering about examples of _Modeling to Learn_ use cases for the pain points you face in your specific team or program. As an example, how does _Modeling to Learn_ benefit substance use disorder, or SUD programs? Watch that video to find out.
## Why is the _MTL Red_ Data User Interface useful?
@@ -57,7 +57,7 @@ Hi, I'm Lindsey and this is Debbie. Leaders and clinicians often ask how can _Mo
[](https://bcove.video/3A6i7Py)
-Hi, I'm Lindsey and this is Debbie. Why is _Modeling to Learn Red_ useful and how does the _Modeling to Learn_ Data User Interface provide new insights? The primary value of _MTL_ Red is its power to efficiently query the VA Corporate Data Warehouse directly, and we've come a long way over the years. Yes we have. When we first began, we were using an Excel workbook so that frontline teams could carefully review the clinic selections that define their _Modeling to Learn_ team data sets. Fast forward to the present and now teams have real time data available to them from within the VA domain from any computer with PIV badge access. Since the Data UI includes PHI, if you go to [mtl.how/data](https://app.powerbigov.us/groups/me/apps/b9686a29-6857-46c9-bdf9-043ca2b29138/reports/05dd8dbd-313f-4993-b406-6feea2fdb060/ReportSection?ctid=e95f1b23-abaf-45ee-821d-b7ab251ab3bf), you will be able to see the same data you have permissions to access in the electronic health record. But given that clinicians, managers, data leads, quality improvement staff, evaluators … well, basically everyone is so busy, no one has time to review another data dashboard unless it offers something of really high value that distinguishes it from other resources. When you request a _Modeling to Learn Red_ consultation, we work with your team or VA to explore care data that can address your locally identified priority. Clinics or scheduling grids are often changing. For that reason, we wanted a user interface where clinic selections can include active or inactive clinics over the last two years. That way, the team can filter the information to find the most appropriate clinics to include in their data set to gain new insights. And in Team Flow, clinic selections can be used to evaluate transitions between an episode of care in one team and the start of another episode of care in a higher or lower intensity care setting. _Modeling to Learn Red_ also enables zooming in to check on the care of an individual patient at the start of the clinical day or during case reviews at a team meeting. But with the _MTL_ Red Data User Interface, you can also zoom out to view teen care trends, bringing patient-level care coordination and trend-level process improvement decisions together. And that's where things start to get interesting. Based on the clinic selections, the next set of tabs to find local data values for common care problems, including care coordination, psychotherapy, medication management, team care, and team flow. Each tab features simple definitions of how data were estimated for the common care problem. Detailed definitions with technical specifications are also provided to allow valid comparison of these data to other VA dashboards. That said, a focus on data details could be frustrating and add limited value. _Modeling to Learn_ emphasizes understanding system problems in care flow over time. These care flow problems can be defined accurately with just five key time-based variables that drive care quality. How do five key variables drive care quality? Watch that video to find out.
+Why is _Modeling to Learn Red_ useful and how does the _Modeling to Learn_ Data User Interface provide new insights? The primary value of _MTL_ Red is its power to efficiently query the VA Corporate Data Warehouse directly, and we've come a long way over the years. Yes we have. When we first began, we were using an Excel workbook so that frontline teams could carefully review the clinic selections that define their _Modeling to Learn_ team data sets. Fast forward to the present and now teams have real time data available to them from within the VA domain from any computer with PIV badge access. Since the Data UI includes PHI, if you go to [mtl.how/data](https://app.powerbigov.us/groups/me/apps/b9686a29-6857-46c9-bdf9-043ca2b29138/reports/05dd8dbd-313f-4993-b406-6feea2fdb060/ReportSection?ctid=e95f1b23-abaf-45ee-821d-b7ab251ab3bf), you will be able to see the same data you have permissions to access in the electronic health record. But given that clinicians, managers, data leads, quality improvement staff, evaluators … well, basically everyone is so busy, no one has time to review another data dashboard unless it offers something of really high value that distinguishes it from other resources. When you request a _Modeling to Learn Red_ consultation, we work with your team or VA to explore care data that can address your locally identified priority. Clinics or scheduling grids are often changing. For that reason, we wanted a user interface where clinic selections can include active or inactive clinics over the last two years. That way, the team can filter the information to find the most appropriate clinics to include in their data set to gain new insights. And in Team Flow, clinic selections can be used to evaluate transitions between an episode of care in one team and the start of another episode of care in a higher or lower intensity care setting. _Modeling to Learn Red_ also enables zooming in to check on the care of an individual patient at the start of the clinical day or during case reviews at a team meeting. But with the _MTL_ Red Data User Interface, you can also zoom out to view teen care trends, bringing patient-level care coordination and trend-level process improvement decisions together. And that's where things start to get interesting. Based on the clinic selections, the next set of tabs to find local data values for common care problems, including care coordination, psychotherapy, medication management, team care, and team flow. Each tab features simple definitions of how data were estimated for the common care problem. Detailed definitions with technical specifications are also provided to allow valid comparison of these data to other VA dashboards. That said, a focus on data details could be frustrating and add limited value. _Modeling to Learn_ emphasizes understanding system problems in care flow over time. These care flow problems can be defined accurately with just five key time-based variables that drive care quality. How do five key variables drive care quality? Watch that video to find out.
## Why is the _MTL Blue_ Simulation User Interface useful?
@@ -65,7 +65,7 @@ Hi, I'm Lindsey and this is Debbie. Why is _Modeling to Learn Red_ useful and ho
[](https://bcove.video/4fnG5Gb)
-Hi, I'm Lindsey and this is Debbie. Why is _Modeling to Learn Blue_ useful? Well, why is it wisest to focus on the dynamics of care over time? The short answer is that clinical and improvement teams cannot adjust one part of the care equation without everything else changing. In _Modeling to Learn Blue_, we zoom out to see how care variables are locked in relationship with one another over time. The key variables that define either a poor quality or high quality episode of care must be understood together. Building from _MTL_ Red, you can export your local data set created in the _Modeling to Learn_ Data User Interface at [mtl.how/data](https://app.powerbigov.us/groups/me/apps/b9686a29-6857-46c9-bdf9-043ca2b29138?ctid=e95f1b23-abaf-45ee-821d-b7ab251ab3bf). Then, if you navigate to [mtl.how/sim](https://forio.com/app/va/va-psd-sim/login.html), you can find the _Modeling to Learn Blue_ Simulation User Interface, which is a dynamic and interactive way to understand why problems with care coordination, medication management, psychotherapy, team care, and team flow persist over time. The Simulation User Interface is a way to see how adjustments in one part of an episode of care explain subsequent impacts in the care system. For any of the common care problems, the simulation saves you time and energy by accounting for the local new patient start rate in patients per week and the local appointment supply and appointments per week. The simulation also keeps track of the local new patient wait time in weeks, time between visits and weeks, and the engagement duration over time, again in weeks. All are calculated for you automatically in the Data User Interface, but their interdependence is accounted for in the Simulation User Interface. The _Modeling to Learn Blue_ Simulation User Interface empowers teams to avoid ineffective strategies because you very quickly learn to develop new insights that would be inefficient, if not impossible, to figure out in your head or by hand. Learning from simulation is designed to help upgrade local decision-making. Teams develop new rules of thumb and insights in which the dependent dynamics among these variables that define care are all taken into account. With a _Modeling to Learn_ consult, we come alongside with partners mid stride in their daily clinical activities who may have limited insight into what is likely to happen over the near future if they keep making the same decisions every day. With _Modeling to Learn Blue_ simulation learning, sites and teams can safely see the impact of new decisions while building new capacities for systems thinking. Why is applied systems thinking more likely to help us avoid costly mistakes? Watch that video to find out.
+Why is _Modeling to Learn Blue_ useful? Well, why is it wisest to focus on the dynamics of care over time? The short answer is that clinical and improvement teams cannot adjust one part of the care equation without everything else changing. In _Modeling to Learn Blue_, we zoom out to see how care variables are locked in relationship with one another over time. The key variables that define either a poor quality or high quality episode of care must be understood together. Building from _MTL_ Red, you can export your local data set created in the _Modeling to Learn_ Data User Interface at [mtl.how/data](https://app.powerbigov.us/groups/me/apps/b9686a29-6857-46c9-bdf9-043ca2b29138?ctid=e95f1b23-abaf-45ee-821d-b7ab251ab3bf). Then, if you navigate to [mtl.how/sim](https://forio.com/app/va/va-psd-sim/login.html), you can find the _Modeling to Learn Blue_ Simulation User Interface, which is a dynamic and interactive way to understand why problems with care coordination, medication management, psychotherapy, team care, and team flow persist over time. The Simulation User Interface is a way to see how adjustments in one part of an episode of care explain subsequent impacts in the care system. For any of the common care problems, the simulation saves you time and energy by accounting for the local new patient start rate in patients per week and the local appointment supply and appointments per week. The simulation also keeps track of the local new patient wait time in weeks, time between visits and weeks, and the engagement duration over time, again in weeks. All are calculated for you automatically in the Data User Interface, but their interdependence is accounted for in the Simulation User Interface. The _Modeling to Learn Blue_ Simulation User Interface empowers teams to avoid ineffective strategies because you very quickly learn to develop new insights that would be inefficient, if not impossible, to figure out in your head or by hand. Learning from simulation is designed to help upgrade local decision-making. Teams develop new rules of thumb and insights in which the dependent dynamics among these variables that define care are all taken into account. With a _Modeling to Learn_ consult, we come alongside with partners mid stride in their daily clinical activities who may have limited insight into what is likely to happen over the near future if they keep making the same decisions every day. With _Modeling to Learn Blue_ simulation learning, sites and teams can safely see the impact of new decisions while building new capacities for systems thinking. Why is applied systems thinking more likely to help us avoid costly mistakes? Watch that video to find out.
## Why is applied systems thinking more likely to help us avoid costly mistakes?
@@ -73,7 +73,7 @@ Hi, I'm Lindsey and this is Debbie. Why is _Modeling to Learn Blue_ useful? Well
[](https://bcove.video/3AelyUg)
-Hi, I'm Lindsey and this is David. Why is applied systems thinking more likely to help us avoid costly mistakes? Unless we understand interdependent system effects over time, we're very likely to choose ineffective strategies for our clinical work. And when chronic impairment, relapse, suicide, and overdose are the critical concerns for our patients, we need system insights about how to best organize our clinical practice. Applied systems thinking includes understanding complexity, feedback, and system behaviors over time. By complexity, we mean understanding the relationship or the interaction among two or more variables, such as wait times and the improvement rate. By feedback, we mean moving beyond simple cause and effect events to consider how some effects are reinforced and get stronger over time and other effects are reduced over time. To explain feedback using David's example, it's not just that if you have long wait times then the patient improvement rate will go down because patients aren't starting care. It's also that as the improvement rate goes up, then at a system level wait times will go down as Veterans get their needs met and graduate from care. Our systems are perpetually causing themselves over time through feedback. Causal feedbacks explain trends over time, called system behaviors. Given that feedback can be a reinforcing effect or a balancing effect over time, _Modeling to Learn_ provides better insights about short- and long-term understanding of change over time. This can include decisions that may make things worse before they get better, or may make things better before they get worse. For example, longer wait times for new patients may reduce their improvement rate in the short term, but if this enables higher quality care engagement for them and existing patients over time, then more Veterans can get better and graduate from care and wait times will go down for the entire community of patients. It is through improving our systems thinking that _Modeling to Learn_ really helps us move beyond making costly mistakes. If you'd like to learn more about how _Modeling to Learn_ helps us improve the dynamics driving where Veterans get stuck or drop out of care, watch that video to find out.
+Why is applied systems thinking more likely to help us avoid costly mistakes? Unless we understand interdependent system effects over time, we're very likely to choose ineffective strategies for our clinical work. And when chronic impairment, relapse, suicide, and overdose are the critical concerns for our patients, we need system insights about how to best organize our clinical practice. Applied systems thinking includes understanding complexity, feedback, and system behaviors over time. By complexity, we mean understanding the relationship or the interaction among two or more variables, such as wait times and the improvement rate. By feedback, we mean moving beyond simple cause and effect events to consider how some effects are reinforced and get stronger over time and other effects are reduced over time. To explain feedback using David's example, it's not just that if you have long wait times then the patient improvement rate will go down because patients aren't starting care. It's also that as the improvement rate goes up, then at a system level wait times will go down as Veterans get their needs met and graduate from care. Our systems are perpetually causing themselves over time through feedback. Causal feedbacks explain trends over time, called system behaviors. Given that feedback can be a reinforcing effect or a balancing effect over time, _Modeling to Learn_ provides better insights about short- and long-term understanding of change over time. This can include decisions that may make things worse before they get better, or may make things better before they get worse. For example, longer wait times for new patients may reduce their improvement rate in the short term, but if this enables higher quality care engagement for them and existing patients over time, then more Veterans can get better and graduate from care and wait times will go down for the entire community of patients. It is through improving our systems thinking that _Modeling to Learn_ really helps us move beyond making costly mistakes. If you'd like to learn more about how _Modeling to Learn_ helps us improve the dynamics driving where Veterans get stuck or drop out of care, watch that video to find out.
## Why is _Modeling to Learn_ able to provide new insights?
@@ -81,7 +81,7 @@ Hi, I'm Lindsey and this is David. Why is applied systems thinking more likely t
[](https://bcove.video/3XEKDAY)
-Hi, I'm Lindsey and this is Debbie. Why is _Modeling to Learn_ able to provide new insights about where Veterans get stuck or drop out of care? Because the _Modeling to Learn_ Simulation User Interface depicts the states of care as stocks based on the flows in and out of that state. Take for example, patients waiting to start. We depict patients waiting to start as a stock, a rectangle with an accumulated volume indicator. The number of Veterans waiting to start in a typical week is determined by the difference between the number of Veterans that flow into that state and the number of Veterans who flow out of that state each week. The rate dials depict the flows in and out of stocks, and the primary rates in _Modeling to Learn_ are patients per week, appointments per week, and episodes of care per week. The initial values for any team or site depicted in the Simulation User Interface are based on the last two years of data exported from the Data User Interface. The important idea is that in any dynamic system, things get to be the way they are over time. Things either stay the same or improve decision by decision over time. For example, in a clinic or site, the number of Veterans engaged in a given state of care, such as medication management or psychotherapy, reflects decisions that lead to accumulation in that state over time. The dynamic stock and flow diagram provides a way of understanding where Veterans get stuck. As clinicians know, often there is more than one way out of a state of care. With psychotherapy, you may flow from the first session to the second session, but you may drop out at any point in that flow. Why does _Modeling to Learn_ emphasize flows through care over time? Watch that video to find out.
+Why is _Modeling to Learn_ able to provide new insights about where Veterans get stuck or drop out of care? Because the _Modeling to Learn_ Simulation User Interface depicts the states of care as stocks based on the flows in and out of that state. Take for example, patients waiting to start. We depict patients waiting to start as a stock, a rectangle with an accumulated volume indicator. The number of Veterans waiting to start in a typical week is determined by the difference between the number of Veterans that flow into that state and the number of Veterans who flow out of that state each week. The rate dials depict the flows in and out of stocks, and the primary rates in _Modeling to Learn_ are patients per week, appointments per week, and episodes of care per week. The initial values for any team or site depicted in the Simulation User Interface are based on the last two years of data exported from the Data User Interface. The important idea is that in any dynamic system, things get to be the way they are over time. Things either stay the same or improve decision by decision over time. For example, in a clinic or site, the number of Veterans engaged in a given state of care, such as medication management or psychotherapy, reflects decisions that lead to accumulation in that state over time. The dynamic stock and flow diagram provides a way of understanding where Veterans get stuck. As clinicians know, often there is more than one way out of a state of care. With psychotherapy, you may flow from the first session to the second session, but you may drop out at any point in that flow. Why does _Modeling to Learn_ emphasize flows through care over time? Watch that video to find out.
## Why does _Modeling to Learn_ emphasize flows through care?
@@ -89,4 +89,4 @@ Hi, I'm Lindsey and this is Debbie. Why is _Modeling to Learn_ able to provide n
[](https://bcove.video/3ME6cvm)
-Hi, I'm Lindsey, and this is David. Why does _Modeling to Learn_ emphasize flows through care over time? Our goal is to ensure Veterans flow through our addiction and mental health care services to recovery. A basic idea is that care occurs over time. Flows are powerful because they express a rate of change, which means they give us key information about where we will be in the future. Imagine your next trip in the car. We need the miles per hour to know when we will arrive at our destination. But knowing the number of miles to our destination is not enough. We must also know how fast we're traveling to know when we will get there. In other words, miles per hour. The same is true in our clinical care. It's not enough to know how many new medication management appointments we have on the books today. We need to know how quickly those appointments were added to the books or the booking rate, in appointments per week, as well as how quickly those appointments are completed or the completing rate, in appointments per week. When the inflow is greater than the outflow, the level of the appointment or patient stock will rise. When the outflow is greater than the inflow, then the level of the appointment or patient stock will fall. Opening the _Modeling to Learn_ Simulation User Interface at [mtl.how/sim](https://forio.com/app/va/va-psd-sim/login.html), you can see where all the Veterans are accumulating in stocks. You can understand the inflows and outflows for each stock and which accumulations are likely to get better or worse over time. Many clinical teams talk about feeling inundated by the addiction and mental health needs of the communities that rely on them. They can feel like they're drowning, providing the volume of care needed, especially when short staffed. We understand this kind of flooding from the natural world. To stay safe, it's critical to know how fast the waters are rising and where. We understand this by understanding flows. You may relate to this feeling of being overwhelmed and want to know more about how _Modeling to Learn_ can help. For example, how does _Modeling to Learn_ help improve medication management? Watch that video to find out.
+Why does _Modeling to Learn_ emphasize flows through care over time? Our goal is to ensure Veterans flow through our addiction and mental health care services to recovery. A basic idea is that care occurs over time. Flows are powerful because they express a rate of change, which means they give us key information about where we will be in the future. Imagine your next trip in the car. We need the miles per hour to know when we will arrive at our destination. But knowing the number of miles to our destination is not enough. We must also know how fast we're traveling to know when we will get there. In other words, miles per hour. The same is true in our clinical care. It's not enough to know how many new medication management appointments we have on the books today. We need to know how quickly those appointments were added to the books or the booking rate, in appointments per week, as well as how quickly those appointments are completed or the completing rate, in appointments per week. When the inflow is greater than the outflow, the level of the appointment or patient stock will rise. When the outflow is greater than the inflow, then the level of the appointment or patient stock will fall. Opening the _Modeling to Learn_ Simulation User Interface at [mtl.how/sim](https://forio.com/app/va/va-psd-sim/login.html), you can see where all the Veterans are accumulating in stocks. You can understand the inflows and outflows for each stock and which accumulations are likely to get better or worse over time. Many clinical teams talk about feeling inundated by the addiction and mental health needs of the communities that rely on them. They can feel like they're drowning, providing the volume of care needed, especially when short staffed. We understand this kind of flooding from the natural world. To stay safe, it's critical to know how fast the waters are rising and where. We understand this by understanding flows. You may relate to this feeling of being overwhelmed and want to know more about how _Modeling to Learn_ can help. For example, how does _Modeling to Learn_ help improve medication management? Watch that video to find out.
diff --git a/docs/search_index.json b/docs/search_index.json
index 473759ce..451f4a93 100644
--- a/docs/search_index.json
+++ b/docs/search_index.json
@@ -1 +1 @@
-[["index.html", "Modeling to Learn 3.7 Consult Manual Chapter 1 Why use Modeling to Learn 1.1 How might Modeling to Learn help? 1.2 How can a Modeling to Learn consult help? 1.3 What gets in the way of meeting patients’ needs? 1.4 If we keep making the same care decisions, will things get better or worse? 1.5 Why is Modeling to Learn useful when we have critical staffing and hiring needs? 1.6 Why is the MTL Red Data User Interface useful? 1.7 Why is the MTL Blue Simulation User Interface useful? 1.8 Why is applied systems thinking more likely to help us avoid costly mistakes? 1.9 Why is Modeling to Learn able to provide new insights? 1.10 Why does Modeling to Learn emphasize flows through care?", " Modeling to Learn 3.7 Consult Manual Team PSD 2024-10-22 Chapter 1 Why use Modeling to Learn Click on the hyperlinks or video thumbnails to watch the videos. Transcripts of the videos are also provided. 1.1 How might Modeling to Learn help? Watch the video here. Hi, I’m Lindsey and this is Debbie. You might wonder What is Modeling to Learn, why use it, and how might it help? We often don’t understand what is driving our care problems over time. Modeling to Learn helps to upgrade the decisions you make all day, every day, because the decisions made most frequently will be the most powerful for improving our care. We’re introducing Modeling to Learn, which has been developed and evaluated in the VA for over 9 years. Modeling to Learn is a way to upgrade care decisions by understanding the dynamics of common care problems over time. Are you very, very busy? Modeling to Learn includes step by step guides with gifs showing each click. These guides are available at mtl.how. Are you unsure what data to use, how it’s defined, and are you wishing it was current enough to guide your care decisions? Modeling to Learn includes a transparent, real-time locally defined data user interface available at mtl.how/data, where you specify the clinics that define your data views. Are you unsure what is realistic for improving care locally, given your patients’ needs and your existing staff and resources? Modeling to Learn also includes a simulation modeling interface available at mtl.how/sim, so you can try out new decisions before you implement them in the clinic. We all want to meet Veterans needs for timely, high-quality care. So, what gets in our way? Watch that video to find out. 1.2 How can a Modeling to Learn consult help? Watch the video here. Hi, I’m Lindsey and this is David. Modeling to Learn improves visibility and provides new insights into how common care problems persist over time. How? Modeling to Learn is based on over 60 years of scholarship known as participatory system dynamics. For this reason, we call ourselves Team PSD for Team Participatory System Dynamics. The Data User Interface and Simulation User Interface comprise two versions of Modeling to Learn. The data-only version is known as Modeling to Learn Red; Modeling to Learn Blue ads participatory learning from simulation. Team PSD supports MTL Red and MTL Blue and is carefully evaluating how each works to support VA in meeting Veterans needs. Learning from simulation can help us to place a better initial bet on what is likely to work locally by evaluating alternative decisions via simulation before we implement them in the real world. MTL Red tells us where we’ve been over the last two years based on the clinic selections made to produce the patient data reports and team trends or visualizations. Many staff report that viewing the Data UI real-time patient data tabs or the team trends is efficient and encouraging. The data tabs help with clinical decision making. The visualizations of team trends provide leading indicators that improvement efforts are paying off, which can be validating. Why is Modeling to Learn useful when we have critical staffing and hiring needs? Watch that video to find out. 1.3 What gets in the way of meeting patients’ needs? Watch the video here. Hi, I’m Lindsey and this is David. We all want to meet Veterans’ needs for timely, high-quality care. So, what gets in our way? Modeling to Learn is a way to upgrade care decisions by understanding the dynamics of common care problems over time. Demanding clinical days mean we don’t often have bandwidth to gain new insights, particularly in our understanding of interdependent clinic dynamics over time. We’re often flying blind to the impacts that our daily decisions have on the overall community that relies on us. As clinicians, we look at the patient in front of us and decide when we think they should be seen again. But have you ever been told you cannot see a patient as soon as you would like due to the constraints of the clinic? For example, an evidence-based course of care requires starting treatment without delay and keeping the Veteran engaged in a therapeutic dose of care over time to meet their needs. If we only emphasize Veterans starting care, but the time between visits extends way out, this interferes with evidence-based care and Veterans getting better. How can we get more Veterans better without adding hours in our day? Watch that video to find out. 1.4 If we keep making the same care decisions, will things get better or worse? Watch the video here. Hi, I’m Lindsey and this is Debbie. What if we keep making the same care decisions? Will things get better or worse? This is almost a trick question. If we keep making the same care decisions, more than likely things will stay the same in the clinic. Very rarely do clinical teams not have any local improvements that they would like to make. So, what do we mean by staying the same? We know that things in the clinic are not static but are always dynamic and always changing. We’ll describe how start delays affect treatment decisions to explain what we mean. The Modeling to Learn Simulation User Interface at mtl.how/sim is designed to help a team see what is likely to happen over the next two years if they continue to make the same daily clinical decisions that were made over the past two years. We call this the base case of no new decisions. Any number of feasible alternative decisions a team or site could make can be compared against this base case to find the best options for improving care over time. Because feedback dynamics produce non-linear trends or system behaviors, this means even the base case run can be surprising for some teams. The base case may show even more undesirable likely futures which teams would need to make new decisions to avoid. An example is the way start delays affect treatment decisions through balancing feedback. As the number of patients waiting to start a specific service grows, clinicians must adjust and try to find another way to meet their Veterans’ needs. Over time, as the clinicians learn about a local delay, it starts to affect what they think the most ethical, clinically appropriate treatment for the Veteran will be. An analogy from daily life will help. If there’s a traffic jam outside your VA, you may try to find another route home. As more drivers choose alternative routes, traffic returns to normal. Of course, when drivers don’t know about the jam, it will get worse over time. And whether drivers choose alternative routes or not—with balancing feedback like a traffic jam, where more cars are getting on the highway than are getting off—cars will build up and slow down the rate of traffic for everyone, whether we like it or not. We have these kinds of experiences in our clinics, too. Balancing feedback can be a blessing or curse. A blessing in that the balancing feedback causes the clinical system to return to its status quo. A curse if the status quo is undesirable for quality of patient care or provider quality of work life. When you start Modeling to Learn, it is surprising to see how powerful feedback is in driving care dynamics in any clinical care setting—VA or non-VA, VAMC or CBOC, urban and rural, BHIP, or PCT. General system insights, like the link between balancing feedbacks and nonlinear system behaviors, can be understood quickly from Modeling to Learn. You can understand exactly how many patients will be waiting and when. And then discover options for addressing the number of patients waiting for specific services with your existing staff mix. If you’d like specific recommendations for your local site or team based on these dynamics, request a Modeling to Learn consult or use the guides and videos at mtl.how. Clinical oscillations can really whipsaw clinicians and patients around, which is stressful and can increase potential clinical risks. If the number of patients waiting to start grows for a long time, it’s overwhelming. But it can also be exhausting when things go up and down quickly and clinics feel like they’re chasing their tails trying to find improvements. Especially given these dynamics can vary service by service, even within the same team. If you want to understand this principle, go to mtl.how/sim and check out the Team Care module. Or are you interested in another common balancing care problem, how the appointment backlog extends the weeks between visits? Watch that video to find out. 1.5 Why is Modeling to Learn useful when we have critical staffing and hiring needs? Watch the video here. Hi, I’m Lindsey and this is Debbie. Leaders and clinicians often ask how can Modeling to Learn be useful when facing critical staffing and hiring needs? With limited staff coverage, the need to hire is priority number one, but even then there isn’t a magic wand that produces staff where they may not exist today. What could help now? From the beginning, Modeling to Learn prioritized evidence-based episodes of care within existing staff time. When Modeling to Learn launched nationally in the VA in March of 2020, it was to empower staff to find the highest yield local improvements without asking staff to do more with less. As a result, when staff coverage may not change quickly, Modeling to Learn will find options for improving quality of care for Veterans and quality of work life for providers. How? What does Debbie mean by high yield local improvements? Because evidence-based behavioral healthcare is delivered over time, there are many possibilities. Small decisions made all day, every day by the clinicians, when compounded over time, can be surprisingly powerful. Think of your savings account or your waistline. In Modeling to Learn, we look for the lightest clinical lift teams and VAs can make that have the biggest payoff for Veterans in terms of timely, high-quality care. We work hard to avoid big difficult changes with limited benefit. When working with the Modeling to Learn-read data user interface, clinical teams are often motivated when they see trends that reflect hard-won efforts to implement high quality episodes of care which may not show up in other data systems for some time. The Modeling to Learn Blue simulation user interface saves staff time because alternatives can quickly be assessed during a modeling consultation. Change is hard, but we no longer have to learn by trial and error, wearing out already burdened staff. Does that sound too good to be true? If so, you may be wondering about examples of Modeling to Learn use cases for the pain points you face in your specific team or program. As an example, how does Modeling to Learn benefit substance use disorder, or SUD programs? Watch that video to find out. 1.6 Why is the MTL Red Data User Interface useful? Watch the video here. Hi, I’m Lindsey and this is Debbie. Why is Modeling to Learn Red useful and how does the Modeling to Learn Data User Interface provide new insights? The primary value of MTL Red is its power to efficiently query the VA Corporate Data Warehouse directly, and we’ve come a long way over the years. Yes we have. When we first began, we were using an Excel workbook so that frontline teams could carefully review the clinic selections that define their Modeling to Learn team data sets. Fast forward to the present and now teams have real time data available to them from within the VA domain from any computer with PIV badge access. Since the Data UI includes PHI, if you go to mtl.how/data, you will be able to see the same data you have permissions to access in the electronic health record. But given that clinicians, managers, data leads, quality improvement staff, evaluators … well, basically everyone is so busy, no one has time to review another data dashboard unless it offers something of really high value that distinguishes it from other resources. When you request a Modeling to Learn Red consultation, we work with your team or VA to explore care data that can address your locally identified priority. Clinics or scheduling grids are often changing. For that reason, we wanted a user interface where clinic selections can include active or inactive clinics over the last two years. That way, the team can filter the information to find the most appropriate clinics to include in their data set to gain new insights. And in Team Flow, clinic selections can be used to evaluate transitions between an episode of care in one team and the start of another episode of care in a higher or lower intensity care setting. Modeling to Learn Red also enables zooming in to check on the care of an individual patient at the start of the clinical day or during case reviews at a team meeting. But with the MTL Red Data User Interface, you can also zoom out to view teen care trends, bringing patient-level care coordination and trend-level process improvement decisions together. And that’s where things start to get interesting. Based on the clinic selections, the next set of tabs to find local data values for common care problems, including care coordination, psychotherapy, medication management, team care, and team flow. Each tab features simple definitions of how data were estimated for the common care problem. Detailed definitions with technical specifications are also provided to allow valid comparison of these data to other VA dashboards. That said, a focus on data details could be frustrating and add limited value. Modeling to Learn emphasizes understanding system problems in care flow over time. These care flow problems can be defined accurately with just five key time-based variables that drive care quality. How do five key variables drive care quality? Watch that video to find out. 1.7 Why is the MTL Blue Simulation User Interface useful? Watch the video here. Hi, I’m Lindsey and this is Debbie. Why is Modeling to Learn Blue useful? Well, why is it wisest to focus on the dynamics of care over time? The short answer is that clinical and improvement teams cannot adjust one part of the care equation without everything else changing. In Modeling to Learn Blue, we zoom out to see how care variables are locked in relationship with one another over time. The key variables that define either a poor quality or high quality episode of care must be understood together. Building from MTL Red, you can export your local data set created in the Modeling to Learn Data User Interface at mtl.how/data. Then, if you navigate to mtl.how/sim, you can find the Modeling to Learn Blue Simulation User Interface, which is a dynamic and interactive way to understand why problems with care coordination, medication management, psychotherapy, team care, and team flow persist over time. The Simulation User Interface is a way to see how adjustments in one part of an episode of care explain subsequent impacts in the care system. For any of the common care problems, the simulation saves you time and energy by accounting for the local new patient start rate in patients per week and the local appointment supply and appointments per week. The simulation also keeps track of the local new patient wait time in weeks, time between visits and weeks, and the engagement duration over time, again in weeks. All are calculated for you automatically in the Data User Interface, but their interdependence is accounted for in the Simulation User Interface. The Modeling to Learn Blue Simulation User Interface empowers teams to avoid ineffective strategies because you very quickly learn to develop new insights that would be inefficient, if not impossible, to figure out in your head or by hand. Learning from simulation is designed to help upgrade local decision-making. Teams develop new rules of thumb and insights in which the dependent dynamics among these variables that define care are all taken into account. With a Modeling to Learn consult, we come alongside with partners mid stride in their daily clinical activities who may have limited insight into what is likely to happen over the near future if they keep making the same decisions every day. With Modeling to Learn Blue simulation learning, sites and teams can safely see the impact of new decisions while building new capacities for systems thinking. Why is applied systems thinking more likely to help us avoid costly mistakes? Watch that video to find out. 1.8 Why is applied systems thinking more likely to help us avoid costly mistakes? Watch the video here. Hi, I’m Lindsey and this is David. Why is applied systems thinking more likely to help us avoid costly mistakes? Unless we understand interdependent system effects over time, we’re very likely to choose ineffective strategies for our clinical work. And when chronic impairment, relapse, suicide, and overdose are the critical concerns for our patients, we need system insights about how to best organize our clinical practice. Applied systems thinking includes understanding complexity, feedback, and system behaviors over time. By complexity, we mean understanding the relationship or the interaction among two or more variables, such as wait times and the improvement rate. By feedback, we mean moving beyond simple cause and effect events to consider how some effects are reinforced and get stronger over time and other effects are reduced over time. To explain feedback using David’s example, it’s not just that if you have long wait times then the patient improvement rate will go down because patients aren’t starting care. It’s also that as the improvement rate goes up, then at a system level wait times will go down as Veterans get their needs met and graduate from care. Our systems are perpetually causing themselves over time through feedback. Causal feedbacks explain trends over time, called system behaviors. Given that feedback can be a reinforcing effect or a balancing effect over time, Modeling to Learn provides better insights about short- and long-term understanding of change over time. This can include decisions that may make things worse before they get better, or may make things better before they get worse. For example, longer wait times for new patients may reduce their improvement rate in the short term, but if this enables higher quality care engagement for them and existing patients over time, then more Veterans can get better and graduate from care and wait times will go down for the entire community of patients. It is through improving our systems thinking that Modeling to Learn really helps us move beyond making costly mistakes. If you’d like to learn more about how Modeling to Learn helps us improve the dynamics driving where Veterans get stuck or drop out of care, watch that video to find out. 1.9 Why is Modeling to Learn able to provide new insights? Watch the video here. Hi, I’m Lindsey and this is Debbie. Why is Modeling to Learn able to provide new insights about where Veterans get stuck or drop out of care? Because the Modeling to Learn Simulation User Interface depicts the states of care as stocks based on the flows in and out of that state. Take for example, patients waiting to start. We depict patients waiting to start as a stock, a rectangle with an accumulated volume indicator. The number of Veterans waiting to start in a typical week is determined by the difference between the number of Veterans that flow into that state and the number of Veterans who flow out of that state each week. The rate dials depict the flows in and out of stocks, and the primary rates in Modeling to Learn are patients per week, appointments per week, and episodes of care per week. The initial values for any team or site depicted in the Simulation User Interface are based on the last two years of data exported from the Data User Interface. The important idea is that in any dynamic system, things get to be the way they are over time. Things either stay the same or improve decision by decision over time. For example, in a clinic or site, the number of Veterans engaged in a given state of care, such as medication management or psychotherapy, reflects decisions that lead to accumulation in that state over time. The dynamic stock and flow diagram provides a way of understanding where Veterans get stuck. As clinicians know, often there is more than one way out of a state of care. With psychotherapy, you may flow from the first session to the second session, but you may drop out at any point in that flow. Why does Modeling to Learn emphasize flows through care over time? Watch that video to find out. 1.10 Why does Modeling to Learn emphasize flows through care? Watch the video here. Hi, I’m Lindsey, and this is David. Why does Modeling to Learn emphasize flows through care over time? Our goal is to ensure Veterans flow through our addiction and mental health care services to recovery. A basic idea is that care occurs over time. Flows are powerful because they express a rate of change, which means they give us key information about where we will be in the future. Imagine your next trip in the car. We need the miles per hour to know when we will arrive at our destination. But knowing the number of miles to our destination is not enough. We must also know how fast we’re traveling to know when we will get there. In other words, miles per hour. The same is true in our clinical care. It’s not enough to know how many new medication management appointments we have on the books today. We need to know how quickly those appointments were added to the books or the booking rate, in appointments per week, as well as how quickly those appointments are completed or the completing rate, in appointments per week. When the inflow is greater than the outflow, the level of the appointment or patient stock will rise. When the outflow is greater than the inflow, then the level of the appointment or patient stock will fall. Opening the Modeling to Learn Simulation User Interface at mtl.how/sim, you can see where all the Veterans are accumulating in stocks. You can understand the inflows and outflows for each stock and which accumulations are likely to get better or worse over time. Many clinical teams talk about feeling inundated by the addiction and mental health needs of the communities that rely on them. They can feel like they’re drowning, providing the volume of care needed, especially when short staffed. We understand this kind of flooding from the natural world. To stay safe, it’s critical to know how fast the waters are rising and where. We understand this by understanding flows. You may relate to this feeling of being overwhelmed and want to know more about how Modeling to Learn can help. For example, how does Modeling to Learn help improve medication management? Watch that video to find out. "],["2-how-to-use-modeling-to-learn.html", "Chapter 2 How to use Modeling to Learn 2.1 How can we get more patients better without more hours in the day? 2.2 How does Modeling to Learn benefit Substance Use Disorder or SUD programs? 2.3 How does Modeling to Learn benefit PTSD Clinical Teams or PCTs? 2.4 Can Modeling to Learn benefit Behavioral Health Integration Program or BHIP teams? 2.5 How does Modeling to Learn support stepped care? 2.6 How does measurement-based care detect patient improvement or risk? 2.7 How can we improve SAIL with Modeling to Learn? 2.8 How do five key variables drive care quality over time? 2.9 How does Modeling to Learn help improve medication management? 2.10 How does Modeling to Learn help improve psychotherapy? 2.11 How does our appointment backlog extend weeks between visits? 2.12 How can we better balance needs of new and existing patients? 2.13 How can we leverage the feedback, rates, and volume of our local care system? 2.14 What key drivers of higher care quality improve recovery and prevent suicide?", " Chapter 2 How to use Modeling to Learn Click on the hyperlinks or video thumbnails to watch the videos. Transcripts of the videos are also provided. 2.1 How can we get more patients better without more hours in the day? Watch the video here. I’m Lindsey, and this is Debbie. How can we get more Veterans better without adding hours in our day? To get Veterans better, we have to account for the number of hours available in the day. Yet because we don’t have the right information about how we manage our time, we can pursue unrealistic treatment plans or improvement strategies that violate the laws of physics. Modeling to Learn encourages us to account for clinician time differently, to make our plans feasible locally. Most VA data systems offer snapshots of cross-sectional data, but this kind of information doesn’t help us account for the dynamics of evidence-based care over time. While many VA data systems are available, most display either quarterly lags from a high-level view or an in the weeds patient by patient view. Modeling to Learn helps to bridge these two views. Modeling to Learn introduces trends over time based on the clinic selections carefully reviewed by the staff on site. Trends help us zoom out to understand how we’re doing for most of the Veterans we serve. Modeling to Learn also introduces time-based definitions of care in the units of patients per week, appointments per week, or episodes of care per week. We use a week because over the last several years clinicians told us this is how they think in the clinic. There often is no typical day and a lot has happened by the end of a month. Defining care by week is more beneficial. High quality care in the clinic is always constrained by the available staffing capacity and the hours in the day to see patients, the number of weeks between visits, and the weeks of engagement over time. But do you have time to run these equations in your head or calculate them from the other dashboards you use? As an alternative, Modeling to Learn puts these components of care together to understand their interdependence over time. How can Modeling to Learn consultation help? Watch that video to find out. 2.2 How does Modeling to Learn benefit Substance Use Disorder or SUD programs? Watch the video here. Hi, I’m Lindsey and this is David. How does Modeling to Learn benefit substance use disorder, or SUD programs? When Modeling to Learn began almost 10 years ago, as a partnership among patients, providers and policy makers across VA, we worked iteratively to define the dynamics of common care problems for the primary reasons Veterans seek addiction and mental health care. We found that if we focused on improving timely, high-quality evidence-based psychotherapy and pharmacotherapy for alcohol use disorder, depression, opioid use disorder and PTSD, we could support VA staff in meeting around 80% of Veterans needs for behavioral healthcare. We work with frontline teams and partners from the VA Academic Detailing Program and the Psychotropic Drug Safety Initiative or PDSI and developed the Medication Management Module of Modeling to Learn. This module helps manage the dynamics of evidence-based pharmacotherapies for alcohol use disorder and opioid use disorder based on their effectiveness in reducing craving, relapse, and overdose. You can find the Medication Management module in Modeling to Learn data mtl.how/data and Simulation User Interface mtl.how/sim. The Modeling to Learn Medication Management module supports improvement in the AlcTop or Alcohol Topiramate indicator from the Mental Health Information System or MIS and the SUD 16 indicator from the Strategic Analytics for Improvement in Learning or SAIL. How does Modeling to Learn help? Prescribers describe the need to better visualize how to locally optimize their medication management appointment supply, including top of license care from staff with and without a Drug Enforcement Agency X-waiver for OUD medication. Modeling to Learn helps a team assess this appointment supply, new patient start rate, no show or missed appointment rate, and the clinically appropriate return-to-clinic visit interval needed for a therapeutic response to alcohol use disorder and opioid use disorder medications. The Medication Management module helps teams and sites find the optimal number of patients who can be engaged in AUD and OUD therapies over time and helps them distinguish the flow of their patients who require antidepressants or other medication needs. Given that community needs and team staffing is dynamic and can change over time, these resources can empower teams to find local improvements that otherwise may be hard to find. Another common module for addressing Veterans SUD needs for group and other therapies is the Team Care module, which helps teams assess and optimize their overall mix of local multidisciplinary services as a function of staffing and patient needs. We are evaluating how Modeling to Learn improves Veterans who start and complete a therapeutic course of medication for AUD and OUD with research funding from the VA and the National Institute on Drug Abuse or NIDA. What about other common SUD comorbidities and presenting concerns? For example, how does Modeling to Learn benefit PTSD, clinical teams or PCTs? Watch that video to find out. 2.3 How does Modeling to Learn benefit PTSD Clinical Teams or PCTs? Watch the video here. Hi, I’m Lindsey and this is Debbie. How does Modeling to Learn benefit PTSD clinical teams, or PCTs? Since I’m at the National Center for PTSD, this priority is very close to my heart. When Modeling to Learn began almost 10 years ago as a partnership among patients, providers and policy makers across VA, we worked iteratively to define the dynamics of common care problems for the primary reasons Veterans seek addiction and mental health care. We found that if we focused on improving timely, high-quality evidence-based psychotherapy and pharmacotherapy for alcohol use disorder, depression, opioid use disorder and PTSD, we could support VA staff in meeting around 80% of Veterans needs for behavioral healthcare. Modeling to Learn helps teams better evaluate the local dynamics of balancing the needs of new and existing patients for evidence-based psychotherapies, particularly cognitive processing therapy and prolonged exposure. Teams described the challenges of starting new patients who had never had their needs met without compromising care for Veterans already engaged in therapy. Modeling to Learn enables teams to see where their Veterans get stuck, to decide when to graduate patients, or to recognize when the weeks between return to clinic visits start to get too long. Have you been trying to improve PTSD 56 in the Strategic Analytics for Improvement and Learning, or SAIL? We worked with local PTSD clinical teams from the beginning and have supported many more teams over the years who have engaged with the evidence-based psychotherapy programs or the PTSD mentorship program to meet these needs. The Psychotherapy module of the Modeling to Learn data user interface, and simulation user interface helps teams to see how they’re doing for patients who are starting psychotherapy, how many flow through to complete a therapeutic dose, and whether Veterans are getting near weekly therapy consistent with the PTSD treatment evidence base. There’s also the ability to assess emerging approaches such as masked or intensive outpatient psychotherapy and other scenarios that are commonly considered and implemented in PCTs. You may also be wondering about the flow from primary care or from primary care mental health integration, or PCMHI, teams into general mental health and specialty mental health programs like PCTs. Stepped care up and down the mental health continuum of care can also be optimized to local needs using the Modeling to Learn Team Flow module. Speaking of the care continuum, you may be wondering How does Modeling to Learn benefit Behavioral Health Integration program or BHIP teams? Watch that video to find out. 2.4 Can Modeling to Learn benefit Behavioral Health Integration Program or BHIP teams? Watch the video here. Hi, I’m Lindsey and this is David. How does modeling to learn benefit Behavioral Health Integration Program or BHIP teams? What are the challenges with providing excellent team care? One challenge is the data sources to support team decision making jump from the individual patient and clinician counter all the way to the entire clinic or facility. It can mean that teams are flying blind to how they’re coordinating care for their Veterans. This can be especially hard when teams are short staffed or when staff, patient needs, and local policies continually change. It’s hard to know what improvements you might be able to find in your own team for coordinating evidence-based psychotherapies and pharmacotherapies over time. One way that Modeling to Learn can complement other BHIP resources is through the Data User Interface available at mtl.how/data. The Clinic Selection and Team Flow Selection tabs empower teams to carefully review the clinics that comprise their data views. Data are never all good or all bad. Modeling to Learn emphasizes transparency. Teams need to know what data they’re getting and its strengths and weaknesses for guiding a particular decision. The Clinic Selection tabs update every day to reflect all the Mental Health 500 series Stop Code, Clinic, or Grid updates that may be going on locally. Teams can save bookmarks that reflect their specific clinic selections, for example to include trainees or to select any of the clinics they have used within the last two years. This can be helpful when new BHIP teams are being formed or remapped. The team Patient Data tabs update daily too, so teams can view the patient diagnosis, encounter, health factor, measurement-based care, and high-risk flag information all in one place, and then the team can zoom out to filter and view these data as team trends. Coordinating care within a team is challenging. No single clinician can offer all the services a Veteran may need, so treatment plans must be coordinated in real time, over time, to support Veteran improvement. Clinicians may not have visibility on exactly how appointment supplies divvied up into intake evaluations, individual and group psychotherapy, medication management, or other adjunctive supportive therapies. What is particularly important about the Modeling to Learn simulation user interface at mtl.how/sim is the ability to assess not only the appointment supply, but even more important, the service proportions of patients that flow through to these services after intake or treatment plan review. One of the most common scenarios teams assess is how to respond to patient needs when they may not have the perfect staffing mix. Even when you know patient flow through care is divvied up into a mix of services, the dynamics over time are too much to understand in your head. The Modeling to Learn simulation user interface enables teams to quickly assess the dynamics of available appointment supply, service proportions, and the other two major factors that govern patient improvement, the return-to-clinic visit interval in weeks and the overall duration in care. We’ve often used Modeling to Learn to help teams find a way to ensure evidence-based care through the decisions that they make all day and find improvements in quality of care and quality of work life that they didn’t think was possible due to limited staffing. You might wonder How does Modeling to Learn support stepped care up to BHIP from primary care or primary care mental health integration, or from general mental health up or down into specialty mental health programs such as substance use disorder programs or PTSD clinical teams? Watch that video to find out. 2.5 How does Modeling to Learn support stepped care? Watch the video here. Hi, I’m Lindsey and this is Debbie. How does Modeling to Learn support stepped care? We developed the measurement-based stepped care module of Modeling to Learn to help teams assess episodes of care when stepping patients up or down across the continuum of care. General mental health or behavioral health integration program, or BHIP teams, and specialty mental health teams such as substance use disorder, or SUD, programs and PTSD clinical teams, or PCTs, use the team flow selection tab in the Modeling to Learn Data User Interface to define flow from their care up or down to another team. Teams make these team flow selections at mtl.how/data. Based on years of testing the Data User Interface, we found that teams are most accurate in defining who they refer Veterans to after completing an episode of care within their team, rather than where they receive referrals from. And the teams also define the gap between visits that they believe is locally most reasonable for defining a new episode of care in each setting. The primary evaluations in the Team Flow module of the Modeling to Learn Simulation User Interface include assessing the flow through care of Veterans with high-risk flags as well as the proportions of high- and low-symptom patients as they flow through care to recovery and step down or discharge. We know that higher care quality improves recovery among Veterans, but how does that vary as a function of the total patients a team is serving? Team Flow can be used to assess the total manageable patients in a team and the impact of the patient load on care quality. Teams can also account for use of community care, which reduces some of the patients actively managed in the team. But as teams know, it does not reduce this care management to zero as Veterans using community care may still trigger an emergency care response or coordination of other services within a team. Perhaps most helpful is the view at a glance of where all Veterans are accumulating, waiting to step up or down across the continuum of care. This can help leadership and teams understand where there may need to be improved referral flows or service agreements between these settings. Teams can evaluate the flows through care when implementing measurement-based care to reduce the time it takes to detect when Veterans may be getting better or worse, which can also help improve agreements or decisions about when transitions across service settings are warranted. Would you like to know more about using measurement-based care to improve the time to detect patient improvement or risk? Watch that video to find out. 2.6 How does measurement-based care detect patient improvement or risk? Watch the video here. Hi, I’m Lindsey and this is David. Why use measurement-based care to improve the time to detect patient improvement or risk? To improve care quality in the flow of patients through care to recovery, the major value of measurement-based care is ensuring Veterans are getting appropriate care and are responding to the care they receive. Clinicians are likely familiar with the emphasis on measurement-based care, but from the perspective of care flow, what is powerful is reducing the time it takes for a team to detect whether a patient is getting better or worse. In the Modeling to Learn Data User Interface at mtl.how/data, the dataMeas tab provides tables of all the patient measures recorded at the VA Corporate Data Warehouse, whether they were captured via an evidence-based psychotherapy template, mental health assistant, or other approach. These include measures such as the GAD-7 for anxiety, PHQ-2 or PHQ-9 for depression, PCL-5 for PTSD, and the Brief Addiction Monitor or BAM. As a clinician, I like to use the patient-level views to sort scores from high to low to identify patients who may need to step up to a higher level of care or might benefit from referral to a specialty substance use disorder or PTSD service. The lowest score may indicate Veterans who can graduate from outpatient mental health care and step back down to primary care, which is an amazing day for the Veteran, their loved ones, and their treatment team. The vizMeas tab helps teams see trends in these measures to confirm when their efforts to implement measurement-based care are paying off. Teams can toggle to see what proportion of services include measurement-based care by comparing the vizMeas and vizEnc views. The Modeling to Learn Simulation User Interface at mtl.how/sim also empowers teams to assess the impacts of improved implementation of measurement-based care and how it can drive improvements in care quality. Do you want to learn more about the key drivers of higher care quality to improve recovery and prevent suicide? Watch that video to find out. 2.7 How can we improve SAIL with Modeling to Learn? Watch the video here. Hi, I’m Lindsey and this is David. If you’re watching this video, you want to know about strategic analytics improvement in learning, or SAIL metrics, with Modeling to Learn. Modeling to Learn was developed to complement existing VA priorities. One important benchmark is SAIL. Some of the more frustrating aspects of SAIL are the lag in data to support daily decision-making in the clinic and emphasis on high-level facility views and, of course, how challenging it can be to improve sale metrics over time, especially when you’re short-staffed. In fact, when SAIL metrics change over time, either in a positive or negative direction, it is difficult to know what’s driving the change. Facility leadership and clinical teams may not know how to interpret these changes or what improvement strategy to try next. Modeling to Learn supports two primary dimensions of the SAIL mental health quality domain—care coverage and care continuity. But importantly, Modeling to Learn recognizes that these two dimensions may work against each other in a dynamic tension over time. If you’re expanding population coverage measures, such as PSY 32 for depression, PSY 38, or PTSD 56 for PTSD, or SUD 16 for opioid use disorder, then it may be more challenging to ensure continuity of care measures such as MDD43h, MDD47h, and PSY 33 for depression and PSY 39 for PTSD. Of course, the reverse is also true. If you’re doing great with care continuity on those measures, then it may be challenging to expand population coverage at the same time. I’m fond of calling this the physics of our care quality problems. What I mean is that we cannot change the laws of physics, such as the law of conservation, which states that matter and energy in a physical process cannot be created or destroyed, but can only be transformed. In Modeling to Learn, staff time is neither created nor destroyed. This means that when data is exported from the MTL Data User Interface and uploaded to the MTL Simulation User Interface, the scenarios a team or VA explores to find improvements in care quality will always account for or conserve the total local staff time available. This helps to avoid fixes that fail, such as something that improves population coverage in the short term but reduces care continuity in the long term. Have you ever felt like you were playing Whac-A-Mole® with these metrics, where you improve one indicator only to have a problem later somewhere else? Modeling to Learn can help. Let’s start with why Modeling to Learn Red is useful. And why does the Modeling to Learn Data User Interface provide new insights? Watch that video to find out. 2.8 How do five key variables drive care quality over time? Watch the video here. Hi, I’m Lindsey and this is David. How do five key variables drive care quality? Modeling to Learn emphasizes the dynamics of care over time, which can be accurately simplified to the key time-based variables that drive care quality. But be careful, the important principle is that these variables operate together over time to define an episode of care. That means care quality cannot be improved without understanding how these variables influence one another. If you want to see what Lindsey’s talking about, navigate to the Modeling to Learn Data User Interface at mtl.how/data and review each care problem—care coordination, medication management, psychotherapy, team care, and team flow. Our partners across VA describe how challenging it is to review data in one information system and then in another and end up unsure how to reconcile them, especially when you think that they indicate a different course of action. Folks are extremely busy and any new data resources must really add value to be worth learning. Let’s see if these Modeling to Learn variables meet the commonsense test of value added. Well, what do you think the five time-based variables are that make up an evidence-based episode of care? All outpatient care is defined by whether you can get an appointment when you need help. We focus on this all the time in VA. But then, and this is critical, you must be able to be seen again to complete a therapeutic course of care adequate to meet your need. Clinicians told us that a week was the way they think clinically. So, in Modeling to Learn, teams make their clinic selections to obtain an estimate of their local new patient start rate in patients per week and their appointment supply in appointments per week. Then we define evidence-based engagement as the new patient wait time in weeks, time between visits and weeks, and the engagement duration over time, again in weeks. This is a simplified definition of an evidence-based episode of care that is accurate for time. Why is it wise to focus on the dynamics of care over time? That’s why Modeling to Learn Blue is useful. So watch that video to find out. 2.9 How does Modeling to Learn help improve medication management? Watch the video here. Hi, I’m Lindsey and this is Debbie. How does Modeling to Learn improve medication management? Medication management requires identifying a patient need, starting a medication, and evaluating for a therapeutic response over time. Some of the challenges and effective medication management over time include the realities that not all clinicians can prescribe. And of course, it requires follow up care to ensure that a patient is benefiting from their medication. So, when should the patient be seen again? One thing that can be so frustrating to prescribers is that a scheduler tells them when a patient can be seen again, rather than them telling the scheduler when the patient should be seen based on top of licensed care. The frustration is that medication management return-to-clinic visits may not occur at the appropriate interval to balance patient needs and risks. The VA has guidelines for some patient cohorts such as SAIL, MDD43h, and MDD47h, which define high quality continuity of follow up care as a valuation once every 12 weeks for Veterans diagnosed with depression who are using antidepressants. Other needs may reflect a different follow-up interval. For example, many addiction psychiatrists have described the appropriate medication management return-to-clinic visit interval for buprenorphine or methadone for opioid use disorder as every four weeks. These differences in medication needs, and the evidence-based pharmacotherapy standards that go with them, make it hard for prescribers to evaluate at a patient population level in a real time, especially when you’re completing another medication management encounter every 20 minutes. And of course, depending on the community at any given time, the staffing mix may change and the size or volume of patient needs can change too. Some SUD teams may have a high volume of patients receiving evidence-based pharmacotherapy for alcohol use disorder, whereas the modal presenting concern in a BHIP may be more likely to be depression. And the decisions the prescribers are making using current rules of thumb may be boxing them in to a pattern of care that makes it even harder to be responsive to crises or reduce future wait times. The Medication Management module of Modeling to Learn helps a team evaluate the trade-offs among all these factors as compared to the status quo over the last two years in the clinic. Comparing to the base case of no new decisions, the team can evaluate changes in their X-waivered MOUD appointment supply or evaluate and tailor the return-to-clinic visit interval by patient cohort. The goal is to help prescribers see what daily medication management decisions are adding up to over time and find feasible and effective local rules of thumb for clinically appropriate return-to-clinic orders in weeks that reduce wait times and ensure Veterans are getting better. For coordinating medication management in a multidisciplinary team such as a BHIP or SUD program, the Team Care module of Modeling to Learn can also help. If you want to know how Modeling to Learn helps to improve psychotherapy, watch that video to find out. 2.10 How does Modeling to Learn help improve psychotherapy? Watch the video here. Hi, I’m Lindsey and this is David. How does Modeling to Learn help improve psychotherapy? VA led the nation in dissemination and training for evidence-based psychotherapy. Many clinicians are ready to offer Veterans the highest quality care available. So, what makes it hard to get Veterans through an evidence-based episode of care when they start? What I’ve learned from working with clinical psychologists like Lindsey and other VA therapists from around the country is that the dynamics of psychotherapy in the clinic are challenging for the same reasons most care problems persist over time. Clinical teams typically do not have good visibility of where all their Veterans are in care. You may know that you plan to do CPT session six today with an individual Veteran, but how many Veterans are currently in the middle of therapy now? How many appointments are on the books? How many weeks are there before they return to clinic for follow-up care and how many graduate when they complete a course of an evidence-based psychotherapy? And how many drop out of therapy after one session or early in care but return to the team later? Evidence-based psychotherapy is typically defined by getting near weekly psychotherapy and having clinically meaningful improvement, which usually starts around session 8 to 12. Many VA data systems provide useful counts and proportions of evidence-based psychotherapy templates and the SAIL continuity of care metrics to find quality based on the number of sessions Veterans get within a specific time period. But the SAIL metric itself is a quarterly snapshot, and we know Veterans don’t all start therapy at the beginning of a new quarter. In the Modeling to Learn Data User Interface, a one-year patient cohort is identified that includes all Veterans seen from 18 months to six months ago based on the clinic selections made with the Data User Interface at mtl.how/data. Then, for the Veterans seen within those 12 months, we look back to see when those individual Veterans began therapy and how they were engaged over time. Once this psychotherapy cohort is defined and uploaded to the Simulation User Interface at mtl.how/sim, we can look at the stocks and see where all the Veterans have accumulated, where they are in care in a typical week, and how many patients per week are progressing through key therapy milestones. Finally, at a local team level, psychotherapists can see how they are doing for most of the Veterans they serve. And even better, they can run experiments to see what the impact will be of making new decisions over time. You might wonder What if we keep making the same care decisions? Will things get better or worse? Watch that video to find out. 2.11 How does our appointment backlog extend weeks between visits? Watch the video here. Hi, I’m Lindsey and this is David. How does an appointment backlog extend the weeks between visits? At first, this sounds like a simple system story that many clinicians describe as we work together to co-develop Modeling to Learn. But they also told us when an appointment backlog extends the week between visits, a long interval between visits effects care too. That’s right. As the number of appointments for a specific service grows on the books, then the clinic has no choice but to increase the number of weeks or push out the time between visits to get everyone seen. We call this the return-to-clinic visit interval. This quantifies the return-to-clinic order in a unit of time—one week—which is critical for finding realistic local improvements in our partnerships with sites. Okay, so the balancing feedback principle operating here is that if you have a clinic calendar filled up with appointments, it’s harder for new patients to start care and it becomes harder for all existing patients to be seen with appropriate therapeutic follow-up standards for psychotherapy and pharmacotherapy. Lindsey, you and Debbie also talked about balancing feedback in the video, What if we keep making the same care decisions, will things get better or worse? For an explanation of how start delays affect treatment decisions over time, watch that video. As I said at the beginning of this video, it’s not just that a growing appointment backlog forces teams and clinics to extend the number of weeks between visits to get Veterans seen. It’s that long return-to-clinic visit intervals impact treatment decisions and care quality. None of these effects occur in isolation. All these components of care interact over time. It’s why system scientists will often point out there are no side effects and systems, there are only effects. Some we want and some we don’t. As clinicians identify Veterans who might benefit from an evidence-based course of psychotherapy for depression or PTSD, they may find that many patients are waiting to start, or they know that those 90833 or 90834 psychotherapy encounters are occurring 12 weeks apart—or both. Clinicians recognize that they cannot see patients in a way that is consistent with an evidence-based episode of care and will have to start finding alternatives. Modeling to Learn can help. What a relief that hundreds of clinicians, managers, data experts, quality improvement leads, and patients have already worked with us over the last decade to develop scalable Modeling to Learn resources for clinical and improvement teams to use today. The Modeling to Learn Simulation User Interface accounts for all these feedback effects so the team can upload their data, review what they see, and get immediate feedback about what the most feasible alternatives are to ensure quality of care for patients and quality of work life for providers and local staff. Based on this discussion, you may wonder How can we better balance the needs of new and existing patients? Should we prioritize new patient start rate or weeks between visits for quality? Watch that video to find out. 2.12 How can we better balance needs of new and existing patients? Watch the video here. Hi, I’m Lindsey and this is Debbie. How can we better balance the needs of new and existing patients? Should we prioritize new patient start rate or weeks between visits? These trade-offs are challenging for clinicians when you know there is a whole community of patients who need help. When teams are struggling with the limits of their available time in the day to see patients, it can feel like a clinical, ethical, and even moral quandary about how to best balance patients’ needs for services with the staff resources available. Of course, staff resources are always changing. We know many teams that are excelling in providing the highest quality addiction and mental health care available. But the behavioral health workforce shortage in the US is much bigger than VA. And VAs and teams need the right tools to make sure Veterans get the care they need for recovery. Challenges balancing the new patient start rate and the weeks between visits for existing patients apply systems thinking insights that we’ve talked about in other MTL videos, such as the physics of conserving staff time in order to ensure a clinically beneficial, realistic approach to care decisions and quality improvement that meets VA quality standards and meets Veterans needs for evidence-based episodes of care. The Modeling to Learn Blue Simulation User Interface available at mtl.how/sim enables a site or team to evaluate these two balancing system stories as a function of their data for the last two years exported from Modeling to Learn Red Data User Interface at mtl.how/data. As we talked about in the How does an appointment backlog extend the weeks between visits? video and What if we keep making the same care decisions, will things get better or worse? video, balancing feedbacks occur in any system that has a goal, including our healthcare and clinical systems. Balancing feedbacks occur in systems that have constraints of resources, staff, and time. And as a result, the trends that occur over time tend to reset to a status quo or oscillate around the status quo. A brief Modeling to Learn consult uses the site or team reviewed local data plus simulation to efficiently find local improvements that account for all these balancing trade-offs. We aim to help clinical and improvement teams find a couple empowering clinical heuristics, or rules of thumb, that are more effective for ensuring evidence-based care to get more Veterans better. When we partner with a VA or a team, we aim to find the lightest lift we can, like increase groups by 10% or adjust the return-to-clinic order for three weeks for these presenting concerns. And we often find something small that has a big payoff for Veterans. Do you want to be empowered to leverage the feedback, flow, and volume of your local care system? Watch that video to find out. 2.13 How can we leverage the feedback, rates, and volume of our local care system? Watch the video here. Hi, I’m Lindsey and this is David. Do you want to be empowered to leverage the feedback, flow, and volume of your local care system? In any system, leverage describes the places where a small shift in one thing can produce big changes in everything. For a big benefit for Veterans in their care teams, the primary places to focus are where the greatest volume is, the fastest flower rates are, and of course to account for feedback dynamics over time. When we partner with the VA or team, we aim to find the lightest lift we can, like finding for one BHIP that increasing group psychotherapy by 10% eliminates wait times and gets five times the number of Veterans near weekly therapy. Or, adjusting the return-to-clinic order by three weeks for some presenting concerns can provide the same medication management supply as hiring another prescriber. We can often find small care decisions that when implemented frequently over time, have a big payoff for Veterans because they add to more than the sum of the parts in the interdependent system over time. The last thing we want to do is make big changes that are difficult for Veterans and clinical teams that produce limited benefit over time. In my experience over many years now, finding something clinically powerful that leverages the dynamics of the system can be a huge boost to morale, productivity, and staff empowerment. You might wonder How can you find powerful insights that you are sure will work locally? We do this by reviewing the Modeling to Learn Simulation User Interface stocks to see where all the Veterans are accumulating the most. We review the rates to see where the floodwaters are rising fastest and require action. Or where things are trending in the right direction and our attention could be better focused elsewhere. Modeling to Learn consultation offers clinical and improvement teams real-time efficient support. We can do this quickly because we’ve partnered with patients, providers, policy makers, data experts, evaluators, and quality improvement consultants from the frontline to VA Central Office for nearly 10 years now to define the primary drivers of care problems and care quality using system dynamics. Modeling to Learn consultation involves empowering clinicians, managers, improvers, data leads, and all of VA to leverage the system for getting more Veterans better. How can Modeling to Learn consultation help? Watch that video to find out. 2.14 What key drivers of higher care quality improve recovery and prevent suicide? Watch the video here. Hi, I’m Lindsey and this is Debbie. What are key drivers of higher care quality to improve recovery and prevent suicide? A central idea of Modeling to Learn resources is that delivering consistent, timely, high-quality care is challenging for teams because we don’t accurately account for the primary dynamics driving care quality over time. You might be thinking Care dynamics, what does that mean? We literally mean that due to time pressures and cognitive limits, we don’t effectively understand relationships among key care decisions that we make every day over time, especially decisions made in the aggregate across a clinical team. For example, we may try to break down problems by looking at referrals or wait times, but we don’t look at how clinical teams’ knowledge of wait times influence referrals to a specific service or team. In the Modeling to Learn Data User Interface, we define care dynamics in terms of rates over time, like new patients per week, so that we can understand the flow of Veterans through care. As another example in terms of care quality, recovery and suicide prevention, the Modeling to Learn Team Flow module looks not only at the number of patients served by a team or site, or the number of patients with a high-risk flag for suicide, but also the time it takes to unflag high-risk patients in a team and the typical time to improve within the team over the last two years. The Modeling to Learn Simulation User Interface uses an interactive structure fed by local team data from the Data User Interface. The Modeling to Learn Simulation User Interface visually depicts accumulation of patients in desirable or undesirable states of care, like waiting to start a new service. These states are defined by their inflows and outflows. A lot of clinical teams and mental health leadership wonder whether they can get any new insights when they are already doing everything that they can to support the Veterans in their care. Would you like to know how to improve SAIL with Modeling to Learn? Watch that video to find out. "],["3-technical-documentation.html", "Chapter 3 Technical Documentation 3.1 Cheatsheets 3.2 Field User General Manual 3.3 Administrator Technical Manual", " Chapter 3 Technical Documentation 3.1 Cheatsheets 3.1.1 CPT Code Cheatsheet CPT Code Cheatsheet 3.1.2 MTL 3.7 Data Cheatsheet Data Cheatsheet 3.1.3 MTL 3.7 Sim UI Cheatsheet Sim UI Cheatsheet 3.2 Field User General Manual 3.2.1 Create an Account or Team Follow the instructions below to create an account or team in the Sim UI. Once the registration form is completed, a notice will be sent to MTL for approval. Once approved, the user will be notified by email they now have access to the dashboard. Tip: Save and refer to this one-page Sim UI Cheatsheet when creating an account or team! There are two ways to navigate to the registration form: Click on Create Account or Team in the Sim login window located at mtl.how/sim: Navigate to mtl.how/register in your Edge, Firefox, or Chrome browser. Complete the form as shown below: 1 - VISN: Select your VISN from the dropdown menu. 2 - Facility: Select your facility from the dropdown menu. 3 - Clinic: Select your clinic from the dropdown menu. This field is not required. 4 - Select Facility or Create New Team: If you are a team member: Select your existing team from the dropdown menu. If you are a team lead: Select Create New Team. If you are creating a team or the first person to register from your Facility Team, the following pop-up window will appear (with your VISN, Facility, and Clinic pre-populated). Enter your Facility Team name in the pop-up window (immediately after the pre-populated information). Notes: You must only use lowercase letters. You must use the underscore character (_) instead of spaces. Click the Submit button. Even though the dropdown field remains unchanged on the form, the submitted team name is saved, as confirmed by the green text as shown in this example: 5 - Select VHA Primary Role: Select your primary VHA role from the dropdown menu. Choices are: Clinician, Data Subject Matter Expert, Leader/Manager, Quality Improvement, Researcher/Evaluator, Systems Modeler, and Other. 6 - First Name: Enter your first name. 7 - Last Name: Enter your last name. 8 - VA Email: Enter your VA email. 9 - New Password: Create a password and enter it here. Tip: Write it down so you don’t forget! Note: Your password must contain at least 1 number. 10 - Repeat Password: Re-enter the password you just created. Tip: To check if both passwords match, click the eye with a slash icon in each field to reveal what you typed. Click the green Submit button. The Registration Detail pop-up will appear. Ensure the registration details displayed on the pop-up are accurate. If the information is not correct, close the pop-up window and make adjustements to the registration form where needed. If the information is correct, click the green Submit button. A success page will appear: After your registration request is reviewed and approved, an approval email will be sent to your VA address that contains login information for accessing the MTL Sim UI. 3.2.2 Log in to the Sim UI You must already have an account set up to log into the Sim UI. If you do not have an account yet, follow the Create and Account or Team instructions and wait to receive an approval email. Navigate to mtl.how/sim, enter your credentials, and click the Login button. A Please select a World pop-up will appear. The Sim will load teams and, once you select a team, you will be directed to that team’s Home page. Notes: The login dropdown only shows worlds in which you are a team lead or participant (not an Administrator). If you have the Administrator role in any team and have been added into “administrator_login” in Epicenter, you will be allowed to login using the “administrator_login” choice. Upon logging in, Administrators are directed to the Administrator Dashboard. The screenshots below display what you will see upon logging in, depending on your role. Note: To familiarize yourself with a team’s Home page, refer to Orient to a Team’s Home Page Team Member login = team’s Home page Administrator login = Admin Dashboard 3.2.3 Make, View, and Export Team Clinic Selections for Import to Sim UI Tip: Refer to (or save) the one-page Data Cheatsheet when using the Data UI and making clinic and flow selections. The general workflow for the Data UI is that teams: Select their clinics and team flow (if applicable). Review and look at the data and viz tabs to observe trends in the clinics. Generate the Data UI file from Power BI to export into the Sim UI. Notes: All users must follow certain naming conventions, which are explained below, when downloading Team Data files. When a Team Data file is downloaded from the Data UI, it is in a specific format that the Sim UI reformats during upload. This means that if a Team Data file is downloaded from the Sim UI and then re-uploaded to the Sim UI it will not work! Clinic selections and data tabs refresh every night. Viz tabs update monthly at the beginning of the month. Open Chrome, Edge, Safari, or Firefox browser and go to mtl.how/data. Using the left-hand navigation pane, navigate to the desired Region and VISN. Click on the Clinic Selection tab and select and view your local team data. Filter by Division Name, Physical Location, PrimaryStopCode, SecondaryStopCode, and/or Is an Active Clinic? (de-activated clinics denoted by ZZ) in the left and middle columns. Search and filter for your team’s clinics by clinic name or by provider in the Select Clinics to Generate Report Data column after making selections in the left and middle columns. Click the green Apply button to pull Patient Level Data from all clinics selected in the last two years. Select clinics that your team refers to in the Team Flow Selection tab to produce data tracking patients stepped up/down for the Team Flow module. For example, if you are a GMH team, you would select SMH in the dropdown for “The clinics selected below that my team refers to are” and add the SMH clinics your team refers to in the Green column, second row. In the bottom right-hand columns for the three possible care settings (GMH, SMH, PC/PCMHI) indicate the threshold for minimum gap in patient care required for subsequent visit to be considered a new care episode. In general, for the (GMH or SMH or PC/PCMHI) setting, how long of a gap in patient care is required for the subsequent visit to be considered a new care episode? Set the threshold below in months. The default threshold is 9 months (39 weeks). Click Apply. Notes: Are the clinics and data not populating? Request and verify your local server views by following steps outlined in Request and Verify Local Server View Permissions. If you wish to have quick access to these Clinic Selections in the future, you can make a personal bookmark to return to it. Go to the Bookmarks drop-down menu and click on Add a personal bookmark. In the bookmark name field, use Team PSD convention for the name: Sta3n_Division Name_Date (date in yyyy_mm_dd format), then select Save. Scroll until you see the View Clinic Selections tab on the left-hand side. Hover over the Clinic Name > Providers heading and click on the ellipsis that appears. Click Export data > Data with Current Layout > Export. The Team Data file is exported to your browser’s download location. Save using our Team naming convention “Sta3n_geo-location_clinic_year_month_day” and post the file on the shared Team channel for Team members to view and confirm selections. Repeat steps 2-6 with Team Flow Selections. Tips & Tricks: Link to CPT Code Cheatsheet. Use the most recent clinic selection table to find the full Sta6a. The folder for the site list xlsb file is located in the TeamPSD shared drive > Grants > 0_final_post_covid19_site_list. Note: This site list is for SMEs and the internal team; it is not applicable for field users. To clear filters/columns, click on the Eraser icon in the upper right of the filters/columns. To reset ALL selections across ALL tabs, click the Reset button in the upper right hand of the bar. To zoom into the screen, hover in the upper right-hand corner of the screen until a set of icons appear. Click on the square Focus Mode icon. Click Back to Report to return to the previous view. Note: Focus Mode is found in the data and viz tabs only. Click to view the vizDiag tab which shows team trends. Viz tabs are your reference modes - trends over time. Zoom out and see team trends for diagnoses (Diag), encounters (Enc), health factor data (HF; e.g., suicide plans, evidence-based practice templates), measures (Meas), and behavioral and suicide risks (Flag) from Mental Health Assistant. Click on the respective tabs (Diag, Enc, etc.) to view these trends. Another way to see encounters or services decreasing or increasing is by viewing the number of visits by right clicking on any trend line and click Show As Table. Filter trends to zoom into a specific team trend by navigating to the right side of the screen and click on the Funnel icon to reveal the filter panel. vizDiag in the Data UI shows a trend lines by diagnoses or combination of diagnoses. For example, the PTSD line is for patients with only PTSD. The PTSD & AUD line is for patients with both these diagnoses but does not discriminate between which is the primary, secondary, etc. diagnoses. Tips & Tricks: Right click on any trend line and click Show As Table to see encounters or services decrease or increase by viewing the number of visits. Filter trends to zoom into a specific team trend by navigating to the right side of the screen and click on the “Funnel” icon to reveal the filter panel. If you do not see something that you would expect or previously saw, check to see if a filter is applied. Click on the funnel icon in the top upper right. Filters are active if it says “Filters and slicers affecting this visual.” Click to view the “data” tabs, which show your team’s individual patient information. In the data tabs, providers can zoom in on specific veterans. Patient information corresponds to the same categories as the team trends: diagnoses (Diag), encounters (Enc), health factor data (HF; e.g., suicide plans, evidence-based practice templates), measures (Meas), and behavioral and suicide risks (Flag) from Mental Health Assistant. Providers can filter to find specific patients or produce reports by navigating to the right side of the screen and clicking on the “Funnel” icon to reveal the filter panel. Tips & Tricks: dataDiag in the Data UI has an additional column after Diagnoses of Interest, called Primary Diagnoses, specifying which diagnosis is primary to help providers better understand the characteristics of the patient population. Patients who have requested restricted access to their information have asterisks (******) in the Patient Name column. If you are a VA provider, you can still identify patients from the Patient SSN column, or Patient Name column. If you do not see something that you would expect or previously saw, check to see if a filter is applied. Click on the funnel icon in the top upper right. Filters are active if it says “Filters and slicers affecting this visual.” 3.2.4 Request and Verify Local Server View Permissions To pull patient and clinic level data, you need LSV permissions. Check which facilities you have LSV access for here through VA intranet. Clinical Resource Hub Providers are granted to multiple local server access. If you do not have access to facilities that you believe you should, please verify: CPRS access/permissions at your station (i.e., COR tab access for CPRS). Your network username recorded in VistA. To have your network username recording in VistA, please follow instructions here through VA intranet. If you are still unable to pull data or access the Data UI, please email us at MTL.Help@va.gov Note: If you need access to all sites (e.g., national), click here and follow the steps in Request Access to Data Tools section. 3.2.5 Request Access to Data Tools Click here to access the OI&T Electronic Permission Access System. Verify VHA NDS Access Form for Health Operations is selected in the Select Document dropdown, then click Create Document. Fill in the form as follows: Note: the following fields will be pre-populated: Full Legal Name, Job Title, Email Address, Active Directory Name with Domain, and Work Telephone Number. 1 - Employee Agency: Veterans Health Administration 2 - Employee Type: VA Employee (yes, for all of you, none of you are “contractors”). New fields will appear. 3 - Lookup Supervisor: Search for “zimmerman, l”. 4 - Purpose of Use: Healthcare Operations 5 - Data Sources: Select Corporate Data Warehouse (CDW) 6 - Duration of Access: Select “I will need access during the duration of my position.” from the dropdown. 7 - Affirmation: Check the box. Scroll towards the top of the screen and select the Corporate Data Warehouse (CDW) tab. Make the following selections after selecting the CDW SQL Datasets checkbox: Basic Read Access (CDW_Full) Privileged Read patient Access (CDW_SPatient) Staff Real SSN Access (CDW_SStaff) MCA (DSS) Access (CDW_DSS) Outpatient SQL Files (CDW_SE_IE_SF) Now select the Comments tab. Copy/paste in the following rationale provided to you: I support the Modeling to Learn national operational quality improvement project led by the VA Office of Mental Health and Suicide Prevention (OMHSP). This initiative includes a national Power BI Report in which frontline staff access local operations data as part of their quality improvement activities. Click Save and Submit. 3.2.6 Navigate to the Home Page of Any Team Tip: Refer to (or save) the one-page Sim UI Cheatsheet when familiarizing yourself with the Home Page and Dynamic Decision Dashboard. Upon logging in to the Sim UI, field users will be automatically directed to the selected team’s Home page. Notes: Team PSD members will have uploaded the same Team Data file into the _team world for everyone at that consult location. Learners can see the same accumulations and learn where to prioritize based on the location of the greatest accumuation to leverage for change. Although learners see the same information, it is still your own unique session (behind the scenes). There are ways to play in the same world and learn together within each module: Care Coordination, Team Care, Medication Management, Psychotherapy, and Team Flow. Administrators can access a team’s Home page via the Administrator Dashboard. Select a team from the Select Team dropdown, then click Home (see screenshot below). The Home page of the selected team will appear. 3.2.6.1 Orient to a Team’s Home Page Below is a basic orientation to the Home page of any team: MTL logo: Click this logo to return to the Dynamic Decision Dashboard. Team Name: The selected team name appears here with the world extension added at the end of its name. If you are logged in as an Administrator, you will see “Administrator” here. Home: Click to return to the Dynamic Decision Dashboard. Play: Click to return to the Play page. More Info: Click for more information on how to use the Sim UI. News: Click for the latest news about Modeling to Learn. An alert will appear if you have unread news items. Report Bug: Click to report a bug. Tutor: Click any Tutor icon for an “over the shoulder” overview. Logoff: Click to log off of the Sim UI. Avatar: Click to upload your profile picture. Session Start window: You can either join the simulation session in progress or start a new one. Select the team data file you wish to use from the dropdown menu, then press GO! Experiment window: This section allows you to delete, rename, export, and sort your team’s simulation experiment results. The experiments listing shows your current team first. Notes: Administrators can see all experiment data across all worlds, organized by associated groups (team + world), so they are able to associate experiments with their respective groups. New Administrators can access all teams, data, and experiments—without being manually added to any other teams. Team Data window: This section allows you to delete, upload, and sort team data files (which will then show in the Start a Session window). Legacy team data files are located under the MTL 3.0 gray bar. Dynamic Decision Dashboard: Use this section to navigate between the 5 modules and their associated variables. Community: Click to give feedback to system administrators and participate in our MTL Community of Practice. 3.2.7 Begin a Session in the Dynamic Decision Dashboard Tip: Refer to this one-page Sim UI Cheatsheet when familiarizing yourself with the Dynamic Decision Dashboard. Beginning a session in the Dynamic Decision Dashboard (DDD) allows users to look at the stocks and flows across all five modules. It also helps determine which module(s) to prioritize, based on the location of the greatest accumulation to leverage for change. Learners can see the same accumulations and learn where to prioritize based on the location of the greatest accumuation to leverage for change. Navigate to the selected team’s Home page. If you aren’t sure how, see Navigate to the Home Page of Any Team. Select the desired team data file from the Please Select Team Data dropdown and click GO! A Learning Mode pop-up will appear. Tip: If multiple members of a team want to review the DDD at the same time, it is highly recommended to take turns initializing (pressing Go!). Otherwise, some users may not be able to successfully initialize the DDD modules. Notes: Team PSD members will have uploaded the same Team Data file into the _team world for everyone at that consult location. Learners can see the same accumulations and learn where to prioritize based on the location of the greatest accumuation to leverage for change. Although learners see the same information, it is still your own unique session (behind the scenes). There are ways to play in the same world and learn together within each module: Care Coordination, Team Care, Medication Management, Psychotherapy, and Team Flow. If there is not a team data file in the dropdown, follow these steps. Make CC and MM Learning Mode selections using the radio buttons, then click Continue to initialize all modules within the Dynamic Decision Dashboard. Notes: Existing Patients Return-to-Clinic Visit Interval (EPRVI) = learning mode “0”. New Patient Start Rate (NPSR) = learning mode “1”. Refer to the Care Coordination (CCParams) or Medication Management (MMParams) tab of the Team Data Table at mtl.how/data, for data regarding the team’s New Patient Start Rates and Return-to-Clinic Visit Intervals. 3.2.7.1 Upload Team Data Files from the Home Page Expand the Team Data blue bar and select Upload: A pop-up window will appear asking if the data file is in Power BI format. Ensure it is and click Yes to proceed. The File Manager will appear. Navigate to the location of the downloaded team data file and click on the file name. Click Open to upload the file. Be patient; it may take a while for the file to upload. Notes: Be patient; it may take a while for the file to upload. As the data file is uploaded, you will see “Data is uploading…” and three flashing loading bars in the Session Start and Team Data sections. These messages clear once the file is loaded into the Sim. The team data file name should now appear in the Team Data list and in the Please Select Team Data dropdown. Return to Begin a Session in the Dynamic Decision Dashboard, Step 2. 3.2.8 Dynamic Decision Dashboard The Dynamic Decision Dashboard (DDD) is a convenient visual way to help answer the question Where is the volume (of Veterans) across all modules? The DDD is for everyone (clinicians, consultants, nurses, etc.) to test their assumptions and hypotheses. The primary purpose of these visuals is to help understand the dynamic system that is Veteran care. The dashboard shows a visual representation of where Veterans accumulate (creating lags in the system), where Veterans are waiting to start, and the services in which these behaviors are observed. This aids in understanding how different quantities change and interact over time and identifying key variables that contribute to the accumulation or depletion of stocks. The DDD allows clinicians to make new clinical decisions that are among their most common decisions, resulting in high-leverage change(s) that are the most influential decisions that impact the behaviors of the Veteran care system, and empowers teams to understand how to zoom in to get a particular Veteran the right treatment at the right time or zoom out to understand how all the individual decisions aggregate over time to determine the dynamics of the system. 3.2.8.1 Understand the Five Modules There are five modules within the Dynamic Decision Dashboard, each of which is described below: Care Coordination: shows how a variety of team decisions interact to impact multiple outcomes at the same time. You can explore trade-offs between decisions such as desired new patient wait times, current patient return visit intervals, and overbooking, and how they affect team outcomes like total missed appointments, number of patients in care, and number of patients completing care. Team Care: gives insights about RVI and service proportions, and shows how a variety of team decisions interact to affect your overall mix of services. You can explore trade-offs between decisions such as desired new patient wait times, current patient return visit intervals, and overbooking, and how they affect team outcomes like total missed appointments, number of patients in care, and number of patients completing care. You can compare the effects of different decisions on outcomes in one service, or you can compare the effects of one decision on different services. Medication Management: enables you to experiment with how team decisions about referrals, time allocation, and visit frequency impact the quality of medication management for patients. Your team can explore scenarios to balance these trade-offs and better meet the specific needs of patients with depression, alcohol use disorder, and opioid use disorder. Insights you gain can help you increase local reach of evidence-based pharmacotherapies. Psychotherapy: shows how team psychotherapy decisions interact and depicts patterns of psychotherapy engagement in your team. You can explore solutions to the challenges of getting patients to complete a full course of therapy, while also achieving goals to start enough new patients in psychotherapy. Insights about these dynamics can be used to increase the local reach of evidence-based psychotherapies. Team Flow: shows the effects of measurement-based stepped care on patients’ symptoms and risk. It allows you to explore the impacts of implementing measurement-based care to reduce delays in detecting patients at high risk for suicide, and to improve the quality of care by making better team decisions about when to step patients up to a higher level of care, or step them down to a lower level of care. It is also possible to experiment with team decisions related to new patient wait times and access, the use of community care, and the impacts of provider overwork and burnout on the quality of care. 3.2.8.2 Orient to the Dynamic Decision Dashboard Below is an orientation to the Dynamic Decision Dashboard. Tutor: Click for a guided tour of page features. Caret: Click carets next to each module to contract or expand that module from view. Directives: Click to show a Directive 1160.01 to MTL crosswalk table that identifies specific references within Directive 1160.01 that requires actions by facilities that are supported by the Data UI and the Sim UI. These actions are linked to MTL reports and MTL use cases that reinforce how the directives are supported in application by other facilities. More: Click to reveal other key views of the Care Coordination module. All modules have this feature. Depending on the module selected, the More menu may include: 1 - Feedback Loops: Interactive diagrams that aid in visualizing how different variables in a system are causally interrelated. Clicking here will open a pop-up window that shows an associated loop diagram along with Causal Loop Dynamics selections. 2 - Flow: Visualized in Sankey form, these diagrams help visualize the rate of flow into and out of stocks. A stock is a state of care in which patients can accumulate over time. Sankey is a type of flow diagram where the width of the arrows is proportional to the flow quantity. 3 - Decision Meters: A quick method to visualize the relevance or significance of different aspects within a group. 4 - Episodes of Care: Graphically shows evidence-based episode of care intervals to get a bird’s eye view of how long the length of engagement will be. 5 - Play: Enter the main Sim UI page to see the system story and run experiments. You can learn together if you click Play and join an ongoing session in one of the modules. 6 - Team Data: A quick link to the Modeling to Learn Data User Interface that will open in a new browser tab. 7 - SAIL Data: A quick link to Strategic Analytics for Improvement and Learning Value Model (SAIL), a system for summarizing hospital system performance within Veterans Health Administration (VHA). This will open in a new browser tab. Learning Mode Labels: Indicates which Learning Mode the data prioritizes. Click these buttons to change the learning modes directly from the DDD; the system will re-initialize based on the new learning mode selected. Existing Patients Return-to-Clinic Visit Interval (EPRVI) = learning mode “0” and New Patient Start Rate (NPSR) = learning mode “1”. Team Care Services Radio Buttons: Select desired service radio button to see stock values for specific services in Team Care (Intake, Care Coordination, Medication Management, Psychotherapy, Evidence-based Psychotherapy, Adjunctive Services, Group Therapy). The default display is Intake evaluations. Medication Management Services Radio Buttons: Select desired service radio button to see stock values for medication-specific services in Medication Management (Alcohol Use Disorder, Depression, Opioid Use Disorder, and Other). The defaut display is Alcolol Use Disorder. “i” icon: Click “i” icons to read the definition of each stock label. Team Flow Services Radio Buttons: Select desired service radio button to see stock values for setting-specific services in Team Flow (General Mental Health, Specialty Mental Health, Primary Care/Primary Care Mental Health Integration). The default display is General Mental Health. Community: Click here to participate by sharing an idea, letting the team know what you like, asking a question, or joining our community. 3.2.8.2.1 Feedback Loops A systems view and understanding is reinforced through the use of funcational causal loop diagrams accessible directly from the DDD. These diagrams are interactive and help visualize how different variables within the system of veteran care are causally interrelated. Feedback loops involve the output of a system being fed back into the system as input. They are essential for understanding and analyzing the behavior of dynamic systems. Each story has its own unique feedback loop that shows named balancing and reinforcing feedback loops and the whole system story with polarities. For illustrative purposes, the Team Flow Story is used as an example below (with all causal loop dynamics shown) to orient you to general feedback loop window features. Note: Click and hold the blue bar at the top to move the pop-up to a desired position on the screen. Tutor: The Tutor button will introduce you to the features of this page. Key: This displays the key for all feedback loops. Positive Causal Arrow: Values at the arrow’s tail (cause) and point (effect) change in the same direction; both increase or both decrease. The plus sign means it is a direct relationship. Negative Causal Arrow: Change in the value at the arrow’s tail (cause) make the value at the point (effect) change in the opposite direction. The minus sign means it is an inverse relationship; when one increases, the other decreases. Widget: Use the checkboxes to show or hide different loop dynamics. Link polarities Checkbox: Select and deselect to show or hide link polarities; the diagram will build as defined. Loop polarities Checkbox: Select and deselect to show or hide loop polarities; the diagram will build as defined. Increase or decrease Checkbox: Select and deselect to show or hide increase and decrease polarity arrows; the diagram will build as defined. Widget Dropdown: Click here to show or hide the widget. Polarity Arrow: When these arrows are displayed, users can select any of the arrows by clicking and flip the increase or decrease icons. All the icons in the loop will then flip according to their relationship. R Reinforcing Loop: This is a reinforcing loop; it reinforces runaway increase or decrease - virtuous or vicious behavior. B Balancing Loop: This is a balancing loop; it keeps things in balance - stable or stubborn behavior. Window Size: Click and drag to resize this window. Exit: Click to close this window. 3.2.8.2.2 Decision Meters Decision Meters are a quick infographic-based way to identify the highest leverage points and are highly supportive of clinical decision-making; they are for “decider” users who need systems thinking insights. The meters provide a visual interface that provides real-time feedback on ranges in our primary MTL units of patients/week and appointments/week. The traffic light reflects the current state results and recommendation of future state results. Through dialogue with decision-makers, consultants and field users manually input values into this graphic generator so the decision meters reflect the recommendations based on the alternative scenarios/experiments run by the learner. Decision meter numbers should be reflected in the “D” box of the Q/H/F/D outputs text field within the Sim UI, usually as ranges of numbers. Care Decision Meters are for MTL Blue teams only. Click within each decision meter to edit. Remember to click Save if you want the values to remain! For illustrative purposes, the Medication Management Decisions window is used as an example below to orient you to general Decisions pop-up window features. Tutor: The Tutor button will introduce you to the features of this page. Service Options: Click a radio button to see stock values for specific services. Hovering over the abbreviations will display the entire service name. Patient and Appointment Flow: Suggested Patient and Appointment flow. Episodes of Care: Suggest treatment length and frequency. Red: Click in the shaded text field of any red circle (greater net inflow to outflow) to input a value (up to 4 digits). Yellow: Click in the shaded text field of any yellow circle (equal inflow to outflow) to input a value (up to 4 digits). Green: Click in the shaded text field of any green circle (lesser net inflow to outflow) to input a value (up to 4 digits). Save: Click to save the entered values. Exit: Click to close the pop-up window. 3.2.8.2.3 Episodes of Care Timely access to high-quality care includes improved understanding of dynamic tradeoffs in the Return-to-Clinic Visit Interval (RVI) and engagement durations. The Episodes of Care pop-up is used to graphically show evidence-based episode of care intervals to get a bird’s eye view of how long the length of engagement will be. Available within the Medication Management, Psychotherapy, and Team Care modules, this pop-up integrates engagement duration and RVI for both Medication Management and Psychotherapy. The following screenshot will orient you to general features of this pop-up. Tutor: The Tutosr button will introduce you to the features of this page. Medication Management example: Medication Management example shown side-by-side with Psychotherapy to illusttrate a tradeoff in service proportions being used. Psychotherapy example: Psychotherapy example shown side-by-side with Medication Management to illustrate a tradeoff in service proportions being used. “i” icon: Click to view the CPT Code Cheatsheet in a new browser tab. Time Box: Each rectangle represents one month. Appointment: “X” represents a Return-to-Clinic appointment. Exit: Click to close the pop-up window. 3.2.9 Enter and Save an Experiment This section explains how to enter and save an experiment in the Sim UI. Tip: Save and refer to this one-page Sim UI Cheatsheet when running an experiment! 3.2.9.1 Start a Play Session in the Sim UI From the team Home page, use the Please Select Team Data dropdown to select your team data file, then click GO! A Learning Mode pop-up will appear. Make the appropriate selections, then click Continue. Note: For CC and MM, select your Learning Mode before hitting Play. Refer to the Care Coordination (CCParams) or Medication Management (MMParams) tab of the Team Data Table at mtl.how/data, for data regarding the team’s New Patient Start Rates and Return-to-Clinic Visit Intervals. To run a simulation for any module within the dashboard, select Play found within each module’s More menu. Tip: You can learn together if you click Play and join an ongoing session in one of the modules. In the Text pop-up window, populate the Our Question, Our Hypothesis, Our Findings, and Our Decisions text fields. Click Save. A Save Experiment pop-up will appear. Click the Save icon on this pop-up. A Scenario saved successfully message will appear. 3.2.9.2 Orient to the Sim UI Tip: Refer to the Sim UI cheatsheet to use when running an experiment. Below is an orientation to the Play Page of the Sim UI. 1: Module name. 2: Team Vision. 3: Reveal Dynamics over Time: Reveal more dynamic relationships by clicking the checkboxes. Click Run to simulate outcomes. 4: Outputs & Text: View 6 output charts. Click on expand “T” to open Text (pin #9). 5: Experiments Section. 6: Select previous runs to set experimental sliders and Questions, Hypothesis, Findings, and Decisions (Q/H/F/D) text to that run. 7: Team Data table shows starting values for variables from your data. 8: Move Sliders from initial values and run. 9: Text: Enter Q/H/F/Ds. Click on expand icon in blue bar for full functions. 3.2.10 Resize a Window for Side-by-Side Viewing An administrator may need to show two items on the screen side-by-side for comparison. Field users may also want to view two items on the screen for comparison or reference. The Sim UI has a responsive design that should accommodate many resolutions and browser window sizes. Follow the steps below to get the best view. Click the “window” icon in the upper right of the browser window. Drag the browser window to the desired size. Notice how the Sim UI design responds by stacking top navigation menu icons and other page contents. Sometimes items on the screen are cutoff or not formatted correctly. Click the refresh button so the browser can detect the new window size and adjust the format. If the format is still not optimum, decrease the zoom below 100% to reduce the size of the items on the screen, or drag the screen slightly larger. 3.2.11 Report a Bug There are two ways to report a bug; through the Sim UI and directly through GitHub. One must have access to either the Sim UI or GitHub in order to report a bug. 3.2.11.1 Report a Bug through the Sim UI From the Home page or from the Play page, click on the Report Bug icon. Describe the issue in the field provided (please be detailed as possible). Do not include PII or other Site identifiable information. Instead, use generic language. If you have a screenshot you want to share, click the checkbox. Click Submit when finished. If you clicked the screenshot checkbox, an additional dialogue will appear to help you upload your picture. Be sure there are no PII in the screenshot! Click Submit again when finished. The Sim UI will report the bug automatically to GitHub. 3.2.11.2 Report a Bug Directly in GitHub To report a bug directly in GitHub, navigate to mtl.how/teampsd to get started. The page below should appear. If not signed in, sign in. Then click the Issues tab. Click “New Issue” button. Click “Get Started” in the “Issue” swim lane. Complete the Issue Form. Do not include PII or other Site identifiable information. Instead, use generic language. Route the form to the “bug_tracker” project for action by Team PSD. Be sure to check the “bug_tracker” is listed under “Projects” (see the second screenshot below). Click the “Submit new issue” button. 3.3 Administrator Technical Manual As a MTL Administrator, the following chapters will help answer any technical questions you may have during prep and live team meetings. 3.3.1 Login as Administrator Navigate to mtl.how/sim. Log in using your credentials and select the administrator_login world. The Sim UI Administrator Dashboard will appear. Note: For Administrators, the login dropdown only shows worlds in which you’re a Team Lead or Member (not Administrator). 3.3.2 Sim UI Administrator Dashboard This section provides familiarization with the Administrator Dashboard and explains how to use this dashboard to set up and manage teams. If you have the Administrator role in any team and have been added into the “administrator_login” in Epicenter, you will be allowed to login using the “administrator_login” dropdown choice. Tutor - Click for a guided tour of page features. Settings - Click to set up a team. It provides team setup input screens and a means to load Team Data. See the Manage Teams section for more information. Team Statistics - Lists experiments, when they started and ended, total time-on-task, team size, attendance percentage, and completion percentage. Team Roster - Lists team members, their clinical role, and team role. MTL Manual - Click to open a comprehensive program and technical guide about Modeling to Learn. Home - Click to go to the selected team’s Home page, which also contains the Dynamic Decision Dashboard. Select Team - Use this dropdown to select your desired team. It enables you to monitor their experiments and navigate to the team’s Home page. Note: This dropdown only shows groups to which you belong (not ones in which you have the Administrator role). Gauges - This indicates the team’s progress with a selected experiment. Teams receive 20% completion for a minimum 50 characters input for each of the Question, Hypothesis, Findings, Decisions fields and another 20% for running the experiment. Select Experiment - Use this dropdown to select the desired experiment to monitor. An experiment will not show in this dropdown until it has been named and saved. Expand / Collapse Dropdown Triangle - Manage and monitor multiple teams by selecting this dropdown. CLOSE - Click to de-select a team. ADD TEAM - Click to add an unlimited number of teams to the dashboard. 3.3.3 Manage Teams This section describes how to set up a team from the Administrator Dashboard. Once team setup is complete, users can access the team’s Home page which contains the Dynamic Decision Dashboard. Click the Settings icon to begin setting up a team. Within the Facility Team Setup area of this pop-up, several team setup options are available and are described below: Add - Click to add a team. Edit - Select a team first, then click here to edit that team. Delete - Select a team first, then click here to delete that team. Note: Teams shown under Facility Team Setup are ones you have joined and in which you have the ‘Administrator’ role. Teams in which you have ‘Lead’ and ‘Member’ permissions will not be visible. 3.3.3.1 Add a Team Click the Add icon. A Setup pop-up will appear. Add the team name using the convention “visn_facility_clinic_facility team name”. Just add the team’s root name. The setup function will automatically add “_team”, “_ind”, and “_shared” extensions in Epicenter. Use only lowercase letters. Use the underscore (_) character instead of spaces. Add the team’s consult vision. This field is required. Click the Add icon to add a team member. A Setup - Add Members pop-up will appear. Enter First Name, Last Name, and Email. Use the Select Permissions dropdown to assign the team member’s permission level on the team. Use the Select VHA Primary Role dropdown to assign the team member’s role on the team. Click the Add icon. Note: Click the checkbox next to a name and then select Edit or Delete to update information or delete a team member. Click the Done button when finished. Notes: If your newly created team (and its associated worlds) do not show in the Administrator Dashboard’s Select Team dropdown, reload the page. The Sim auto-adds members of our internal team as administrators to newly created teams. This is in addition to the default team lead. Keep the auto-added individuals shown below as-is. 3.3.3.2 Edit a Team Click the checkbox next to the team you want to edit, then click the Edit icon. An Edit Setup screen will appear. Within the Edit Setup screen, you may add, edit, or delete team members from your selected team. 2a. To add a new team member, click the Add icon. A Setup - Add Members pop-up will appear. Enter First Name, Last Name, and Email. Use the Select Permissions dropdown to assign the team member’s permission level on the team. Use the Select VHA Primary Role dropdown to assign the team member’s role on the team, then click the Add icon. Notes: Adding a member to _team will automatically add them to _shared and _ind. Adding a member to _shared will not impact _ind or _team. Adding a member to _ind will not impact _team or _shared. 2b. To edit a team member’s information, select the checkbox next to the team member’s name, then click Edit. A Setup - Edit Members screen will appear. Make changes as needed to the First Name, Last Name, Permissions, and/or VHA Primary Role fields. 2c. To delete team members, select the checkbox(es) next to the team member(s) you want to remove from the team, then click Delete. Note: More than one team member can be deleted at a time. A confirmation message will appear; select No if you wish to make further changes, select Yes to confirm deletion of selected team member(s). Click Done when finished editing your team(s). 3.3.3.3 Delete a Team Click the checkbox(es) next to the team(s) you want to delete, then click the Delete icon. A confirmation message will appear; select No if you wish to make further changes, select Yes to confirm deletion of selected team(s). 3.3.4 Manage Team Data Note: Only administrators and the internal team have access to these functions. Field users can upload their data file(s) from their team’s Home page (as well as administrators and the internal team). Field users should refer to the Dynamic Decision Dashboard section. Administrators can upload team data files and assign that file(s) to team(s) of their choice. Please note these specifics about the behavior of team data files: When a team data file is uploaded into a _shared or _team world, it will be available in all worlds (_ind, _team, and _shared) for all users of those worlds. When a team data file is uploaded to an _ind world, it is accessible exclusively to the _ind world only. When a team data file is deleted in a _shared or _team world, they are deleting it from _shared and _team worlds for all users of those worlds. This will not impact _ind worlds. When a team data file is deleted in an _ind world, it will be removed from the _ind world exclusively. 3.3.4.1 Upload a New Team Data File to a Team From the Administrator Dashboard, click Settings. Within the Data side of this pop-up, click Add. A Please select a world pop-up will appear. Select the world(s) in which you want the team data to appear and click Upload. A pop-up window will appear asking if the data file is in Power BI format. Ensure it is and click Yes to proceed. The File Manager will appear. Navigate to the location of the downloaded team data file and click on the file name. Click Open to upload the file. Be patient; it may take a while for the file to upload. Notes: The upload routine must update multiple GitHub repositories and Epicenter, which may take a few minutes. If the file does not appear in the Data side of the Settings pop-up, try clicking the Refresh button. Do not attempt to upload team data file directly to Epicenter, since the GitHub repository will overwrite the files on the next update and they will not work when you try to access them from the Sim UI. Excel (.xlsx) data files exported from the MTL Data UI (Power BI), or uploaded from your MTL MS Teams, are in the correct format. Downloading and re-uploading data files from the MTL Simulation UI will not work. 3.3.4.2 Delete a Team Data File From the Administrator Dashboard, click Settings. Within the Data side of this pop-up, select the team data file you want to remove, then click the Delete icon. Notes: As a reminder, When a team data file is deleted in a _shared or _team world, they are deleting it from _shared and _team worlds for all users of those worlds. This will not impact _ind worlds. When a team data file is deleted in an _ind world, it will be removed from the _ind world exclusively. A message will appear, asking you to confirm deletion of the selected file. Select No if you want to cancel this action. Upon successful deletion, a green message will display in the upper right-hand corner of the browser window. 3.3.5 Modeling to Learn Maintenance Dashboards The MTL Maintenance Dashboards provide customizable information and messaging capabilities for webpages in the Sim UI. Upon logging into the Sim UI as an Administrator, the MTL Maintenance Dashboards icons are viewable along the header bar of the Sim UI. 3.3.5.1 Orient to the Maintenance Dashboards The following screenshot will orient you to general features of the Maintenance Dashboards, located in the top navigation bar of the Administrator Dashboard. Community Function Administration: View lists of issues reported by users from the Report Bug and Community Button, and issue outgoing email messages and targeted ANDON Alerts. More Information Page Administration: Manage images and reference links in the Information Page. Add and edit Module Section information. Add and edit Dynamic Decision Dashboard Directives. Tutor Administrative Dashboard: Manage text descriptions and web links for tutor prompts for the Home Page, each module, the Dynamic Decision Dashboard, and each module’s story. MTL Quick Tip Loading Screen Administration: Add icons and descriptive text for tips that appear when MTL screens are loading. MTL Demo Webpage: Manage the MTL Demo website, such as adding course codes, managing Demo user lists, and managing page content such video and reference links, text descriptions, and others. Team PSD Webpage: Manage the Team PSD website, like adding award winners, adding new team member biographies and photos, or changing page content. 3.3.5.2 Community Function Administration Name of user submitting the issue. Email of user submitting the issue. GitHub issue number assigned by GitHub API. Description of the issue. Click the “Export” button to download an spreadsheet file of the list. 3.3.5.2.1 User Issue List Management Use this section to review issue information. The listing only contains a text description of the issue. Sometimes users submit files or screenshots. To view these details: Click the checkbox next to the issue. Click “See Issue” button. Review information in the pop-up. 3.3.5.2.2 Outgoing Message Management The Sim UI sends out email and other messages to mass communicate important information. There are 5 types of messages. I have an idea: This is an outgoing email that informs the user that we have received their idea. I like something: This is an outgoing email that informs the user that we have received and appreciate their message. Report Bug: This is an outgoing email that informs the user that we have received their bug report and will act on it. It also returns an issue number for their reference. News: Every user has a News button on their Home webpage. When clicked, it will display news items on their webpage that were set up with this function. Alert: Also known as an ANDON alert, this is an outgoing message that displays on the Home webpage when the user logs in. It can target administrators only, users only, or both. ANDONS are normally reserved for problems where the user should stop using the Sim UI until the issue is resolved. 3.3.5.2.2.1 Target ANDON Alert Users As explained in 5. above, you can target specific users with an ANDON Alert. When Alert is selected from the Message dropdown, an additional Show Message to Whom will appear. Select the target audience and click Add Message. 3.3.5.3 More Information Page Administration When a user clicks the More Info icon in the their header bar within the Sim UI, they navigate to a page that contains information about the Sim UI. There are multiple sections: MTL Guides, MTL Links, MTL Navigation, MTL Modules, MTL Sections, and MTL References. While the sections of this page are permanent, you can change elements inside the sections using the Administration page, accessed by clicking More Info from the Administrator Dashboard. 3.3.5.3.1 MTL Guides and Navigation Pages You cannot change the MTL Guides or Navigation pages. 3.3.5.3.2 MTL Links and References Pages Below is the MTL Links page. When the user clicks on this graphic, they navigate to the mtl.how website. This is the text instruction. Below is the MTL References page, a bibliography that supports the research being conducted by MTL. 3.3.5.3.2.1 Update Links and References Below is the Links and References maintenance saection of the More Info Admin page. This is the title bar. This is the sequence that the link will appear on the page. These can be modified by changing the number. Note: All numbers will need to be updated to reflect the new sequence. This identifies the section you’re working in: Links. This is the file name of the graphic file. Upload the graphic file here. A thumbnail will show when it is uploaded. Note: Please use a .png format graphic. The explanatory text for the graphic is shown here. Slightly offscreen is where the url for the graphic goes. This identifies the section you’re working in: References. This is the explanatory text for the bibliography entry. Note that it has several characteristics. You can boldface or italicize using markdown language. The text can have hyperlinks embedded using the ##Link1## code. The number immediately after “Link” indicates the position of the title of the hyperlink from the list. The hyperlink title then calls the url listed in order in this column slightly offscreen. Each entry can have up to six hyperlinks. To add a hyperlink, click on the “Empty” and a dialogue box will appear. Don’t forget to save! 3.3.5.3.3 MTL Modules Page The screenshot below shows a table with a listing of the modules offered in MTL. In case more modules are addedt, it is possible to expand this list. 3.3.5.3.3.1 Update or Add Module Information Click this button to add a section. It will appear as the last row on the list. This is the sequence that the link will appear on the page. These can be modified by changing the number. Note: All numbers will need to be updated to reflect the new sequence. This is the name of the Module which appears as a title over the descriptive text in the next column. This is descriptive text. Click to modify. Use markdown to emphasize boldface, italics and add bullets. These describe the layers or complexity reveals in the module. Click to modify. Use markdown to emphasize boldface, italics and add bullets. These are the experimental controls available in the module. Click to modify. Use markdown to emphasize boldface, italics and add bullets. 3.3.5.4 Tutor Function The Tutor is a function in the Sim UI that explains functions on the webpage to the user. Each section on the Home, Care Coordination, Medication Management, Psychotherapy, Team Care, and Team Flow pages have a Tutor icon. 3.3.5.4.1 Tutor Administrative Dashboard The Tutor Administrative Dashboard enables you to adjust descriptive text and links associated with each Tutor callout box. However, the user cannot add or remove Tutor callouts. These have to be added using computer code. See below for a description of these features. Each page in the Sim UI has Tutor callouts. This points to the call outs on the Home page. Below are Tutor callout listings for Care Coordination, Medication Management, Psychotherapy, Team Care, and Team Flow modules; the Dynamic Decision Dashboard; and Care Coordination, Medication Management, Psychotherapy, Team Care, and Team Flow stories. This is a unique row that enables you to adjust the description below Tutor Administrative Dashboard title. Otherwise, it is the description of the Tutor callout. This is the section on the webpage where the Tutor callouts apply. This is the function the Tutor is describing. This text will show up in the Tutor callout box. To edit, simply click on the text. Don’t forget to save your changes! When the user clicks the link button on this Tutor callout box, the computer will navigate the user to this url. To edit, simply click on the link. Don’t forget to save your changes! 3.3.5.5 MTL Quick Tip Loading Screen Administration When a screen loads in the Sim UI, a blank gray screen is presented. We use this opportunity to offer suggestions or “Tips” to explain functions in the Sim UI. We have an admin webpage to create new Tips, accessible by logging in as an Administrator, then selecting Tips on the Administrator Dashboard top navigation bar: Tips usually have graphics, which are white line drawings on transparent background saved in the .svg format. When paired with text, these present on the loading screen (see example below). A repository of quick tips icons are available in lzim/mtl/images/quick_tips_files. Please note that MTL Demo has its own tips management administration page. To access this, you must: Navigate to the MTL Demo admin page. Click the “Tips” icon.** Note: The steps to add, update, or delete quick tips are the same for the Sim UI and MTL Demo. Verify you are on the right page by looking at the URL. It will say “va-psd-sim/qtipadmin.html” if you are in PROD Tips and “va-psd-demo/qtipadmin.html” if you are in the DEMO Tips. Demo Tips Page Sim UI Tips Page 3.3.5.5.1 Add a Quick Tip From the Administrator Dashboard, click on the “Tips” icon. The MTL Quick Tip Loading Screen Administration page will open. Click the “Add Tip” button. The page will scroll to the bottom of the list and present a new row to input information. Click on the “upload” icon. An upload file pop up will appear. Navigate to your file and click “Save.” The filename should appear immediately to the left of the upload icon and a thumbnail should appear to the right. Click on the “Empty” text. A text field will present. Type in the description and click save (the blue checkmark). You are finished! Be sure to upload the .svg graphic you used to the lzim/mtl/images/quick_tips_files folder. 3.3.5.5.2 Update a Quick Tip Updating a Quick Tip is similar to adding one. See the screenshot below for more information. Remember to click the blue checkmark button (save) when finished! 3.3.5.5.3 Delete a Quick Tip To delete a quick tip, select the tip(s) to delete, then click the delete button. 3.3.5.6 Manage MTL Demo Webpage The MTL Demo webpage provides interested users access to a demonstration simulation. The webpage also offers information about the research project, associated training and video materials, and highlights partners that had a hand in its development. The admin website enables the admin to adjust webpage content and demo users. 3.3.5.6.1 MTL Demo Webpage Navigate to MTL Demo to view example below. To modify this webpage, click on the MTL Demo icon from the Administrator Dashboard. 3.3.5.6.2 Orient to the MTL Demo Admin Webpage Export User Data section lists users that have registered for access to the MTL Demo simulation. It lists their names and contact information. The typical registration is for 14 days. However, with a Course Code, the length of the registration can be longer. A user’s access time can be improved, but it must be done before the original date expires. Resource Management section manages the information contained on the MTL website. The sections supported are Run, Videos, Slides, Papers, and Funding. MTL Demo - Slider section is where the GIF slider files are uploaded. 3.3.5.6.3 Add Course Code Sometimes we present Modeling to Learn in seminars and other academic gatherings. To enable participants to experience MTL, we provide a course code that enables users to access the platform longer than the standard 14 days. From the MTL Demo Admin Webpage, click Add Course Code. An Add Course Code pop-up will appear. Click the Add icon. An Add or Change Course Code pop-up will appear. Enter the course code. Enter the length of access (in days). When finished, click Save. 3.3.6 Forio Epicenter Forio Epicenter is an enterprise platform that hosts our Sim UI and runs the simulations for end users. While the Sim UI can manage most administrator-user needs within the web interface, sometimes it is necessary to go directly into Epicenter to, for example, set up a new end user in Epicenter. Click Epicenter to go to the login. 3.3.6.1 Send Password Reset to End User or Change Password Log into Epicenter. Click on Veterans Affairs. Click the End Users tab in the side navigation bar. Use the search bar to search for a specific user. To the right of the selected user’s name, click the Edit button dropdown. Click Send Reset Password Link if you want to send a password reset. Click Change Password to change the password directly for the user. 3.3.6.2 Setup a New Epicenter End User Navigate to a project. Note: You would navigate to various projects to set up end users in different instances of the Sim UI. For example, you would navigate to VA Sim UI-TEST-Slow to set an end-user up in the TEST environment. Navigate to Group page. In the Epicenter hierarchy, “groups” are the highest entity. Underneath groups are worlds. In TeamPSD, we refer to groups as “teams.” Typically, we auto-setup a _team group and a _ind group to manage team assignments and individual worlds. Select the desired Group. Click the Add End Users button. Add user email to field and click “Add/Invite User” button. We usually put the user’s VA email address in this field. However, it does not have to be an email unless you are sending a notice via email to the user. Sometimes users just need an alias besides their email address. 3.3.6.3 Setup a Team We don’t set up teams directly in Epicenter anymore, because there are multiple dependencies in the Sim UI that would be bypassed. See Manage Teams for more information about setting up a team from the Administrator Dashboard. 3.3.6.4 Setup an Admin To set up an Admin, the end-user id must be assigned to the administrator_login group on the group’s page. "],["404.html", "Page not found", " Page not found The page you requested cannot be found (perhaps it was moved or renamed). You may want to try searching to find the page's new location, or use the table of contents to find the page you are looking for. "]]
+[["index.html", "Modeling to Learn 3.7 Consult Manual Chapter 1 Why use Modeling to Learn 1.1 How might Modeling to Learn help? 1.2 How can a Modeling to Learn consult help? 1.3 What gets in the way of meeting patients’ needs? 1.4 If we keep making the same care decisions, will things get better or worse? 1.5 Why is Modeling to Learn useful when we have critical staffing and hiring needs? 1.6 Why is the MTL Red Data User Interface useful? 1.7 Why is the MTL Blue Simulation User Interface useful? 1.8 Why is applied systems thinking more likely to help us avoid costly mistakes? 1.9 Why is Modeling to Learn able to provide new insights? 1.10 Why does Modeling to Learn emphasize flows through care?", " Modeling to Learn 3.7 Consult Manual Team PSD 2024-11-07 Chapter 1 Why use Modeling to Learn Click on the hyperlinks or video thumbnails to watch the videos. Transcripts of the videos are also provided. 1.1 How might Modeling to Learn help? Watch the video here. You might wonder What is Modeling to Learn, why use it, and how might it help? We often don’t understand what is driving our care problems over time. Modeling to Learn helps to upgrade the decisions you make all day, every day, because the decisions made most frequently will be the most powerful for improving our care. We’re introducing Modeling to Learn, which has been developed and evaluated in the VA for over 9 years. Modeling to Learn is a way to upgrade care decisions by understanding the dynamics of common care problems over time. Are you very, very busy? Modeling to Learn includes step by step guides with gifs showing each click. These guides are available at mtl.how. Are you unsure what data to use, how it’s defined, and are you wishing it was current enough to guide your care decisions? Modeling to Learn includes a transparent, real-time locally defined data user interface available at mtl.how/data, where you specify the clinics that define your data views. Are you unsure what is realistic for improving care locally, given your patients’ needs and your existing staff and resources? Modeling to Learn also includes a simulation modeling interface available at mtl.how/sim, so you can try out new decisions before you implement them in the clinic. We all want to meet Veterans needs for timely, high-quality care. So, what gets in our way? Watch that video to find out. 1.2 How can a Modeling to Learn consult help? Watch the video here. Modeling to Learn improves visibility and provides new insights into how common care problems persist over time. How? Modeling to Learn is based on over 60 years of scholarship known as participatory system dynamics. For this reason, we call ourselves Team PSD for Team Participatory System Dynamics. The Data User Interface and Simulation User Interface comprise two versions of Modeling to Learn. The data-only version is known as Modeling to Learn Red; Modeling to Learn Blue ads participatory learning from simulation. Team PSD supports MTL Red and MTL Blue and is carefully evaluating how each works to support VA in meeting Veterans needs. Learning from simulation can help us to place a better initial bet on what is likely to work locally by evaluating alternative decisions via simulation before we implement them in the real world. MTL Red tells us where we’ve been over the last two years based on the clinic selections made to produce the patient data reports and team trends or visualizations. Many staff report that viewing the Data UI real-time patient data tabs or the team trends is efficient and encouraging. The data tabs help with clinical decision making. The visualizations of team trends provide leading indicators that improvement efforts are paying off, which can be validating. Why is Modeling to Learn useful when we have critical staffing and hiring needs? Watch that video to find out. 1.3 What gets in the way of meeting patients’ needs? Watch the video here. We all want to meet Veterans’ needs for timely, high-quality care. So, what gets in our way? Modeling to Learn is a way to upgrade care decisions by understanding the dynamics of common care problems over time. Demanding clinical days mean we don’t often have bandwidth to gain new insights, particularly in our understanding of interdependent clinic dynamics over time. We’re often flying blind to the impacts that our daily decisions have on the overall community that relies on us. As clinicians, we look at the patient in front of us and decide when we think they should be seen again. But have you ever been told you cannot see a patient as soon as you would like due to the constraints of the clinic? For example, an evidence-based course of care requires starting treatment without delay and keeping the Veteran engaged in a therapeutic dose of care over time to meet their needs. If we only emphasize Veterans starting care, but the time between visits extends way out, this interferes with evidence-based care and Veterans getting better. How can we get more Veterans better without adding hours in our day? Watch that video to find out. 1.4 If we keep making the same care decisions, will things get better or worse? Watch the video here. What if we keep making the same care decisions? Will things get better or worse? This is almost a trick question. If we keep making the same care decisions, more than likely things will stay the same in the clinic. Very rarely do clinical teams not have any local improvements that they would like to make. So, what do we mean by staying the same? We know that things in the clinic are not static but are always dynamic and always changing. We’ll describe how start delays affect treatment decisions to explain what we mean. The Modeling to Learn Simulation User Interface at mtl.how/sim is designed to help a team see what is likely to happen over the next two years if they continue to make the same daily clinical decisions that were made over the past two years. We call this the base case of no new decisions. Any number of feasible alternative decisions a team or site could make can be compared against this base case to find the best options for improving care over time. Because feedback dynamics produce non-linear trends or system behaviors, this means even the base case run can be surprising for some teams. The base case may show even more undesirable likely futures which teams would need to make new decisions to avoid. An example is the way start delays affect treatment decisions through balancing feedback. As the number of patients waiting to start a specific service grows, clinicians must adjust and try to find another way to meet their Veterans’ needs. Over time, as the clinicians learn about a local delay, it starts to affect what they think the most ethical, clinically appropriate treatment for the Veteran will be. An analogy from daily life will help. If there’s a traffic jam outside your VA, you may try to find another route home. As more drivers choose alternative routes, traffic returns to normal. Of course, when drivers don’t know about the jam, it will get worse over time. And whether drivers choose alternative routes or not—with balancing feedback like a traffic jam, where more cars are getting on the highway than are getting off—cars will build up and slow down the rate of traffic for everyone, whether we like it or not. We have these kinds of experiences in our clinics, too. Balancing feedback can be a blessing or curse. A blessing in that the balancing feedback causes the clinical system to return to its status quo. A curse if the status quo is undesirable for quality of patient care or provider quality of work life. When you start Modeling to Learn, it is surprising to see how powerful feedback is in driving care dynamics in any clinical care setting—VA or non-VA, VAMC or CBOC, urban and rural, BHIP, or PCT. General system insights, like the link between balancing feedbacks and nonlinear system behaviors, can be understood quickly from Modeling to Learn. You can understand exactly how many patients will be waiting and when. And then discover options for addressing the number of patients waiting for specific services with your existing staff mix. If you’d like specific recommendations for your local site or team based on these dynamics, request a Modeling to Learn consult or use the guides and videos at mtl.how. Clinical oscillations can really whipsaw clinicians and patients around, which is stressful and can increase potential clinical risks. If the number of patients waiting to start grows for a long time, it’s overwhelming. But it can also be exhausting when things go up and down quickly and clinics feel like they’re chasing their tails trying to find improvements. Especially given these dynamics can vary service by service, even within the same team. If you want to understand this principle, go to mtl.how/sim and check out the Team Care module. Or are you interested in another common balancing care problem, how the appointment backlog extends the weeks between visits? Watch that video to find out. 1.5 Why is Modeling to Learn useful when we have critical staffing and hiring needs? Watch the video here. Leaders and clinicians often ask how can Modeling to Learn be useful when facing critical staffing and hiring needs? With limited staff coverage, the need to hire is priority number one, but even then there isn’t a magic wand that produces staff where they may not exist today. What could help now? From the beginning, Modeling to Learn prioritized evidence-based episodes of care within existing staff time. When Modeling to Learn launched nationally in the VA in March of 2020, it was to empower staff to find the highest yield local improvements without asking staff to do more with less. As a result, when staff coverage may not change quickly, Modeling to Learn will find options for improving quality of care for Veterans and quality of work life for providers. How? What does Debbie mean by high yield local improvements? Because evidence-based behavioral healthcare is delivered over time, there are many possibilities. Small decisions made all day, every day by the clinicians, when compounded over time, can be surprisingly powerful. Think of your savings account or your waistline. In Modeling to Learn, we look for the lightest clinical lift teams and VAs can make that have the biggest payoff for Veterans in terms of timely, high-quality care. We work hard to avoid big difficult changes with limited benefit. When working with the Modeling to Learn-read data user interface, clinical teams are often motivated when they see trends that reflect hard-won efforts to implement high quality episodes of care which may not show up in other data systems for some time. The Modeling to Learn Blue simulation user interface saves staff time because alternatives can quickly be assessed during a modeling consultation. Change is hard, but we no longer have to learn by trial and error, wearing out already burdened staff. Does that sound too good to be true? If so, you may be wondering about examples of Modeling to Learn use cases for the pain points you face in your specific team or program. As an example, how does Modeling to Learn benefit substance use disorder, or SUD programs? Watch that video to find out. 1.6 Why is the MTL Red Data User Interface useful? Watch the video here. Why is Modeling to Learn Red useful and how does the Modeling to Learn Data User Interface provide new insights? The primary value of MTL Red is its power to efficiently query the VA Corporate Data Warehouse directly, and we’ve come a long way over the years. Yes we have. When we first began, we were using an Excel workbook so that frontline teams could carefully review the clinic selections that define their Modeling to Learn team data sets. Fast forward to the present and now teams have real time data available to them from within the VA domain from any computer with PIV badge access. Since the Data UI includes PHI, if you go to mtl.how/data, you will be able to see the same data you have permissions to access in the electronic health record. But given that clinicians, managers, data leads, quality improvement staff, evaluators … well, basically everyone is so busy, no one has time to review another data dashboard unless it offers something of really high value that distinguishes it from other resources. When you request a Modeling to Learn Red consultation, we work with your team or VA to explore care data that can address your locally identified priority. Clinics or scheduling grids are often changing. For that reason, we wanted a user interface where clinic selections can include active or inactive clinics over the last two years. That way, the team can filter the information to find the most appropriate clinics to include in their data set to gain new insights. And in Team Flow, clinic selections can be used to evaluate transitions between an episode of care in one team and the start of another episode of care in a higher or lower intensity care setting. Modeling to Learn Red also enables zooming in to check on the care of an individual patient at the start of the clinical day or during case reviews at a team meeting. But with the MTL Red Data User Interface, you can also zoom out to view teen care trends, bringing patient-level care coordination and trend-level process improvement decisions together. And that’s where things start to get interesting. Based on the clinic selections, the next set of tabs to find local data values for common care problems, including care coordination, psychotherapy, medication management, team care, and team flow. Each tab features simple definitions of how data were estimated for the common care problem. Detailed definitions with technical specifications are also provided to allow valid comparison of these data to other VA dashboards. That said, a focus on data details could be frustrating and add limited value. Modeling to Learn emphasizes understanding system problems in care flow over time. These care flow problems can be defined accurately with just five key time-based variables that drive care quality. How do five key variables drive care quality? Watch that video to find out. 1.7 Why is the MTL Blue Simulation User Interface useful? Watch the video here. Why is Modeling to Learn Blue useful? Well, why is it wisest to focus on the dynamics of care over time? The short answer is that clinical and improvement teams cannot adjust one part of the care equation without everything else changing. In Modeling to Learn Blue, we zoom out to see how care variables are locked in relationship with one another over time. The key variables that define either a poor quality or high quality episode of care must be understood together. Building from MTL Red, you can export your local data set created in the Modeling to Learn Data User Interface at mtl.how/data. Then, if you navigate to mtl.how/sim, you can find the Modeling to Learn Blue Simulation User Interface, which is a dynamic and interactive way to understand why problems with care coordination, medication management, psychotherapy, team care, and team flow persist over time. The Simulation User Interface is a way to see how adjustments in one part of an episode of care explain subsequent impacts in the care system. For any of the common care problems, the simulation saves you time and energy by accounting for the local new patient start rate in patients per week and the local appointment supply and appointments per week. The simulation also keeps track of the local new patient wait time in weeks, time between visits and weeks, and the engagement duration over time, again in weeks. All are calculated for you automatically in the Data User Interface, but their interdependence is accounted for in the Simulation User Interface. The Modeling to Learn Blue Simulation User Interface empowers teams to avoid ineffective strategies because you very quickly learn to develop new insights that would be inefficient, if not impossible, to figure out in your head or by hand. Learning from simulation is designed to help upgrade local decision-making. Teams develop new rules of thumb and insights in which the dependent dynamics among these variables that define care are all taken into account. With a Modeling to Learn consult, we come alongside with partners mid stride in their daily clinical activities who may have limited insight into what is likely to happen over the near future if they keep making the same decisions every day. With Modeling to Learn Blue simulation learning, sites and teams can safely see the impact of new decisions while building new capacities for systems thinking. Why is applied systems thinking more likely to help us avoid costly mistakes? Watch that video to find out. 1.8 Why is applied systems thinking more likely to help us avoid costly mistakes? Watch the video here. Why is applied systems thinking more likely to help us avoid costly mistakes? Unless we understand interdependent system effects over time, we’re very likely to choose ineffective strategies for our clinical work. And when chronic impairment, relapse, suicide, and overdose are the critical concerns for our patients, we need system insights about how to best organize our clinical practice. Applied systems thinking includes understanding complexity, feedback, and system behaviors over time. By complexity, we mean understanding the relationship or the interaction among two or more variables, such as wait times and the improvement rate. By feedback, we mean moving beyond simple cause and effect events to consider how some effects are reinforced and get stronger over time and other effects are reduced over time. To explain feedback using David’s example, it’s not just that if you have long wait times then the patient improvement rate will go down because patients aren’t starting care. It’s also that as the improvement rate goes up, then at a system level wait times will go down as Veterans get their needs met and graduate from care. Our systems are perpetually causing themselves over time through feedback. Causal feedbacks explain trends over time, called system behaviors. Given that feedback can be a reinforcing effect or a balancing effect over time, Modeling to Learn provides better insights about short- and long-term understanding of change over time. This can include decisions that may make things worse before they get better, or may make things better before they get worse. For example, longer wait times for new patients may reduce their improvement rate in the short term, but if this enables higher quality care engagement for them and existing patients over time, then more Veterans can get better and graduate from care and wait times will go down for the entire community of patients. It is through improving our systems thinking that Modeling to Learn really helps us move beyond making costly mistakes. If you’d like to learn more about how Modeling to Learn helps us improve the dynamics driving where Veterans get stuck or drop out of care, watch that video to find out. 1.9 Why is Modeling to Learn able to provide new insights? Watch the video here. Why is Modeling to Learn able to provide new insights about where Veterans get stuck or drop out of care? Because the Modeling to Learn Simulation User Interface depicts the states of care as stocks based on the flows in and out of that state. Take for example, patients waiting to start. We depict patients waiting to start as a stock, a rectangle with an accumulated volume indicator. The number of Veterans waiting to start in a typical week is determined by the difference between the number of Veterans that flow into that state and the number of Veterans who flow out of that state each week. The rate dials depict the flows in and out of stocks, and the primary rates in Modeling to Learn are patients per week, appointments per week, and episodes of care per week. The initial values for any team or site depicted in the Simulation User Interface are based on the last two years of data exported from the Data User Interface. The important idea is that in any dynamic system, things get to be the way they are over time. Things either stay the same or improve decision by decision over time. For example, in a clinic or site, the number of Veterans engaged in a given state of care, such as medication management or psychotherapy, reflects decisions that lead to accumulation in that state over time. The dynamic stock and flow diagram provides a way of understanding where Veterans get stuck. As clinicians know, often there is more than one way out of a state of care. With psychotherapy, you may flow from the first session to the second session, but you may drop out at any point in that flow. Why does Modeling to Learn emphasize flows through care over time? Watch that video to find out. 1.10 Why does Modeling to Learn emphasize flows through care? Watch the video here. Why does Modeling to Learn emphasize flows through care over time? Our goal is to ensure Veterans flow through our addiction and mental health care services to recovery. A basic idea is that care occurs over time. Flows are powerful because they express a rate of change, which means they give us key information about where we will be in the future. Imagine your next trip in the car. We need the miles per hour to know when we will arrive at our destination. But knowing the number of miles to our destination is not enough. We must also know how fast we’re traveling to know when we will get there. In other words, miles per hour. The same is true in our clinical care. It’s not enough to know how many new medication management appointments we have on the books today. We need to know how quickly those appointments were added to the books or the booking rate, in appointments per week, as well as how quickly those appointments are completed or the completing rate, in appointments per week. When the inflow is greater than the outflow, the level of the appointment or patient stock will rise. When the outflow is greater than the inflow, then the level of the appointment or patient stock will fall. Opening the Modeling to Learn Simulation User Interface at mtl.how/sim, you can see where all the Veterans are accumulating in stocks. You can understand the inflows and outflows for each stock and which accumulations are likely to get better or worse over time. Many clinical teams talk about feeling inundated by the addiction and mental health needs of the communities that rely on them. They can feel like they’re drowning, providing the volume of care needed, especially when short staffed. We understand this kind of flooding from the natural world. To stay safe, it’s critical to know how fast the waters are rising and where. We understand this by understanding flows. You may relate to this feeling of being overwhelmed and want to know more about how Modeling to Learn can help. For example, how does Modeling to Learn help improve medication management? Watch that video to find out. "],["2-how-to-use-modeling-to-learn.html", "Chapter 2 How to use Modeling to Learn 2.1 How can we get more patients better without more hours in the day? 2.2 How does Modeling to Learn benefit Substance Use Disorder or SUD programs? 2.3 How does Modeling to Learn benefit PTSD Clinical Teams or PCTs? 2.4 Can Modeling to Learn benefit Behavioral Health Integration Program or BHIP teams? 2.5 How does Modeling to Learn support stepped care? 2.6 How does measurement-based care detect patient improvement or risk? 2.7 How can we improve SAIL with Modeling to Learn? 2.8 How do five key variables drive care quality over time? 2.9 How does Modeling to Learn help improve medication management? 2.10 How does Modeling to Learn help improve psychotherapy? 2.11 How does our appointment backlog extend weeks between visits? 2.12 How can we better balance needs of new and existing patients? 2.13 How can we leverage the feedback, rates, and volume of our local care system? 2.14 What key drivers of higher care quality improve recovery and prevent suicide?", " Chapter 2 How to use Modeling to Learn Click on the hyperlinks or video thumbnails to watch the videos. Transcripts of the videos are also provided. 2.1 How can we get more patients better without more hours in the day? Watch the video here. How can we get more Veterans better without adding hours in our day? To get Veterans better, we have to account for the number of hours available in the day. Yet because we don’t have the right information about how we manage our time, we can pursue unrealistic treatment plans or improvement strategies that violate the laws of physics. Modeling to Learn encourages us to account for clinician time differently, to make our plans feasible locally. Most VA data systems offer snapshots of cross-sectional data, but this kind of information doesn’t help us account for the dynamics of evidence-based care over time. While many VA data systems are available, most display either quarterly lags from a high-level view or an in the weeds patient by patient view. Modeling to Learn helps to bridge these two views. Modeling to Learn introduces trends over time based on the clinic selections carefully reviewed by the staff on site. Trends help us zoom out to understand how we’re doing for most of the Veterans we serve. Modeling to Learn also introduces time-based definitions of care in the units of patients per week, appointments per week, or episodes of care per week. We use a week because over the last several years clinicians told us this is how they think in the clinic. There often is no typical day and a lot has happened by the end of a month. Defining care by week is more beneficial. High quality care in the clinic is always constrained by the available staffing capacity and the hours in the day to see patients, the number of weeks between visits, and the weeks of engagement over time. But do you have time to run these equations in your head or calculate them from the other dashboards you use? As an alternative, Modeling to Learn puts these components of care together to understand their interdependence over time. How can Modeling to Learn consultation help? Watch that video to find out. 2.2 How does Modeling to Learn benefit Substance Use Disorder or SUD programs? Watch the video here. How does Modeling to Learn benefit substance use disorder, or SUD programs? When Modeling to Learn began almost 10 years ago, as a partnership among patients, providers and policy makers across VA, we worked iteratively to define the dynamics of common care problems for the primary reasons Veterans seek addiction and mental health care. We found that if we focused on improving timely, high-quality evidence-based psychotherapy and pharmacotherapy for alcohol use disorder, depression, opioid use disorder and PTSD, we could support VA staff in meeting around 80% of Veterans needs for behavioral healthcare. We work with frontline teams and partners from the VA Academic Detailing Program and the Psychotropic Drug Safety Initiative or PDSI and developed the Medication Management Module of Modeling to Learn. This module helps manage the dynamics of evidence-based pharmacotherapies for alcohol use disorder and opioid use disorder based on their effectiveness in reducing craving, relapse, and overdose. You can find the Medication Management module in Modeling to Learn data mtl.how/data and Simulation User Interface mtl.how/sim. The Modeling to Learn Medication Management module supports improvement in the AlcTop or Alcohol Topiramate indicator from the Mental Health Information System or MIS and the SUD 16 indicator from the Strategic Analytics for Improvement in Learning or SAIL. How does Modeling to Learn help? Prescribers describe the need to better visualize how to locally optimize their medication management appointment supply, including top of license care from staff with and without a Drug Enforcement Agency X-waiver for OUD medication. Modeling to Learn helps a team assess this appointment supply, new patient start rate, no show or missed appointment rate, and the clinically appropriate return-to-clinic visit interval needed for a therapeutic response to alcohol use disorder and opioid use disorder medications. The Medication Management module helps teams and sites find the optimal number of patients who can be engaged in AUD and OUD therapies over time and helps them distinguish the flow of their patients who require antidepressants or other medication needs. Given that community needs and team staffing is dynamic and can change over time, these resources can empower teams to find local improvements that otherwise may be hard to find. Another common module for addressing Veterans SUD needs for group and other therapies is the Team Care module, which helps teams assess and optimize their overall mix of local multidisciplinary services as a function of staffing and patient needs. We are evaluating how Modeling to Learn improves Veterans who start and complete a therapeutic course of medication for AUD and OUD with research funding from the VA and the National Institute on Drug Abuse or NIDA. What about other common SUD comorbidities and presenting concerns? For example, how does Modeling to Learn benefit PTSD, clinical teams or PCTs? Watch that video to find out. 2.3 How does Modeling to Learn benefit PTSD Clinical Teams or PCTs? Watch the video here. How does Modeling to Learn benefit PTSD clinical teams, or PCTs? Since I’m at the National Center for PTSD, this priority is very close to my heart. When Modeling to Learn began almost 10 years ago as a partnership among patients, providers and policy makers across VA, we worked iteratively to define the dynamics of common care problems for the primary reasons Veterans seek addiction and mental health care. We found that if we focused on improving timely, high-quality evidence-based psychotherapy and pharmacotherapy for alcohol use disorder, depression, opioid use disorder and PTSD, we could support VA staff in meeting around 80% of Veterans needs for behavioral healthcare. Modeling to Learn helps teams better evaluate the local dynamics of balancing the needs of new and existing patients for evidence-based psychotherapies, particularly cognitive processing therapy and prolonged exposure. Teams described the challenges of starting new patients who had never had their needs met without compromising care for Veterans already engaged in therapy. Modeling to Learn enables teams to see where their Veterans get stuck, to decide when to graduate patients, or to recognize when the weeks between return to clinic visits start to get too long. Have you been trying to improve PTSD 56 in the Strategic Analytics for Improvement and Learning, or SAIL? We worked with local PTSD clinical teams from the beginning and have supported many more teams over the years who have engaged with the evidence-based psychotherapy programs or the PTSD mentorship program to meet these needs. The Psychotherapy module of the Modeling to Learn data user interface, and simulation user interface helps teams to see how they’re doing for patients who are starting psychotherapy, how many flow through to complete a therapeutic dose, and whether Veterans are getting near weekly therapy consistent with the PTSD treatment evidence base. There’s also the ability to assess emerging approaches such as masked or intensive outpatient psychotherapy and other scenarios that are commonly considered and implemented in PCTs. You may also be wondering about the flow from primary care or from primary care mental health integration, or PCMHI, teams into general mental health and specialty mental health programs like PCTs. Stepped care up and down the mental health continuum of care can also be optimized to local needs using the Modeling to Learn Team Flow module. Speaking of the care continuum, you may be wondering How does Modeling to Learn benefit Behavioral Health Integration program or BHIP teams? Watch that video to find out. 2.4 Can Modeling to Learn benefit Behavioral Health Integration Program or BHIP teams? Watch the video here. How does modeling to learn benefit Behavioral Health Integration Program or BHIP teams? What are the challenges with providing excellent team care? One challenge is the data sources to support team decision making jump from the individual patient and clinician counter all the way to the entire clinic or facility. It can mean that teams are flying blind to how they’re coordinating care for their Veterans. This can be especially hard when teams are short staffed or when staff, patient needs, and local policies continually change. It’s hard to know what improvements you might be able to find in your own team for coordinating evidence-based psychotherapies and pharmacotherapies over time. One way that Modeling to Learn can complement other BHIP resources is through the Data User Interface available at mtl.how/data. The Clinic Selection and Team Flow Selection tabs empower teams to carefully review the clinics that comprise their data views. Data are never all good or all bad. Modeling to Learn emphasizes transparency. Teams need to know what data they’re getting and its strengths and weaknesses for guiding a particular decision. The Clinic Selection tabs update every day to reflect all the Mental Health 500 series Stop Code, Clinic, or Grid updates that may be going on locally. Teams can save bookmarks that reflect their specific clinic selections, for example to include trainees or to select any of the clinics they have used within the last two years. This can be helpful when new BHIP teams are being formed or remapped. The team Patient Data tabs update daily too, so teams can view the patient diagnosis, encounter, health factor, measurement-based care, and high-risk flag information all in one place, and then the team can zoom out to filter and view these data as team trends. Coordinating care within a team is challenging. No single clinician can offer all the services a Veteran may need, so treatment plans must be coordinated in real time, over time, to support Veteran improvement. Clinicians may not have visibility on exactly how appointment supplies divvied up into intake evaluations, individual and group psychotherapy, medication management, or other adjunctive supportive therapies. What is particularly important about the Modeling to Learn simulation user interface at mtl.how/sim is the ability to assess not only the appointment supply, but even more important, the service proportions of patients that flow through to these services after intake or treatment plan review. One of the most common scenarios teams assess is how to respond to patient needs when they may not have the perfect staffing mix. Even when you know patient flow through care is divvied up into a mix of services, the dynamics over time are too much to understand in your head. The Modeling to Learn simulation user interface enables teams to quickly assess the dynamics of available appointment supply, service proportions, and the other two major factors that govern patient improvement, the return-to-clinic visit interval in weeks and the overall duration in care. We’ve often used Modeling to Learn to help teams find a way to ensure evidence-based care through the decisions that they make all day and find improvements in quality of care and quality of work life that they didn’t think was possible due to limited staffing. You might wonder How does Modeling to Learn support stepped care up to BHIP from primary care or primary care mental health integration, or from general mental health up or down into specialty mental health programs such as substance use disorder programs or PTSD clinical teams? Watch that video to find out. 2.5 How does Modeling to Learn support stepped care? Watch the video here. How does Modeling to Learn support stepped care? We developed the measurement-based stepped care module of Modeling to Learn to help teams assess episodes of care when stepping patients up or down across the continuum of care. General mental health or behavioral health integration program, or BHIP teams, and specialty mental health teams such as substance use disorder, or SUD, programs and PTSD clinical teams, or PCTs, use the team flow selection tab in the Modeling to Learn Data User Interface to define flow from their care up or down to another team. Teams make these team flow selections at mtl.how/data. Based on years of testing the Data User Interface, we found that teams are most accurate in defining who they refer Veterans to after completing an episode of care within their team, rather than where they receive referrals from. And the teams also define the gap between visits that they believe is locally most reasonable for defining a new episode of care in each setting. The primary evaluations in the Team Flow module of the Modeling to Learn Simulation User Interface include assessing the flow through care of Veterans with high-risk flags as well as the proportions of high- and low-symptom patients as they flow through care to recovery and step down or discharge. We know that higher care quality improves recovery among Veterans, but how does that vary as a function of the total patients a team is serving? Team Flow can be used to assess the total manageable patients in a team and the impact of the patient load on care quality. Teams can also account for use of community care, which reduces some of the patients actively managed in the team. But as teams know, it does not reduce this care management to zero as Veterans using community care may still trigger an emergency care response or coordination of other services within a team. Perhaps most helpful is the view at a glance of where all Veterans are accumulating, waiting to step up or down across the continuum of care. This can help leadership and teams understand where there may need to be improved referral flows or service agreements between these settings. Teams can evaluate the flows through care when implementing measurement-based care to reduce the time it takes to detect when Veterans may be getting better or worse, which can also help improve agreements or decisions about when transitions across service settings are warranted. Would you like to know more about using measurement-based care to improve the time to detect patient improvement or risk? Watch that video to find out. 2.6 How does measurement-based care detect patient improvement or risk? Watch the video here. Why use measurement-based care to improve the time to detect patient improvement or risk? To improve care quality in the flow of patients through care to recovery, the major value of measurement-based care is ensuring Veterans are getting appropriate care and are responding to the care they receive. Clinicians are likely familiar with the emphasis on measurement-based care, but from the perspective of care flow, what is powerful is reducing the time it takes for a team to detect whether a patient is getting better or worse. In the Modeling to Learn Data User Interface at mtl.how/data, the dataMeas tab provides tables of all the patient measures recorded at the VA Corporate Data Warehouse, whether they were captured via an evidence-based psychotherapy template, mental health assistant, or other approach. These include measures such as the GAD-7 for anxiety, PHQ-2 or PHQ-9 for depression, PCL-5 for PTSD, and the Brief Addiction Monitor or BAM. As a clinician, I like to use the patient-level views to sort scores from high to low to identify patients who may need to step up to a higher level of care or might benefit from referral to a specialty substance use disorder or PTSD service. The lowest score may indicate Veterans who can graduate from outpatient mental health care and step back down to primary care, which is an amazing day for the Veteran, their loved ones, and their treatment team. The vizMeas tab helps teams see trends in these measures to confirm when their efforts to implement measurement-based care are paying off. Teams can toggle to see what proportion of services include measurement-based care by comparing the vizMeas and vizEnc views. The Modeling to Learn Simulation User Interface at mtl.how/sim also empowers teams to assess the impacts of improved implementation of measurement-based care and how it can drive improvements in care quality. Do you want to learn more about the key drivers of higher care quality to improve recovery and prevent suicide? Watch that video to find out. 2.7 How can we improve SAIL with Modeling to Learn? Watch the video here. If you’re watching this video, you want to know about strategic analytics improvement in learning, or SAIL metrics, with Modeling to Learn. Modeling to Learn was developed to complement existing VA priorities. One important benchmark is SAIL. Some of the more frustrating aspects of SAIL are the lag in data to support daily decision-making in the clinic and emphasis on high-level facility views and, of course, how challenging it can be to improve sale metrics over time, especially when you’re short-staffed. In fact, when SAIL metrics change over time, either in a positive or negative direction, it is difficult to know what’s driving the change. Facility leadership and clinical teams may not know how to interpret these changes or what improvement strategy to try next. Modeling to Learn supports two primary dimensions of the SAIL mental health quality domain—care coverage and care continuity. But importantly, Modeling to Learn recognizes that these two dimensions may work against each other in a dynamic tension over time. If you’re expanding population coverage measures, such as PSY 32 for depression, PSY 38, or PTSD 56 for PTSD, or SUD 16 for opioid use disorder, then it may be more challenging to ensure continuity of care measures such as MDD43h, MDD47h, and PSY 33 for depression and PSY 39 for PTSD. Of course, the reverse is also true. If you’re doing great with care continuity on those measures, then it may be challenging to expand population coverage at the same time. I’m fond of calling this the physics of our care quality problems. What I mean is that we cannot change the laws of physics, such as the law of conservation, which states that matter and energy in a physical process cannot be created or destroyed, but can only be transformed. In Modeling to Learn, staff time is neither created nor destroyed. This means that when data is exported from the MTL Data User Interface and uploaded to the MTL Simulation User Interface, the scenarios a team or VA explores to find improvements in care quality will always account for or conserve the total local staff time available. This helps to avoid fixes that fail, such as something that improves population coverage in the short term but reduces care continuity in the long term. Have you ever felt like you were playing Whac-A-Mole® with these metrics, where you improve one indicator only to have a problem later somewhere else? Modeling to Learn can help. Let’s start with why Modeling to Learn Red is useful. And why does the Modeling to Learn Data User Interface provide new insights? Watch that video to find out. 2.8 How do five key variables drive care quality over time? Watch the video here. How do five key variables drive care quality? Modeling to Learn emphasizes the dynamics of care over time, which can be accurately simplified to the key time-based variables that drive care quality. But be careful, the important principle is that these variables operate together over time to define an episode of care. That means care quality cannot be improved without understanding how these variables influence one another. If you want to see what Lindsey’s talking about, navigate to the Modeling to Learn Data User Interface at mtl.how/data and review each care problem—care coordination, medication management, psychotherapy, team care, and team flow. Our partners across VA describe how challenging it is to review data in one information system and then in another and end up unsure how to reconcile them, especially when you think that they indicate a different course of action. Folks are extremely busy and any new data resources must really add value to be worth learning. Let’s see if these Modeling to Learn variables meet the commonsense test of value added. Well, what do you think the five time-based variables are that make up an evidence-based episode of care? All outpatient care is defined by whether you can get an appointment when you need help. We focus on this all the time in VA. But then, and this is critical, you must be able to be seen again to complete a therapeutic course of care adequate to meet your need. Clinicians told us that a week was the way they think clinically. So, in Modeling to Learn, teams make their clinic selections to obtain an estimate of their local new patient start rate in patients per week and their appointment supply in appointments per week. Then we define evidence-based engagement as the new patient wait time in weeks, time between visits and weeks, and the engagement duration over time, again in weeks. This is a simplified definition of an evidence-based episode of care that is accurate for time. Why is it wise to focus on the dynamics of care over time? That’s why Modeling to Learn Blue is useful. So watch that video to find out. 2.9 How does Modeling to Learn help improve medication management? Watch the video here. How does Modeling to Learn improve medication management? Medication management requires identifying a patient need, starting a medication, and evaluating for a therapeutic response over time. Some of the challenges and effective medication management over time include the realities that not all clinicians can prescribe. And of course, it requires follow up care to ensure that a patient is benefiting from their medication. So, when should the patient be seen again? One thing that can be so frustrating to prescribers is that a scheduler tells them when a patient can be seen again, rather than them telling the scheduler when the patient should be seen based on top of licensed care. The frustration is that medication management return-to-clinic visits may not occur at the appropriate interval to balance patient needs and risks. The VA has guidelines for some patient cohorts such as SAIL, MDD43h, and MDD47h, which define high quality continuity of follow up care as a valuation once every 12 weeks for Veterans diagnosed with depression who are using antidepressants. Other needs may reflect a different follow-up interval. For example, many addiction psychiatrists have described the appropriate medication management return-to-clinic visit interval for buprenorphine or methadone for opioid use disorder as every four weeks. These differences in medication needs, and the evidence-based pharmacotherapy standards that go with them, make it hard for prescribers to evaluate at a patient population level in a real time, especially when you’re completing another medication management encounter every 20 minutes. And of course, depending on the community at any given time, the staffing mix may change and the size or volume of patient needs can change too. Some SUD teams may have a high volume of patients receiving evidence-based pharmacotherapy for alcohol use disorder, whereas the modal presenting concern in a BHIP may be more likely to be depression. And the decisions the prescribers are making using current rules of thumb may be boxing them in to a pattern of care that makes it even harder to be responsive to crises or reduce future wait times. The Medication Management module of Modeling to Learn helps a team evaluate the trade-offs among all these factors as compared to the status quo over the last two years in the clinic. Comparing to the base case of no new decisions, the team can evaluate changes in their X-waivered MOUD appointment supply or evaluate and tailor the return-to-clinic visit interval by patient cohort. The goal is to help prescribers see what daily medication management decisions are adding up to over time and find feasible and effective local rules of thumb for clinically appropriate return-to-clinic orders in weeks that reduce wait times and ensure Veterans are getting better. For coordinating medication management in a multidisciplinary team such as a BHIP or SUD program, the Team Care module of Modeling to Learn can also help. If you want to know how Modeling to Learn helps to improve psychotherapy, watch that video to find out. 2.10 How does Modeling to Learn help improve psychotherapy? Watch the video here. How does Modeling to Learn help improve psychotherapy? VA led the nation in dissemination and training for evidence-based psychotherapy. Many clinicians are ready to offer Veterans the highest quality care available. So, what makes it hard to get Veterans through an evidence-based episode of care when they start? What I’ve learned from working with clinical psychologists like Lindsey and other VA therapists from around the country is that the dynamics of psychotherapy in the clinic are challenging for the same reasons most care problems persist over time. Clinical teams typically do not have good visibility of where all their Veterans are in care. You may know that you plan to do CPT session six today with an individual Veteran, but how many Veterans are currently in the middle of therapy now? How many appointments are on the books? How many weeks are there before they return to clinic for follow-up care and how many graduate when they complete a course of an evidence-based psychotherapy? And how many drop out of therapy after one session or early in care but return to the team later? Evidence-based psychotherapy is typically defined by getting near weekly psychotherapy and having clinically meaningful improvement, which usually starts around session 8 to 12. Many VA data systems provide useful counts and proportions of evidence-based psychotherapy templates and the SAIL continuity of care metrics to find quality based on the number of sessions Veterans get within a specific time period. But the SAIL metric itself is a quarterly snapshot, and we know Veterans don’t all start therapy at the beginning of a new quarter. In the Modeling to Learn Data User Interface, a one-year patient cohort is identified that includes all Veterans seen from 18 months to six months ago based on the clinic selections made with the Data User Interface at mtl.how/data. Then, for the Veterans seen within those 12 months, we look back to see when those individual Veterans began therapy and how they were engaged over time. Once this psychotherapy cohort is defined and uploaded to the Simulation User Interface at mtl.how/sim, we can look at the stocks and see where all the Veterans have accumulated, where they are in care in a typical week, and how many patients per week are progressing through key therapy milestones. Finally, at a local team level, psychotherapists can see how they are doing for most of the Veterans they serve. And even better, they can run experiments to see what the impact will be of making new decisions over time. You might wonder What if we keep making the same care decisions? Will things get better or worse? Watch that video to find out. 2.11 How does our appointment backlog extend weeks between visits? Watch the video here. How does an appointment backlog extend the weeks between visits? At first, this sounds like a simple system story that many clinicians describe as we work together to co-develop Modeling to Learn. But they also told us when an appointment backlog extends the week between visits, a long interval between visits effects care too. That’s right. As the number of appointments for a specific service grows on the books, then the clinic has no choice but to increase the number of weeks or push out the time between visits to get everyone seen. We call this the return-to-clinic visit interval. This quantifies the return-to-clinic order in a unit of time—one week—which is critical for finding realistic local improvements in our partnerships with sites. Okay, so the balancing feedback principle operating here is that if you have a clinic calendar filled up with appointments, it’s harder for new patients to start care and it becomes harder for all existing patients to be seen with appropriate therapeutic follow-up standards for psychotherapy and pharmacotherapy. Lindsey, you and Debbie also talked about balancing feedback in the video, What if we keep making the same care decisions, will things get better or worse? For an explanation of how start delays affect treatment decisions over time, watch that video. As I said at the beginning of this video, it’s not just that a growing appointment backlog forces teams and clinics to extend the number of weeks between visits to get Veterans seen. It’s that long return-to-clinic visit intervals impact treatment decisions and care quality. None of these effects occur in isolation. All these components of care interact over time. It’s why system scientists will often point out there are no side effects and systems, there are only effects. Some we want and some we don’t. As clinicians identify Veterans who might benefit from an evidence-based course of psychotherapy for depression or PTSD, they may find that many patients are waiting to start, or they know that those 90833 or 90834 psychotherapy encounters are occurring 12 weeks apart—or both. Clinicians recognize that they cannot see patients in a way that is consistent with an evidence-based episode of care and will have to start finding alternatives. Modeling to Learn can help. What a relief that hundreds of clinicians, managers, data experts, quality improvement leads, and patients have already worked with us over the last decade to develop scalable Modeling to Learn resources for clinical and improvement teams to use today. The Modeling to Learn Simulation User Interface accounts for all these feedback effects so the team can upload their data, review what they see, and get immediate feedback about what the most feasible alternatives are to ensure quality of care for patients and quality of work life for providers and local staff. Based on this discussion, you may wonder How can we better balance the needs of new and existing patients? Should we prioritize new patient start rate or weeks between visits for quality? Watch that video to find out. 2.12 How can we better balance needs of new and existing patients? Watch the video here. How can we better balance the needs of new and existing patients? Should we prioritize new patient start rate or weeks between visits? These trade-offs are challenging for clinicians when you know there is a whole community of patients who need help. When teams are struggling with the limits of their available time in the day to see patients, it can feel like a clinical, ethical, and even moral quandary about how to best balance patients’ needs for services with the staff resources available. Of course, staff resources are always changing. We know many teams that are excelling in providing the highest quality addiction and mental health care available. But the behavioral health workforce shortage in the US is much bigger than VA. And VAs and teams need the right tools to make sure Veterans get the care they need for recovery. Challenges balancing the new patient start rate and the weeks between visits for existing patients apply systems thinking insights that we’ve talked about in other MTL videos, such as the physics of conserving staff time in order to ensure a clinically beneficial, realistic approach to care decisions and quality improvement that meets VA quality standards and meets Veterans needs for evidence-based episodes of care. The Modeling to Learn Blue Simulation User Interface available at mtl.how/sim enables a site or team to evaluate these two balancing system stories as a function of their data for the last two years exported from Modeling to Learn Red Data User Interface at mtl.how/data. As we talked about in the How does an appointment backlog extend the weeks between visits? video and What if we keep making the same care decisions, will things get better or worse? video, balancing feedbacks occur in any system that has a goal, including our healthcare and clinical systems. Balancing feedbacks occur in systems that have constraints of resources, staff, and time. And as a result, the trends that occur over time tend to reset to a status quo or oscillate around the status quo. A brief Modeling to Learn consult uses the site or team reviewed local data plus simulation to efficiently find local improvements that account for all these balancing trade-offs. We aim to help clinical and improvement teams find a couple empowering clinical heuristics, or rules of thumb, that are more effective for ensuring evidence-based care to get more Veterans better. When we partner with a VA or a team, we aim to find the lightest lift we can, like increase groups by 10% or adjust the return-to-clinic order for three weeks for these presenting concerns. And we often find something small that has a big payoff for Veterans. Do you want to be empowered to leverage the feedback, flow, and volume of your local care system? Watch that video to find out. 2.13 How can we leverage the feedback, rates, and volume of our local care system? Watch the video here. Do you want to be empowered to leverage the feedback, flow, and volume of your local care system? In any system, leverage describes the places where a small shift in one thing can produce big changes in everything. For a big benefit for Veterans in their care teams, the primary places to focus are where the greatest volume is, the fastest flower rates are, and of course to account for feedback dynamics over time. When we partner with the VA or team, we aim to find the lightest lift we can, like finding for one BHIP that increasing group psychotherapy by 10% eliminates wait times and gets five times the number of Veterans near weekly therapy. Or, adjusting the return-to-clinic order by three weeks for some presenting concerns can provide the same medication management supply as hiring another prescriber. We can often find small care decisions that when implemented frequently over time, have a big payoff for Veterans because they add to more than the sum of the parts in the interdependent system over time. The last thing we want to do is make big changes that are difficult for Veterans and clinical teams that produce limited benefit over time. In my experience over many years now, finding something clinically powerful that leverages the dynamics of the system can be a huge boost to morale, productivity, and staff empowerment. You might wonder How can you find powerful insights that you are sure will work locally? We do this by reviewing the Modeling to Learn Simulation User Interface stocks to see where all the Veterans are accumulating the most. We review the rates to see where the floodwaters are rising fastest and require action. Or where things are trending in the right direction and our attention could be better focused elsewhere. Modeling to Learn consultation offers clinical and improvement teams real-time efficient support. We can do this quickly because we’ve partnered with patients, providers, policy makers, data experts, evaluators, and quality improvement consultants from the frontline to VA Central Office for nearly 10 years now to define the primary drivers of care problems and care quality using system dynamics. Modeling to Learn consultation involves empowering clinicians, managers, improvers, data leads, and all of VA to leverage the system for getting more Veterans better. How can Modeling to Learn consultation help? Watch that video to find out. 2.14 What key drivers of higher care quality improve recovery and prevent suicide? Watch the video here. What are key drivers of higher care quality to improve recovery and prevent suicide? A central idea of Modeling to Learn resources is that delivering consistent, timely, high-quality care is challenging for teams because we don’t accurately account for the primary dynamics driving care quality over time. You might be thinking Care dynamics, what does that mean? We literally mean that due to time pressures and cognitive limits, we don’t effectively understand relationships among key care decisions that we make every day over time, especially decisions made in the aggregate across a clinical team. For example, we may try to break down problems by looking at referrals or wait times, but we don’t look at how clinical teams’ knowledge of wait times influence referrals to a specific service or team. In the Modeling to Learn Data User Interface, we define care dynamics in terms of rates over time, like new patients per week, so that we can understand the flow of Veterans through care. As another example in terms of care quality, recovery and suicide prevention, the Modeling to Learn Team Flow module looks not only at the number of patients served by a team or site, or the number of patients with a high-risk flag for suicide, but also the time it takes to unflag high-risk patients in a team and the typical time to improve within the team over the last two years. The Modeling to Learn Simulation User Interface uses an interactive structure fed by local team data from the Data User Interface. The Modeling to Learn Simulation User Interface visually depicts accumulation of patients in desirable or undesirable states of care, like waiting to start a new service. These states are defined by their inflows and outflows. A lot of clinical teams and mental health leadership wonder whether they can get any new insights when they are already doing everything that they can to support the Veterans in their care. Would you like to know how to improve SAIL with Modeling to Learn? Watch that video to find out. "],["3-technical-documentation.html", "Chapter 3 Technical Documentation 3.1 Cheatsheets 3.2 Field User General Manual 3.3 Administrator Technical Manual", " Chapter 3 Technical Documentation 3.1 Cheatsheets 3.1.1 CPT Code Cheatsheet CPT Code Cheatsheet 3.1.2 MTL 3.7 Data Cheatsheet Data Cheatsheet 3.1.3 MTL 3.7 Sim UI Cheatsheet Sim UI Cheatsheet 3.2 Field User General Manual 3.2.1 Create an Account or Team Follow the instructions below to create an account or team in the Sim UI. Once the registration form is completed, a notice will be sent to MTL for approval. Once approved, the user will be notified by email they now have access to the dashboard. Tip: Save and refer to this one-page Sim UI Cheatsheet when creating an account or team! There are two ways to navigate to the registration form: Click on Create Account or Team in the Sim login window located at mtl.how/sim: Navigate to mtl.how/register in your Edge, Firefox, or Chrome browser. Complete the form as shown below: 1 - VISN: Select your VISN from the dropdown menu. 2 - Facility: Select your facility from the dropdown menu. 3 - Clinic: Select your clinic from the dropdown menu. This field is not required. 4 - Select Facility or Create New Team: If you are a team member: Select your existing team from the dropdown menu. If you are a team lead: Select Create New Team. If you are creating a team or the first person to register from your Facility Team, the following pop-up window will appear (with your VISN, Facility, and Clinic pre-populated). Enter your Facility Team name in the pop-up window (immediately after the pre-populated information). Notes: You must only use lowercase letters. You must use the underscore character (_) instead of spaces. Click the Submit button. Even though the dropdown field remains unchanged on the form, the submitted team name is saved, as confirmed by the green text as shown in this example: 5 - Select VHA Primary Role: Select your primary VHA role from the dropdown menu. Choices are: Clinician, Data Subject Matter Expert, Leader/Manager, Quality Improvement, Researcher/Evaluator, Systems Modeler, and Other. 6 - First Name: Enter your first name. 7 - Last Name: Enter your last name. 8 - VA Email: Enter your VA email. 9 - New Password: Create a password and enter it here. Tip: Write it down so you don’t forget! Note: Your password must contain at least 1 number. 10 - Repeat Password: Re-enter the password you just created. Tip: To check if both passwords match, click the eye with a slash icon in each field to reveal what you typed. Click the green Submit button. The Registration Detail pop-up will appear. Ensure the registration details displayed on the pop-up are accurate. If the information is not correct, close the pop-up window and make adjustements to the registration form where needed. If the information is correct, click the green Submit button. A success page will appear: After your registration request is reviewed and approved, an approval email will be sent to your VA address that contains login information for accessing the MTL Sim UI. 3.2.2 Log in to the Sim UI You must already have an account set up to log into the Sim UI. If you do not have an account yet, follow the Create and Account or Team instructions and wait to receive an approval email. Navigate to mtl.how/sim, enter your credentials, and click the Login button. A Please select a World pop-up will appear. The Sim will load teams and, once you select a team, you will be directed to that team’s Home page. Notes: The login dropdown only shows worlds in which you are a team lead or participant (not an Administrator). If you have the Administrator role in any team and have been added into “administrator_login” in Epicenter, you will be allowed to login using the “administrator_login” choice. Upon logging in, Administrators are directed to the Administrator Dashboard. The screenshots below display what you will see upon logging in, depending on your role. Note: To familiarize yourself with a team’s Home page, refer to Orient to a Team’s Home Page Team Member login = team’s Home page Administrator login = Admin Dashboard 3.2.3 Make, View, and Export Team Clinic Selections for Import to Sim UI Tip: Refer to (or save) the one-page Data Cheatsheet when using the Data UI and making clinic and flow selections. The general workflow for the Data UI is that teams: Select their clinics and team flow (if applicable). Review and look at the data and viz tabs to observe trends in the clinics. Generate the Data UI file from Power BI to export into the Sim UI. Notes: All users must follow certain naming conventions, which are explained below, when downloading Team Data files. When a Team Data file is downloaded from the Data UI, it is in a specific format that the Sim UI reformats during upload. This means that if a Team Data file is downloaded from the Sim UI and then re-uploaded to the Sim UI it will not work! Clinic selections and data tabs refresh every night. Viz tabs update monthly at the beginning of the month. Open Chrome, Edge, Safari, or Firefox browser and go to mtl.how/data. Using the left-hand navigation pane, navigate to the desired Region and VISN. Click on the Clinic Selection tab and select and view your local team data. Filter by Division Name, Physical Location, PrimaryStopCode, SecondaryStopCode, and/or Is an Active Clinic? (de-activated clinics denoted by ZZ) in the left and middle columns. Search and filter for your team’s clinics by clinic name or by provider in the Select Clinics to Generate Report Data column after making selections in the left and middle columns. Click the green Apply button to pull Patient Level Data from all clinics selected in the last two years. Select clinics that your team refers to in the Team Flow Selection tab to produce data tracking patients stepped up/down for the Team Flow module. For example, if you are a GMH team, you would select SMH in the dropdown for “The clinics selected below that my team refers to are” and add the SMH clinics your team refers to in the Green column, second row. In the bottom right-hand columns for the three possible care settings (GMH, SMH, PC/PCMHI) indicate the threshold for minimum gap in patient care required for subsequent visit to be considered a new care episode. In general, for the (GMH or SMH or PC/PCMHI) setting, how long of a gap in patient care is required for the subsequent visit to be considered a new care episode? Set the threshold below in months. The default threshold is 9 months (39 weeks). Click Apply. Notes: Are the clinics and data not populating? Request and verify your local server views by following steps outlined in Request and Verify Local Server View Permissions. If you wish to have quick access to these Clinic Selections in the future, you can make a personal bookmark to return to it. Go to the Bookmarks drop-down menu and click on Add a personal bookmark. In the bookmark name field, use Team PSD convention for the name: Sta3n_Division Name_Date (date in yyyy_mm_dd format), then select Save. Scroll until you see the View Clinic Selections tab on the left-hand side. Hover over the Clinic Name > Providers heading and click on the ellipsis that appears. Click Export data > Data with Current Layout > Export. The Team Data file is exported to your browser’s download location. Save using our Team naming convention “Sta3n_geo-location_clinic_year_month_day” and post the file on the shared Team channel for Team members to view and confirm selections. Repeat steps 2-6 with Team Flow Selections. Tips & Tricks: Link to CPT Code Cheatsheet. Use the most recent clinic selection table to find the full Sta6a. The folder for the site list xlsb file is located in the TeamPSD shared drive > Grants > 0_final_post_covid19_site_list. Note: This site list is for SMEs and the internal team; it is not applicable for field users. To clear filters/columns, click on the Eraser icon in the upper right of the filters/columns. To reset ALL selections across ALL tabs, click the Reset button in the upper right hand of the bar. To zoom into the screen, hover in the upper right-hand corner of the screen until a set of icons appear. Click on the square Focus Mode icon. Click Back to Report to return to the previous view. Note: Focus Mode is found in the data and viz tabs only. Click to view the vizDiag tab which shows team trends. Viz tabs are your reference modes - trends over time. Zoom out and see team trends for diagnoses (Diag), encounters (Enc), health factor data (HF; e.g., suicide plans, evidence-based practice templates), measures (Meas), and behavioral and suicide risks (Flag) from Mental Health Assistant. Click on the respective tabs (Diag, Enc, etc.) to view these trends. Another way to see encounters or services decreasing or increasing is by viewing the number of visits by right clicking on any trend line and click Show As Table. Filter trends to zoom into a specific team trend by navigating to the right side of the screen and click on the Funnel icon to reveal the filter panel. vizDiag in the Data UI shows a trend lines by diagnoses or combination of diagnoses. For example, the PTSD line is for patients with only PTSD. The PTSD & AUD line is for patients with both these diagnoses but does not discriminate between which is the primary, secondary, etc. diagnoses. Tips & Tricks: Right click on any trend line and click Show As Table to see encounters or services decrease or increase by viewing the number of visits. Filter trends to zoom into a specific team trend by navigating to the right side of the screen and click on the “Funnel” icon to reveal the filter panel. If you do not see something that you would expect or previously saw, check to see if a filter is applied. Click on the funnel icon in the top upper right. Filters are active if it says “Filters and slicers affecting this visual.” Click to view the “data” tabs, which show your team’s individual patient information. In the data tabs, providers can zoom in on specific veterans. Patient information corresponds to the same categories as the team trends: diagnoses (Diag), encounters (Enc), health factor data (HF; e.g., suicide plans, evidence-based practice templates), measures (Meas), and behavioral and suicide risks (Flag) from Mental Health Assistant. Providers can filter to find specific patients or produce reports by navigating to the right side of the screen and clicking on the “Funnel” icon to reveal the filter panel. Tips & Tricks: dataDiag in the Data UI has an additional column after Diagnoses of Interest, called Primary Diagnoses, specifying which diagnosis is primary to help providers better understand the characteristics of the patient population. Patients who have requested restricted access to their information have asterisks (******) in the Patient Name column. If you are a VA provider, you can still identify patients from the Patient SSN column, or Patient Name column. If you do not see something that you would expect or previously saw, check to see if a filter is applied. Click on the funnel icon in the top upper right. Filters are active if it says “Filters and slicers affecting this visual.” 3.2.4 Request and Verify Local Server View Permissions To pull patient and clinic level data, you need LSV permissions. Check which facilities you have LSV access for here through VA intranet. Clinical Resource Hub Providers are granted to multiple local server access. If you do not have access to facilities that you believe you should, please verify: CPRS access/permissions at your station (i.e., COR tab access for CPRS). Your network username recorded in VistA. To have your network username recording in VistA, please follow instructions here through VA intranet. If you are still unable to pull data or access the Data UI, please email us at MTL.Help@va.gov Note: If you need access to all sites (e.g., national), click here and follow the steps in Request Access to Data Tools section. 3.2.5 Request Access to Data Tools Click here to access the OI&T Electronic Permission Access System. Verify VHA NDS Access Form for Health Operations is selected in the Select Document dropdown, then click Create Document. Fill in the form as follows: Note: the following fields will be pre-populated: Full Legal Name, Job Title, Email Address, Active Directory Name with Domain, and Work Telephone Number. 1 - Employee Agency: Veterans Health Administration 2 - Employee Type: VA Employee (yes, for all of you, none of you are “contractors”). New fields will appear. 3 - Lookup Supervisor: Search for “zimmerman, l”. 4 - Purpose of Use: Healthcare Operations 5 - Data Sources: Select Corporate Data Warehouse (CDW) 6 - Duration of Access: Select “I will need access during the duration of my position.” from the dropdown. 7 - Affirmation: Check the box. Scroll towards the top of the screen and select the Corporate Data Warehouse (CDW) tab. Make the following selections after selecting the CDW SQL Datasets checkbox: Basic Read Access (CDW_Full) Privileged Read patient Access (CDW_SPatient) Staff Real SSN Access (CDW_SStaff) MCA (DSS) Access (CDW_DSS) Outpatient SQL Files (CDW_SE_IE_SF) Now select the Comments tab. Copy/paste in the following rationale provided to you: I support the Modeling to Learn national operational quality improvement project led by the VA Office of Mental Health and Suicide Prevention (OMHSP). This initiative includes a national Power BI Report in which frontline staff access local operations data as part of their quality improvement activities. Click Save and Submit. 3.2.6 Navigate to the Home Page of Any Team Tip: Refer to (or save) the one-page Sim UI Cheatsheet when familiarizing yourself with the Home Page and Dynamic Decision Dashboard. Upon logging in to the Sim UI, field users will be automatically directed to the selected team’s Home page. Notes: Team PSD members will have uploaded the same Team Data file into the _team world for everyone at that consult location. Learners can see the same accumulations and learn where to prioritize based on the location of the greatest accumuation to leverage for change. Although learners see the same information, it is still your own unique session (behind the scenes). There are ways to play in the same world and learn together within each module: Care Coordination, Team Care, Medication Management, Psychotherapy, and Team Flow. Administrators can access a team’s Home page via the Administrator Dashboard. Select a team from the Select Team dropdown, then click Home (see screenshot below). The Home page of the selected team will appear. 3.2.6.1 Orient to a Team’s Home Page Below is a basic orientation to the Home page of any team: MTL logo: Click this logo to return to the Dynamic Decision Dashboard. Team Name: The selected team name appears here with the world extension added at the end of its name. If you are logged in as an Administrator, you will see “Administrator” here. Home: Click to return to the Dynamic Decision Dashboard. Play: Click to return to the Play page. More Info: Click for more information on how to use the Sim UI. News: Click for the latest news about Modeling to Learn. An alert will appear if you have unread news items. Report Bug: Click to report a bug. Tutor: Click any Tutor icon for an “over the shoulder” overview. Logoff: Click to log off of the Sim UI. Avatar: Click to upload your profile picture. Session Start window: You can either join the simulation session in progress or start a new one. Select the team data file you wish to use from the dropdown menu, then press GO! Experiment window: This section allows you to delete, rename, export, and sort your team’s simulation experiment results. The experiments listing shows your current team first. Notes: Administrators can see all experiment data across all worlds, organized by associated groups (team + world), so they are able to associate experiments with their respective groups. New Administrators can access all teams, data, and experiments—without being manually added to any other teams. Team Data window: This section allows you to delete, upload, and sort team data files (which will then show in the Start a Session window). Legacy team data files are located under the MTL 3.0 gray bar. Dynamic Decision Dashboard: Use this section to navigate between the 5 modules and their associated variables. Community: Click to give feedback to system administrators and participate in our MTL Community of Practice. 3.2.7 Begin a Session in the Dynamic Decision Dashboard Tip: Refer to this one-page Sim UI Cheatsheet when familiarizing yourself with the Dynamic Decision Dashboard. Beginning a session in the Dynamic Decision Dashboard (DDD) allows users to look at the stocks and flows across all five modules. It also helps determine which module(s) to prioritize, based on the location of the greatest accumulation to leverage for change. Learners can see the same accumulations and learn where to prioritize based on the location of the greatest accumuation to leverage for change. Navigate to the selected team’s Home page. If you aren’t sure how, see Navigate to the Home Page of Any Team. Select the desired team data file from the Please Select Team Data dropdown and click GO! A Learning Mode pop-up will appear. Tip: If multiple members of a team want to review the DDD at the same time, it is highly recommended to take turns initializing (pressing Go!). Otherwise, some users may not be able to successfully initialize the DDD modules. Notes: Team PSD members will have uploaded the same Team Data file into the _team world for everyone at that consult location. Learners can see the same accumulations and learn where to prioritize based on the location of the greatest accumuation to leverage for change. Although learners see the same information, it is still your own unique session (behind the scenes). There are ways to play in the same world and learn together within each module: Care Coordination, Team Care, Medication Management, Psychotherapy, and Team Flow. If there is not a team data file in the dropdown, follow these steps. Make CC and MM Learning Mode selections using the radio buttons, then click Continue to initialize all modules within the Dynamic Decision Dashboard. Notes: Existing Patients Return-to-Clinic Visit Interval (EPRVI) = learning mode “0”. New Patient Start Rate (NPSR) = learning mode “1”. Refer to the Care Coordination (CCParams) or Medication Management (MMParams) tab of the Team Data Table at mtl.how/data, for data regarding the team’s New Patient Start Rates and Return-to-Clinic Visit Intervals. 3.2.7.1 Upload Team Data Files from the Home Page Expand the Team Data blue bar and select Upload: A pop-up window will appear asking if the data file is in Power BI format. Ensure it is and click Yes to proceed. The File Manager will appear. Navigate to the location of the downloaded team data file and click on the file name. Click Open to upload the file. Be patient; it may take a while for the file to upload. Notes: Be patient; it may take a while for the file to upload. As the data file is uploaded, you will see “Data is uploading…” and three flashing loading bars in the Session Start and Team Data sections. These messages clear once the file is loaded into the Sim. The team data file name should now appear in the Team Data list and in the Please Select Team Data dropdown. Return to Begin a Session in the Dynamic Decision Dashboard, Step 2. 3.2.8 Dynamic Decision Dashboard The Dynamic Decision Dashboard (DDD) is a convenient visual way to help answer the question Where is the volume (of Veterans) across all modules? The DDD is for everyone (clinicians, consultants, nurses, etc.) to test their assumptions and hypotheses. The primary purpose of these visuals is to help understand the dynamic system that is Veteran care. The dashboard shows a visual representation of where Veterans accumulate (creating lags in the system), where Veterans are waiting to start, and the services in which these behaviors are observed. This aids in understanding how different quantities change and interact over time and identifying key variables that contribute to the accumulation or depletion of stocks. The DDD allows clinicians to make new clinical decisions that are among their most common decisions, resulting in high-leverage change(s) that are the most influential decisions that impact the behaviors of the Veteran care system, and empowers teams to understand how to zoom in to get a particular Veteran the right treatment at the right time or zoom out to understand how all the individual decisions aggregate over time to determine the dynamics of the system. 3.2.8.1 Understand the Five Modules There are five modules within the Dynamic Decision Dashboard, each of which is described below: Care Coordination: shows how a variety of team decisions interact to impact multiple outcomes at the same time. You can explore trade-offs between decisions such as desired new patient wait times, current patient return visit intervals, and overbooking, and how they affect team outcomes like total missed appointments, number of patients in care, and number of patients completing care. Team Care: gives insights about RVI and service proportions, and shows how a variety of team decisions interact to affect your overall mix of services. You can explore trade-offs between decisions such as desired new patient wait times, current patient return visit intervals, and overbooking, and how they affect team outcomes like total missed appointments, number of patients in care, and number of patients completing care. You can compare the effects of different decisions on outcomes in one service, or you can compare the effects of one decision on different services. Medication Management: enables you to experiment with how team decisions about referrals, time allocation, and visit frequency impact the quality of medication management for patients. Your team can explore scenarios to balance these trade-offs and better meet the specific needs of patients with depression, alcohol use disorder, and opioid use disorder. Insights you gain can help you increase local reach of evidence-based pharmacotherapies. Psychotherapy: shows how team psychotherapy decisions interact and depicts patterns of psychotherapy engagement in your team. You can explore solutions to the challenges of getting patients to complete a full course of therapy, while also achieving goals to start enough new patients in psychotherapy. Insights about these dynamics can be used to increase the local reach of evidence-based psychotherapies. Team Flow: shows the effects of measurement-based stepped care on patients’ symptoms and risk. It allows you to explore the impacts of implementing measurement-based care to reduce delays in detecting patients at high risk for suicide, and to improve the quality of care by making better team decisions about when to step patients up to a higher level of care, or step them down to a lower level of care. It is also possible to experiment with team decisions related to new patient wait times and access, the use of community care, and the impacts of provider overwork and burnout on the quality of care. 3.2.8.2 Orient to the Dynamic Decision Dashboard Below is an orientation to the Dynamic Decision Dashboard. Tutor: Click for a guided tour of page features. Caret: Click carets next to each module to contract or expand that module from view. Directives: Click to show a Directive 1160.01 to MTL crosswalk table that identifies specific references within Directive 1160.01 that requires actions by facilities that are supported by the Data UI and the Sim UI. These actions are linked to MTL reports and MTL use cases that reinforce how the directives are supported in application by other facilities. More: Click to reveal other key views of the Care Coordination module. All modules have this feature. Depending on the module selected, the More menu may include: 1 - Feedback Loops: Interactive diagrams that aid in visualizing how different variables in a system are causally interrelated. Clicking here will open a pop-up window that shows an associated loop diagram along with Causal Loop Dynamics selections. 2 - Flow: Visualized in Sankey form, these diagrams help visualize the rate of flow into and out of stocks. A stock is a state of care in which patients can accumulate over time. Sankey is a type of flow diagram where the width of the arrows is proportional to the flow quantity. 3 - Decision Meters: A quick method to visualize the relevance or significance of different aspects within a group. 4 - Episodes of Care: Graphically shows evidence-based episode of care intervals to get a bird’s eye view of how long the length of engagement will be. 5 - Play: Enter the main Sim UI page to see the system story and run experiments. You can learn together if you click Play and join an ongoing session in one of the modules. 6 - Team Data: A quick link to the Modeling to Learn Data User Interface that will open in a new browser tab. 7 - SAIL Data: A quick link to Strategic Analytics for Improvement and Learning Value Model (SAIL), a system for summarizing hospital system performance within Veterans Health Administration (VHA). This will open in a new browser tab. Learning Mode Labels: Indicates which Learning Mode the data prioritizes. Click these buttons to change the learning modes directly from the DDD; the system will re-initialize based on the new learning mode selected. Existing Patients Return-to-Clinic Visit Interval (EPRVI) = learning mode “0” and New Patient Start Rate (NPSR) = learning mode “1”. Team Care Services Radio Buttons: Select desired service radio button to see stock values for specific services in Team Care (Intake, Care Coordination, Medication Management, Psychotherapy, Evidence-based Psychotherapy, Adjunctive Services, Group Therapy). The default display is Intake evaluations. Medication Management Services Radio Buttons: Select desired service radio button to see stock values for medication-specific services in Medication Management (Alcohol Use Disorder, Depression, Opioid Use Disorder, and Other). The defaut display is Alcolol Use Disorder. “i” icon: Click “i” icons to read the definition of each stock label. Team Flow Services Radio Buttons: Select desired service radio button to see stock values for setting-specific services in Team Flow (General Mental Health, Specialty Mental Health, Primary Care/Primary Care Mental Health Integration). The default display is General Mental Health. Community: Click here to participate by sharing an idea, letting the team know what you like, asking a question, or joining our community. 3.2.8.2.1 Feedback Loops A systems view and understanding is reinforced through the use of funcational causal loop diagrams accessible directly from the DDD. These diagrams are interactive and help visualize how different variables within the system of veteran care are causally interrelated. Feedback loops involve the output of a system being fed back into the system as input. They are essential for understanding and analyzing the behavior of dynamic systems. Each story has its own unique feedback loop that shows named balancing and reinforcing feedback loops and the whole system story with polarities. For illustrative purposes, the Team Flow Story is used as an example below (with all causal loop dynamics shown) to orient you to general feedback loop window features. Note: Click and hold the blue bar at the top to move the pop-up to a desired position on the screen. Tutor: The Tutor button will introduce you to the features of this page. Key: This displays the key for all feedback loops. Positive Causal Arrow: Values at the arrow’s tail (cause) and point (effect) change in the same direction; both increase or both decrease. The plus sign means it is a direct relationship. Negative Causal Arrow: Change in the value at the arrow’s tail (cause) make the value at the point (effect) change in the opposite direction. The minus sign means it is an inverse relationship; when one increases, the other decreases. Widget: Use the checkboxes to show or hide different loop dynamics. Link polarities Checkbox: Select and deselect to show or hide link polarities; the diagram will build as defined. Loop polarities Checkbox: Select and deselect to show or hide loop polarities; the diagram will build as defined. Increase or decrease Checkbox: Select and deselect to show or hide increase and decrease polarity arrows; the diagram will build as defined. Widget Dropdown: Click here to show or hide the widget. Polarity Arrow: When these arrows are displayed, users can select any of the arrows by clicking and flip the increase or decrease icons. All the icons in the loop will then flip according to their relationship. R Reinforcing Loop: This is a reinforcing loop; it reinforces runaway increase or decrease - virtuous or vicious behavior. B Balancing Loop: This is a balancing loop; it keeps things in balance - stable or stubborn behavior. Window Size: Click and drag to resize this window. Exit: Click to close this window. 3.2.8.2.2 Decision Meters Decision Meters are a quick infographic-based way to identify the highest leverage points and are highly supportive of clinical decision-making; they are for “decider” users who need systems thinking insights. The meters provide a visual interface that provides real-time feedback on ranges in our primary MTL units of patients/week and appointments/week. The traffic light reflects the current state results and recommendation of future state results. Through dialogue with decision-makers, consultants and field users manually input values into this graphic generator so the decision meters reflect the recommendations based on the alternative scenarios/experiments run by the learner. Decision meter numbers should be reflected in the “D” box of the Q/H/F/D outputs text field within the Sim UI, usually as ranges of numbers. Care Decision Meters are for MTL Blue teams only. Click within each decision meter to edit. Remember to click Save if you want the values to remain! For illustrative purposes, the Medication Management Decisions window is used as an example below to orient you to general Decisions pop-up window features. Tutor: The Tutor button will introduce you to the features of this page. Service Options: Click a radio button to see stock values for specific services. Hovering over the abbreviations will display the entire service name. Patient and Appointment Flow: Suggested Patient and Appointment flow. Episodes of Care: Suggest treatment length and frequency. Red: Click in the shaded text field of any red circle (greater net inflow to outflow) to input a value (up to 4 digits). Yellow: Click in the shaded text field of any yellow circle (equal inflow to outflow) to input a value (up to 4 digits). Green: Click in the shaded text field of any green circle (lesser net inflow to outflow) to input a value (up to 4 digits). Save: Click to save the entered values. Exit: Click to close the pop-up window. 3.2.8.2.3 Episodes of Care Timely access to high-quality care includes improved understanding of dynamic tradeoffs in the Return-to-Clinic Visit Interval (RVI) and engagement durations. The Episodes of Care pop-up is used to graphically show evidence-based episode of care intervals to get a bird’s eye view of how long the length of engagement will be. Available within the Medication Management, Psychotherapy, and Team Care modules, this pop-up integrates engagement duration and RVI for both Medication Management and Psychotherapy. The following screenshot will orient you to general features of this pop-up. Tutor: The Tutosr button will introduce you to the features of this page. Medication Management example: Medication Management example shown side-by-side with Psychotherapy to illusttrate a tradeoff in service proportions being used. Psychotherapy example: Psychotherapy example shown side-by-side with Medication Management to illustrate a tradeoff in service proportions being used. “i” icon: Click to view the CPT Code Cheatsheet in a new browser tab. Time Box: Each rectangle represents one month. Appointment: “X” represents a Return-to-Clinic appointment. Exit: Click to close the pop-up window. 3.2.9 Enter and Save an Experiment This section explains how to enter and save an experiment in the Sim UI. Tip: Save and refer to this one-page Sim UI Cheatsheet when running an experiment! 3.2.9.1 Start a Play Session in the Sim UI From the team Home page, use the Please Select Team Data dropdown to select your team data file, then click GO! A Learning Mode pop-up will appear. Make the appropriate selections, then click Continue. Note: For CC and MM, select your Learning Mode before hitting Play. Refer to the Care Coordination (CCParams) or Medication Management (MMParams) tab of the Team Data Table at mtl.how/data, for data regarding the team’s New Patient Start Rates and Return-to-Clinic Visit Intervals. To run a simulation for any module within the dashboard, select Play found within each module’s More menu. Tip: You can learn together if you click Play and join an ongoing session in one of the modules. In the Text pop-up window, populate the Our Question, Our Hypothesis, Our Findings, and Our Decisions text fields. Click Save. A Save Experiment pop-up will appear. Click the Save icon on this pop-up. A Scenario saved successfully message will appear. 3.2.9.2 Orient to the Sim UI Tip: Refer to the Sim UI cheatsheet to use when running an experiment. Below is an orientation to the Play Page of the Sim UI. 1: Module name. 2: Team Vision. 3: Reveal Dynamics over Time: Reveal more dynamic relationships by clicking the checkboxes. Click Run to simulate outcomes. 4: Outputs & Text: View 6 output charts. Click on expand “T” to open Text (pin #9). 5: Experiments Section. 6: Select previous runs to set experimental sliders and Questions, Hypothesis, Findings, and Decisions (Q/H/F/D) text to that run. 7: Team Data table shows starting values for variables from your data. 8: Move Sliders from initial values and run. 9: Text: Enter Q/H/F/Ds. Click on expand icon in blue bar for full functions. 3.2.10 Resize a Window for Side-by-Side Viewing An administrator may need to show two items on the screen side-by-side for comparison. Field users may also want to view two items on the screen for comparison or reference. The Sim UI has a responsive design that should accommodate many resolutions and browser window sizes. Follow the steps below to get the best view. Click the “window” icon in the upper right of the browser window. Drag the browser window to the desired size. Notice how the Sim UI design responds by stacking top navigation menu icons and other page contents. Sometimes items on the screen are cutoff or not formatted correctly. Click the refresh button so the browser can detect the new window size and adjust the format. If the format is still not optimum, decrease the zoom below 100% to reduce the size of the items on the screen, or drag the screen slightly larger. 3.2.11 Report a Bug There are two ways to report a bug; through the Sim UI and directly through GitHub. One must have access to either the Sim UI or GitHub in order to report a bug. 3.2.11.1 Report a Bug through the Sim UI From the Home page or from the Play page, click on the Report Bug icon. Describe the issue in the field provided (please be detailed as possible). Do not include PII or other Site identifiable information. Instead, use generic language. If you have a screenshot you want to share, click the checkbox. Click Submit when finished. If you clicked the screenshot checkbox, an additional dialogue will appear to help you upload your picture. Be sure there are no PII in the screenshot! Click Submit again when finished. The Sim UI will report the bug automatically to GitHub. 3.2.11.2 Report a Bug Directly in GitHub To report a bug directly in GitHub, navigate to mtl.how/teampsd to get started. The page below should appear. If not signed in, sign in. Then click the Issues tab. Click “New Issue” button. Click “Get Started” in the “Issue” swim lane. Complete the Issue Form. Do not include PII or other Site identifiable information. Instead, use generic language. Route the form to the “bug_tracker” project for action by Team PSD. Be sure to check the “bug_tracker” is listed under “Projects” (see the second screenshot below). Click the “Submit new issue” button. 3.3 Administrator Technical Manual As a MTL Administrator, the following chapters will help answer any technical questions you may have during prep and live team meetings. 3.3.1 Login as Administrator Navigate to mtl.how/sim. Log in using your credentials and select the administrator_login world. The Sim UI Administrator Dashboard will appear. Note: For Administrators, the login dropdown only shows worlds in which you’re a Team Lead or Member (not Administrator). 3.3.2 Sim UI Administrator Dashboard This section provides familiarization with the Administrator Dashboard and explains how to use this dashboard to set up and manage teams. If you have the Administrator role in any team and have been added into the “administrator_login” in Epicenter, you will be allowed to login using the “administrator_login” dropdown choice. Tutor - Click for a guided tour of page features. Settings - Click to set up a team. It provides team setup input screens and a means to load Team Data. See the Manage Teams section for more information. Team Statistics - Lists experiments, when they started and ended, total time-on-task, team size, attendance percentage, and completion percentage. Team Roster - Lists team members, their clinical role, and team role. MTL Manual - Click to open a comprehensive program and technical guide about Modeling to Learn. Home - Click to go to the selected team’s Home page, which also contains the Dynamic Decision Dashboard. Select Team - Use this dropdown to select your desired team. It enables you to monitor their experiments and navigate to the team’s Home page. Note: This dropdown only shows groups to which you belong (not ones in which you have the Administrator role). Gauges - This indicates the team’s progress with a selected experiment. Teams receive 20% completion for a minimum 50 characters input for each of the Question, Hypothesis, Findings, Decisions fields and another 20% for running the experiment. Select Experiment - Use this dropdown to select the desired experiment to monitor. An experiment will not show in this dropdown until it has been named and saved. Expand / Collapse Dropdown Triangle - Manage and monitor multiple teams by selecting this dropdown. CLOSE - Click to de-select a team. ADD TEAM - Click to add an unlimited number of teams to the dashboard. 3.3.3 Manage Teams This section describes how to set up a team from the Administrator Dashboard. Once team setup is complete, users can access the team’s Home page which contains the Dynamic Decision Dashboard. Click the Settings icon to begin setting up a team. Within the Facility Team Setup area of this pop-up, several team setup options are available and are described below: Add - Click to add a team. Edit - Select a team first, then click here to edit that team. Delete - Select a team first, then click here to delete that team. Note: Teams shown under Facility Team Setup are ones you have joined and in which you have the ‘Administrator’ role. Teams in which you have ‘Lead’ and ‘Member’ permissions will not be visible. 3.3.3.1 Add a Team Click the Add icon. A Setup pop-up will appear. Add the team name using the convention “visn_facility_clinic_facility team name”. Just add the team’s root name. The setup function will automatically add “_team”, “_ind”, and “_shared” extensions in Epicenter. Use only lowercase letters. Use the underscore (_) character instead of spaces. Add the team’s consult vision. This field is required. Click the Add icon to add a team member. A Setup - Add Members pop-up will appear. Enter First Name, Last Name, and Email. Use the Select Permissions dropdown to assign the team member’s permission level on the team. Use the Select VHA Primary Role dropdown to assign the team member’s role on the team. Click the Add icon. Note: Click the checkbox next to a name and then select Edit or Delete to update information or delete a team member. Click the Done button when finished. Notes: If your newly created team (and its associated worlds) do not show in the Administrator Dashboard’s Select Team dropdown, reload the page. The Sim auto-adds members of our internal team as administrators to newly created teams. This is in addition to the default team lead. Keep the auto-added individuals shown below as-is. 3.3.3.2 Edit a Team Click the checkbox next to the team you want to edit, then click the Edit icon. An Edit Setup screen will appear. Within the Edit Setup screen, you may add, edit, or delete team members from your selected team. 2a. To add a new team member, click the Add icon. A Setup - Add Members pop-up will appear. Enter First Name, Last Name, and Email. Use the Select Permissions dropdown to assign the team member’s permission level on the team. Use the Select VHA Primary Role dropdown to assign the team member’s role on the team, then click the Add icon. Notes: Adding a member to _team will automatically add them to _shared and _ind. Adding a member to _shared will not impact _ind or _team. Adding a member to _ind will not impact _team or _shared. 2b. To edit a team member’s information, select the checkbox next to the team member’s name, then click Edit. A Setup - Edit Members screen will appear. Make changes as needed to the First Name, Last Name, Permissions, and/or VHA Primary Role fields. 2c. To delete team members, select the checkbox(es) next to the team member(s) you want to remove from the team, then click Delete. Note: More than one team member can be deleted at a time. A confirmation message will appear; select No if you wish to make further changes, select Yes to confirm deletion of selected team member(s). Click Done when finished editing your team(s). 3.3.3.3 Delete a Team Click the checkbox(es) next to the team(s) you want to delete, then click the Delete icon. A confirmation message will appear; select No if you wish to make further changes, select Yes to confirm deletion of selected team(s). 3.3.4 Manage Team Data Note: Only administrators and the internal team have access to these functions. Field users can upload their data file(s) from their team’s Home page (as well as administrators and the internal team). Field users should refer to the Dynamic Decision Dashboard section. Administrators can upload team data files and assign that file(s) to team(s) of their choice. Please note these specifics about the behavior of team data files: When a team data file is uploaded into a _shared or _team world, it will be available in all worlds (_ind, _team, and _shared) for all users of those worlds. When a team data file is uploaded to an _ind world, it is accessible exclusively to the _ind world only. When a team data file is deleted in a _shared or _team world, they are deleting it from _shared and _team worlds for all users of those worlds. This will not impact _ind worlds. When a team data file is deleted in an _ind world, it will be removed from the _ind world exclusively. 3.3.4.1 Upload a New Team Data File to a Team From the Administrator Dashboard, click Settings. Within the Data side of this pop-up, click Add. A Please select a world pop-up will appear. Select the world(s) in which you want the team data to appear and click Upload. A pop-up window will appear asking if the data file is in Power BI format. Ensure it is and click Yes to proceed. The File Manager will appear. Navigate to the location of the downloaded team data file and click on the file name. Click Open to upload the file. Be patient; it may take a while for the file to upload. Notes: The upload routine must update multiple GitHub repositories and Epicenter, which may take a few minutes. If the file does not appear in the Data side of the Settings pop-up, try clicking the Refresh button. Do not attempt to upload team data file directly to Epicenter, since the GitHub repository will overwrite the files on the next update and they will not work when you try to access them from the Sim UI. Excel (.xlsx) data files exported from the MTL Data UI (Power BI), or uploaded from your MTL MS Teams, are in the correct format. Downloading and re-uploading data files from the MTL Simulation UI will not work. 3.3.4.2 Delete a Team Data File From the Administrator Dashboard, click Settings. Within the Data side of this pop-up, select the team data file you want to remove, then click the Delete icon. Notes: As a reminder, When a team data file is deleted in a _shared or _team world, they are deleting it from _shared and _team worlds for all users of those worlds. This will not impact _ind worlds. When a team data file is deleted in an _ind world, it will be removed from the _ind world exclusively. A message will appear, asking you to confirm deletion of the selected file. Select No if you want to cancel this action. Upon successful deletion, a green message will display in the upper right-hand corner of the browser window. 3.3.5 Modeling to Learn Maintenance Dashboards The MTL Maintenance Dashboards provide customizable information and messaging capabilities for webpages in the Sim UI. Upon logging into the Sim UI as an Administrator, the MTL Maintenance Dashboards icons are viewable along the header bar of the Sim UI. 3.3.5.1 Orient to the Maintenance Dashboards The following screenshot will orient you to general features of the Maintenance Dashboards, located in the top navigation bar of the Administrator Dashboard. Community Function Administration: View lists of issues reported by users from the Report Bug and Community Button, and issue outgoing email messages and targeted ANDON Alerts. More Information Page Administration: Manage images and reference links in the Information Page. Add and edit Module Section information. Add and edit Dynamic Decision Dashboard Directives. Tutor Administrative Dashboard: Manage text descriptions and web links for tutor prompts for the Home Page, each module, the Dynamic Decision Dashboard, and each module’s story. MTL Quick Tip Loading Screen Administration: Add icons and descriptive text for tips that appear when MTL screens are loading. MTL Demo Webpage: Manage the MTL Demo website, such as adding course codes, managing Demo user lists, and managing page content such video and reference links, text descriptions, and others. Team PSD Webpage: Manage the Team PSD website, like adding award winners, adding new team member biographies and photos, or changing page content. 3.3.5.2 Community Function Administration Name of user submitting the issue. Email of user submitting the issue. GitHub issue number assigned by GitHub API. Description of the issue. Click the “Export” button to download an spreadsheet file of the list. 3.3.5.2.1 User Issue List Management Use this section to review issue information. The listing only contains a text description of the issue. Sometimes users submit files or screenshots. To view these details: Click the checkbox next to the issue. Click “See Issue” button. Review information in the pop-up. 3.3.5.2.2 Outgoing Message Management The Sim UI sends out email and other messages to mass communicate important information. There are 5 types of messages. I have an idea: This is an outgoing email that informs the user that we have received their idea. I like something: This is an outgoing email that informs the user that we have received and appreciate their message. Report Bug: This is an outgoing email that informs the user that we have received their bug report and will act on it. It also returns an issue number for their reference. News: Every user has a News button on their Home webpage. When clicked, it will display news items on their webpage that were set up with this function. Alert: Also known as an ANDON alert, this is an outgoing message that displays on the Home webpage when the user logs in. It can target administrators only, users only, or both. ANDONS are normally reserved for problems where the user should stop using the Sim UI until the issue is resolved. 3.3.5.2.2.1 Target ANDON Alert Users As explained in 5. above, you can target specific users with an ANDON Alert. When Alert is selected from the Message dropdown, an additional Show Message to Whom will appear. Select the target audience and click Add Message. 3.3.5.3 More Information Page Administration When a user clicks the More Info icon in the their header bar within the Sim UI, they navigate to a page that contains information about the Sim UI. There are multiple sections: MTL Guides, MTL Links, MTL Navigation, MTL Modules, MTL Sections, and MTL References. While the sections of this page are permanent, you can change elements inside the sections using the Administration page, accessed by clicking More Info from the Administrator Dashboard. 3.3.5.3.1 MTL Guides and Navigation Pages You cannot change the MTL Guides or Navigation pages. 3.3.5.3.2 MTL Links and References Pages Below is the MTL Links page. When the user clicks on this graphic, they navigate to the mtl.how website. This is the text instruction. Below is the MTL References page, a bibliography that supports the research being conducted by MTL. 3.3.5.3.2.1 Update Links and References Below is the Links and References maintenance saection of the More Info Admin page. This is the title bar. This is the sequence that the link will appear on the page. These can be modified by changing the number. Note: All numbers will need to be updated to reflect the new sequence. This identifies the section you’re working in: Links. This is the file name of the graphic file. Upload the graphic file here. A thumbnail will show when it is uploaded. Note: Please use a .png format graphic. The explanatory text for the graphic is shown here. Slightly offscreen is where the url for the graphic goes. This identifies the section you’re working in: References. This is the explanatory text for the bibliography entry. Note that it has several characteristics. You can boldface or italicize using markdown language. The text can have hyperlinks embedded using the ##Link1## code. The number immediately after “Link” indicates the position of the title of the hyperlink from the list. The hyperlink title then calls the url listed in order in this column slightly offscreen. Each entry can have up to six hyperlinks. To add a hyperlink, click on the “Empty” and a dialogue box will appear. Don’t forget to save! 3.3.5.3.3 MTL Modules Page The screenshot below shows a table with a listing of the modules offered in MTL. In case more modules are addedt, it is possible to expand this list. 3.3.5.3.3.1 Update or Add Module Information Click this button to add a section. It will appear as the last row on the list. This is the sequence that the link will appear on the page. These can be modified by changing the number. Note: All numbers will need to be updated to reflect the new sequence. This is the name of the Module which appears as a title over the descriptive text in the next column. This is descriptive text. Click to modify. Use markdown to emphasize boldface, italics and add bullets. These describe the layers or complexity reveals in the module. Click to modify. Use markdown to emphasize boldface, italics and add bullets. These are the experimental controls available in the module. Click to modify. Use markdown to emphasize boldface, italics and add bullets. 3.3.5.4 Tutor Function The Tutor is a function in the Sim UI that explains functions on the webpage to the user. Each section on the Home, Care Coordination, Medication Management, Psychotherapy, Team Care, and Team Flow pages have a Tutor icon. 3.3.5.4.1 Tutor Administrative Dashboard The Tutor Administrative Dashboard enables you to adjust descriptive text and links associated with each Tutor callout box. However, the user cannot add or remove Tutor callouts. These have to be added using computer code. See below for a description of these features. Each page in the Sim UI has Tutor callouts. This points to the call outs on the Home page. Below are Tutor callout listings for Care Coordination, Medication Management, Psychotherapy, Team Care, and Team Flow modules; the Dynamic Decision Dashboard; and Care Coordination, Medication Management, Psychotherapy, Team Care, and Team Flow stories. This is a unique row that enables you to adjust the description below Tutor Administrative Dashboard title. Otherwise, it is the description of the Tutor callout. This is the section on the webpage where the Tutor callouts apply. This is the function the Tutor is describing. This text will show up in the Tutor callout box. To edit, simply click on the text. Don’t forget to save your changes! When the user clicks the link button on this Tutor callout box, the computer will navigate the user to this url. To edit, simply click on the link. Don’t forget to save your changes! 3.3.5.5 MTL Quick Tip Loading Screen Administration When a screen loads in the Sim UI, a blank gray screen is presented. We use this opportunity to offer suggestions or “Tips” to explain functions in the Sim UI. We have an admin webpage to create new Tips, accessible by logging in as an Administrator, then selecting Tips on the Administrator Dashboard top navigation bar: Tips usually have graphics, which are white line drawings on transparent background saved in the .svg format. When paired with text, these present on the loading screen (see example below). A repository of quick tips icons are available in lzim/mtl/images/quick_tips_files. Please note that MTL Demo has its own tips management administration page. To access this, you must: Navigate to the MTL Demo admin page. Click the “Tips” icon.** Note: The steps to add, update, or delete quick tips are the same for the Sim UI and MTL Demo. Verify you are on the right page by looking at the URL. It will say “va-psd-sim/qtipadmin.html” if you are in PROD Tips and “va-psd-demo/qtipadmin.html” if you are in the DEMO Tips. Demo Tips Page Sim UI Tips Page 3.3.5.5.1 Add a Quick Tip From the Administrator Dashboard, click on the “Tips” icon. The MTL Quick Tip Loading Screen Administration page will open. Click the “Add Tip” button. The page will scroll to the bottom of the list and present a new row to input information. Click on the “upload” icon. An upload file pop up will appear. Navigate to your file and click “Save.” The filename should appear immediately to the left of the upload icon and a thumbnail should appear to the right. Click on the “Empty” text. A text field will present. Type in the description and click save (the blue checkmark). You are finished! Be sure to upload the .svg graphic you used to the lzim/mtl/images/quick_tips_files folder. 3.3.5.5.2 Update a Quick Tip Updating a Quick Tip is similar to adding one. See the screenshot below for more information. Remember to click the blue checkmark button (save) when finished! 3.3.5.5.3 Delete a Quick Tip To delete a quick tip, select the tip(s) to delete, then click the delete button. 3.3.5.6 Manage MTL Demo Webpage The MTL Demo webpage provides interested users access to a demonstration simulation. The webpage also offers information about the research project, associated training and video materials, and highlights partners that had a hand in its development. The admin website enables the admin to adjust webpage content and demo users. 3.3.5.6.1 MTL Demo Webpage Navigate to MTL Demo to view example below. To modify this webpage, click on the MTL Demo icon from the Administrator Dashboard. 3.3.5.6.2 Orient to the MTL Demo Admin Webpage Export User Data section lists users that have registered for access to the MTL Demo simulation. It lists their names and contact information. The typical registration is for 14 days. However, with a Course Code, the length of the registration can be longer. A user’s access time can be improved, but it must be done before the original date expires. Resource Management section manages the information contained on the MTL website. The sections supported are Run, Videos, Slides, Papers, and Funding. MTL Demo - Slider section is where the GIF slider files are uploaded. 3.3.5.6.3 Add Course Code Sometimes we present Modeling to Learn in seminars and other academic gatherings. To enable participants to experience MTL, we provide a course code that enables users to access the platform longer than the standard 14 days. From the MTL Demo Admin Webpage, click Add Course Code. An Add Course Code pop-up will appear. Click the Add icon. An Add or Change Course Code pop-up will appear. Enter the course code. Enter the length of access (in days). When finished, click Save. 3.3.6 Forio Epicenter Forio Epicenter is an enterprise platform that hosts our Sim UI and runs the simulations for end users. While the Sim UI can manage most administrator-user needs within the web interface, sometimes it is necessary to go directly into Epicenter to, for example, set up a new end user in Epicenter. Click Epicenter to go to the login. 3.3.6.1 Send Password Reset to End User or Change Password Log into Epicenter. Click on Veterans Affairs. Click the End Users tab in the side navigation bar. Use the search bar to search for a specific user. To the right of the selected user’s name, click the Edit button dropdown. Click Send Reset Password Link if you want to send a password reset. Click Change Password to change the password directly for the user. 3.3.6.2 Setup a New Epicenter End User Navigate to a project. Note: You would navigate to various projects to set up end users in different instances of the Sim UI. For example, you would navigate to VA Sim UI-TEST-Slow to set an end-user up in the TEST environment. Navigate to Group page. In the Epicenter hierarchy, “groups” are the highest entity. Underneath groups are worlds. In TeamPSD, we refer to groups as “teams.” Typically, we auto-setup a _team group and a _ind group to manage team assignments and individual worlds. Select the desired Group. Click the Add End Users button. Add user email to field and click “Add/Invite User” button. We usually put the user’s VA email address in this field. However, it does not have to be an email unless you are sending a notice via email to the user. Sometimes users just need an alias besides their email address. 3.3.6.3 Setup a Team We don’t set up teams directly in Epicenter anymore, because there are multiple dependencies in the Sim UI that would be bypassed. See Manage Teams for more information about setting up a team from the Administrator Dashboard. 3.3.6.4 Setup an Admin To set up an Admin, the end-user id must be assigned to the administrator_login group on the group’s page. "],["404.html", "Page not found", " Page not found The page you requested cannot be found (perhaps it was moved or renamed). You may want to try searching to find the page's new location, or use the table of contents to find the page you are looking for. "]]