Conducting Rural Health Research, Needs Assessment, and Program Evaluation
Rural communities and healthcare facilities have limited resources to address many health-related needs. Research and needs assessments can help determine where and how resources may best be targeted, and program evaluations can indicate whether a particular intervention or approach works well in a rural context.
Rural stakeholders who understand the purposes of conducting research, needs assessments, and program evaluations, and who have the tools to undertake such activities, will be better positioned to focus their efforts where they will have the best result.
Likewise, policymakers and funders who understand how to help rural communities by supporting rural health research, assessments, and evaluations can help build our nation's understanding of rural health needs and effective interventions to address those needs.
- Identifies the similarities and differences among rural health research, assessment, and evaluation
- Discusses common methods, such as surveys and focus groups
- Provides contacts within the field of rural health research
- Addresses the importance of community-based participatory research to rural communities
- Looks at the community health needs assessment (CHNA) requirements for non-profit hospitals and public health
- Examines the importance of building the evidence-base so interventions conducted in rural areas have the maximum possible impact
Frequently Asked Questions
- What are the purposes and roles filled by research, needs assessment, and program evaluation in the rural health arena?
- What are common data collection methods used in research, assessment, and evaluation?
- When should we conduct a needs assessment and plan the program evaluation for our rural community health project?
- What kinds of questions does rural health research seek to address?
- Who does rural health research?
- What funding is available to support rural health research?
- How do you select an appropriate rural definition for a research study?
- Are there concerns unique to rural areas researchers should keep in mind?
- What is community-based participatory research (CBPR) and how can it help rural and tribal communities and researchers work effectively together?
- What is comparative effectiveness research (CER) and how can it help us understand how well specific healthcare interventions work for rural residents?
- What is the role of practice-based research networks (PBRNs) and what are some examples of rural PBRNs?
- What are some different types of assessments relevant to rural health?
- What are the main steps in planning for and conducting an assessment?
- What are the Affordable Care Act requirements for hospitals to conduct Community Health Needs Assessments (CHNAs)?
- What are the community needs assessment requirements for public health agencies related to the Public Health Accreditation Board (PHAB) accreditation standards?
- How can rural hospitals and public health agencies work together to conduct community assessments?
- What CHNA tools and resources are available for rural facilities?
- What are the different purposes evaluation can help address?
- Why is it important to evaluate rural health programs?
- How can rural programs plan for and conduct efficient and practical program evaluations?
- What tools are available to help rural grantees learn about program evaluation?
- How can funders ensure their grantees' experiences help build the collective understanding of what is effective in addressing rural health issues?
What are the purposes and roles filled by research, needs assessment, and program evaluation in the rural health arena?
Rural health research, assessment, and evaluation are all endeavors that seek to improve the health of rural populations through better understanding. These three activities often overlap, share similar methods, and may be defined differently depending on who you ask.
For the purpose of this guide, we will discuss them as follows:
Research seeks to discover new knowledge through systematic investigation. Researchers use the scientific method, forming a hypothesis about how some aspect of the world works, and then testing the hypothesis through observation or experiments to determine if it is accurate.
Needs assessments seek to measure, or assess, the circumstances present in a specific community, organization, or program. Assessment can be used to identify needs or gaps in available rural health services, as well as assets and strengths.
Program evaluation focuses on determining how well a program or service is doing in terms of meeting specific goals and objectives. Ideally, the goals and objectives were identified prior to implementation of the program, or at minimum, prior to knowing the program results. The focus of program evaluation is to provide information to help administrators and staff make decisions about how to improve the delivery and impact of a program. What is being evaluated is driven by the community stakeholders' values.
All three activities can be conducted by a community, healthcare facility, government agency, or academic institution, assuming the investigators have the appropriate skills or a mentor to help develop those skills.
Before you begin an inquiry, determine whether you intend to do research, assessment, or evaluation. Clarifying your purpose will help you select the appropriate methods for documenting and sharing what you learn.
A comparison of research, needs assessment, and program evaluation
|Characteristic||Research||Needs Assessments||Program Evaluation|
|Intended to answer a question or set of questions||Yes, a hypothesis||Yes, a desire to understand the state of affairs||Yes, is the implementation going according to plan and is the program having its intended effect?|
|Involves an intervention||May or may not||No||Yes|
|Has measures defined in advance||Yes||Possibly||Yes|
|Includes baseline data||Yes||Can serve as baseline data||May or may not be necessary, depending on how the goals and objectives are defined.|
|May be funded by a grant||Yes||Yes||Yes|
|Findings may be published in a journal article||Yes||Yes||Yes|
|Helps build the knowledge base of rural health||Yes||Yes||Yes|
What are common data collection methods used in research, assessment, and evaluation?
Many methods are available to gather evidence. Selecting the appropriate methods will depend on the questions you have in mind, the resources and expertise available, and time and geographic constraints.
Surveys use set of questions, which can be asked in-person, by phone, on a paper form, or online. These questions can be targeted to an individual, such as a rural resident or healthcare professional, or to an institution, such as a rural hospital. A survey, or questionnaire, might be used by itself for point-in-time information, or might be targeted to before and after an intervention, to see if there is a change in circumstance.
Interviews are discussions between an interviewee and another person, who may represent themselves individually or represent an institution. Interviews are less structured than surveys. A key informant interview, for example, is used to find out in-depth information about a community or organization by asking a stakeholder for their insights.
Focus groups are like group interviews. A facilitator leads a gathering of participants, all of whom share some common characteristic, such as living in the same rural community or belonging to a similar stakeholder group. Like the interview, a focus group is less structured than a survey. Because of interactions among the participants, it may offer rich insights that would not be obtained individually.
Observation involves the investigator watching participants do something that is of interest. For example, a study might observe how healthcare workers interact with a telemedicine program in providing care to patients.
Abstraction is the process of taking data from existing records. For example, a study could involve pulling specific information from a set of medical records to determine if best practices were followed.
Secondary data can be taken from existing data sources. For example, a needs assessment might draw on data collected by various state and federal sources. For more information on data sources, see Finding Statistics and Data Related to Rural Health.
Using indicators, which are signs that show the condition or existence of something. Indicators are especially useful in evaluation because they are often inexpensive to collect and trusted by administrators or program staff to make decisions. For example, many people in rural areas will seek health care in an emergency department after clinic hours or during the weekend. These costly visits are an indicator of another problem: a lack of access to primary care during the evening and on weekends.
When should we conduct a needs assessment and plan the program evaluation for our rural community health project?
Needs assessments and program evaluation planning should be included at the start of a rural project:
- Conduct a needs assessment to identify the issues to address and available resources
- Select and adapt a program with goals aligned to meet the identified need and feasible with the available resources
- Plan how you will evaluate the program
- Begin implementation
It can be tempting when you hear about a successful project in another rural community, or a set of activities that sound useful, to decide this is the intervention to use. However, there is no guarantee the circumstances in the other community are similar enough to your own for the same intervention to work.
A needs assessment conducted prior to the beginning of program planning and implementation will be most effective. Understanding the current circumstances, both needs and strengths, can help program planners select and adapt an intervention that is a good match for their community. Learn more about rural health needs assessments.
Rural communities, in particular, face challenges if they invest resources in a poorly targeted intervention, such as:
- Not addressing the intended goals of the program
- Not being able to afford an alternative intervention, if the first is not successful
- Losing the confidence of community members and stakeholders, who are likely to be needed for future projects
- Losing the confidence of the project's funder, in an area where there are likely limited sources of funding
A program evaluation should be designed in the planning stage of a project. This will clarify the rationale for the program and how the proposed activities target the needs the program intends to address. Planning the evaluation before you begin will ensure you are measuring things within the direct and immediate control of the program. Learn more about rural program evaluation.
Rural Health Research
What kinds of questions does rural health research seek to address?
Rural health research can focus on a wide-range of topics important to the delivery of rural medical services, the viability of rural healthcare facilities, and the health of rural people. It is part of the field of health services research that looks more broadly at healthcare access, quality, and cost.
Rural health research can also examine outcomes for rural populations. See What is comparative effectiveness research (CER) and how can it help us understand how well specific healthcare interventions work for rural residents?
Some examples of questions rural health researchers study include:
- How do health workforce supply and demand compare in rural versus non-rural locations?
- What is the availability of a specific health service, such as obstetrics, in a rural area?
- What is the impact of a rural Medicare reimbursement change (proposed or implemented) on the financial stability of rural hospitals?
- How does the health status of rural residents compare to non-rural residents?
- What are the health behaviors of rural residents, such as tobacco use and physical exercise?
Rural health research or studies may be driven by many things, such as:
- Requests for proposals from federal and state government, foundations, payors, or others with an interest in the efficiency and quality of the healthcare system
- Practical problems a healthcare provider is trying to solve, such as how to best implement a service
- The existing literature, which may identify gaps in knowledge or inspire additional questions
- A research agenda for a specific topic developed by a group with an interest in the field
- The researcher's professional interests
For more information on health services research:
- Introduction to Health Services Research: A Self-Study Course, from the National Information Center on Health Services Research and Health Care Technology (NICHSR)
- What is HSR, AcademyHealth
For more on how research questions are developed:
- Problem Formulation in William Trochim's The Research Methods Knowledge Base
Who does rural health research?
The Federal Office of Rural Health Policy (FORHP) funds rural health research centers to study issues of current, national concern related to rural health. The Rural Health Research Gateway lists current FORHP-funded centers, as well as the research projects and findings underway and completed. It also lists a number of previously funded rural health research centers and their work.
Health services researchers at other organizations also study rural health issues. Some of these researchers work at academic centers focused specifically on rural health. Others may be part of an academic department, research organization, or other group that has another primary focus, such as nutrition or healthcare workforce, and includes "rural" as an aspect of their research because it relates to their primary field.
For a selected list of centers that conduct or otherwise support rural health research, please see the Organizations section of this guide.
You can also find information on health services researchers via HSRProj (Health Services Research Projects in Progress), a searchable database of current health services research underway. HSRProj includes project descriptions and contact information for the investigator.
What funding is available to support rural health research?
In addition to funding provided by the Health Resources and Services Administration through the Office of Rural Health Policy and its Rural Health Research Centers program, these federal organizations also support research on rural health topics:
- The Agency for Healthcare Research and Quality (AHRQ) offers a number of funding programs that support health services research, which includes an interest in rural healthcare delivery
- National Institute on Minority Health and Health Disparities (NIMHD) supports health disparities research including an interest in rural populations
- The U.S. Department of Agriculture (USDA), through its Agriculture and Food Research Initiative
- The U.S. Department of Veterans Affairs, through its Health Services Research & Development Service, funds VA employees to conduct investigator-initiated research that includes a priority area for Healthcare Access/Rural Health
Other national organizations also support rural health research:
- The Patient-Centered Outcomes Research Institute (PCORI) is a non-profit, non-governmental organization authorized by the Affordable Care Act that supports patient-centered comparative clinical effectiveness research and is interested in residents of rural areas as priority population.
For specific funding for rural health research, please see the Funding section of this guide.
How do you select an appropriate rural definition for a research study?
A 2005 American Journal of Public Health article, Rural Definitions for Health Policy and Research, discusses the importance of using an appropriate rural definition in order to develop research findings that offer accurate conclusions. The three key considerations the authors identify for selecting a rural definition to use in a research project are:
- The purpose of the study
- Availability of data
- Appropriateness and availability of the definition
The authors recommend researchers learn about the various definitions and the pros and cons of each early in their research process. A snapshot of these pros and cons for common rural definitions is available in Table 1 of the same paper.
The paper points out not only the differences between urban and rural areas but also across rural areas, depending on their population size, geographic isolation, and other factors. Any definition will either under- or over-represent rural or urban in some way. Definitions that use only two categories, rural and urban, with no gradations within the rural category, may miss local issues that impact particular types of rural communities.
A 2013 article in the Journal of Environmental and Public Health, The Utility of Rural and Underserved Designations in Geospatial Assessments of Distance Traveled to Healthcare Services: Implications for Public Health Research and Practice, addresses many of the same concerns as the 2005 AJPH article and points out the common rural definitions used for health services research were not developed with health policy in mind, and so should be used cautiously.
The article reports on a study comparing eight counties in Central Texas, using various rural and underserved designations to examine the same health issues as part of a community assessment. Of the eight counties, only the most urban and the most remote county had consistent status as urban or rural across the definitions considered, illustrating how the selection of a rural definition can impact rural health research findings and resulting health policy.
For more details about current rural definitions, see the RAC topic guide What is Rural?
Are there concerns unique to rural areas researchers should keep in mind?
Aside from concerns about how to select an appropriate rural definition, additional issues rural health researchers should consider include:
- Challenges related to sample size, given the smaller population pool to draw on.
- Researchers and Institutional Review Boards need to ensure data is reported in a manner that protects individual and community identity. Sometimes, results will not be able to be reported for small geographic units because participants are potentially identifiable.
- The potential need for an on-site healthcare facility collaborator to recruit participants that meet the study requirements.
- When studying only one or a few rural communities, it may not be appropriate to generalize findings to other regions or other types of rural communities. The variety of rural communities, from Appalachia to Alaska, from island communities to mountainous areas, and from tourist-based economies with population fluctuations to areas with a steadier population, can impact many aspects of healthcare.
- Travel time and cost incurred by researchers, for methods that require an in-person presence.
- Cultural differences for both rural and tribal communities that may impact the most effective methods for engaging with study participants.
The 2013 Journal of Rural Health article Community Outreach and Engagement to Prepare for Household Recruitment of National Children’s Study Participants in a Rural Setting, identifies the following recommendations for undertaking a rural research project:
- Build relationships with rural organizations and healthcare facilities for 1-2 years prior to study, to help establish trust
- Connect with cooperative extension agents to learn about the community
- Help build research capacity at rural healthcare facilities, which may have limited research experience and infrastructure
- Engage with community members at parades, county fairs, and rural/ethnic heritage events
- Stay involved by being present at community events as the study continues
Researchers can also partner with the community in sharing findings, as a way of demonstrating the research effort is mutually beneficial.
Additional sources that discuss special concerns rural health researchers should consider:
- Recruiting Rural Participants for a Telehealth Intervention on Diabetes Self-Management, Journal of Rural Health, 2014
- Lessons Learned from the Conduct of a Multisite Cluster Randomized Practical Trial of Decision Aids in Rural and Suburban Primary Care Practices, Trials, 2013
What is community-based participatory research (CBPR) and how can it help rural and tribal communities and researchers work effectively together?
The Agency for Healthcare Research and Quality (AHRQ) defines community-based participatory research (CBPR) as
"an approach to health and environmental research meant to increase the value of studies for both researchers and the communities participating in a study."
When using the CBPR approach, community members, healthcare facilities, and other stakeholders work alongside researchers. This cooperation should begin as early as setting a research agenda and identifying community needs and continue through dissemination of research findings, implementation of interventions, and on to future studies. The development of an ongoing relationship between the community and the researchers is an important piece of CBPR. It helps develop trust and participation and ensures the work undertaken is relevant to the community and that findings and interventions benefit the community and the researcher.
A 2009 editorial in the Online Journal of Rural Nursing and Health Care, Community-Based Participatory Research: Toward Eliminating Rural Health Disparities, discusses the role rural communities can play in addressing the health needs of their residents through CBPR. A 2008 Clinical Interventions in Aging article, Evaluating a Community-Based Participatory Research Project for Elderly Mental Healthcare in Rural America, describes some specific aspects of applying CBPR in a rural setting. Researchers coming to a rural community can be perceived as outsiders, so developing an advisory board and building a presence in the community early on in the research process is important. The involvement for stakeholders may vary throughout the process due to their interests and ability to contribute to different stages of the work. Researchers may also face challenges to community engagement, depending on their research topic. For example, the project described in the Clinical Interventions in Aging article focused on mental health, which can have confidentiality and stigma concerns in a rural community that might be less worrisome in a more densely populated community. As a result, recruitment of community representatives to the advisory board was difficult. It is helpful to use a CBPR approach that can adapt to the needs of a particular community partnership.
The CBPR approach has promise to better meet the research needs of rural minorities and help address the health disparities they experience. American Indians, African Americans, Hispanics, and other racial/ethnic minorities have historically been the subject of research conducted by mostly white academics, rather than participants in the research process. As research subjects, they have experienced institutional racism from and mistreatment by research organizations, including cases such as the infamous Tuskegee syphilis study. The long history of research abuse experienced by American Indians is recounted in a 2012 Ethics & Behavior article No Meaningful Apology for American Indian Unethical Research Abuses. Minorities are also less-represented in clinical studies, another barrier to identifying effective interventions to address health disparities. A 2012 Health Services Research article, A Systematic Review of Community-Based Participatory Research to Enhance Clinical Trials in Racial and Ethnic Minority Groups, found the use of CBPR improved recruitment of minorities into clinical trials and helped determine the effectiveness of interventions for these groups.
Tribal communities have an interest in CBPR, and in particular tribal participatory research (TPR), as a way to ensure research conducted will contribute to the health of tribal members and be respectful and appropriate to a tribe's culture, with power distributed equally between the tribe and the researcher. TPR, as defined in the 2003 American Journal of Community Psychology article, Tribal Participatory Research: Mechanisms of a Collaborative Model, involves:
- Tribal oversight and control of the research process
- A formal resolution from the Tribal Council identifying the goals and methods of the project
- An oversight committee with tribal authority to approve or deny project activities
- A tribal research code that governs the research process
- A facilitator for meetings between the oversight committee and researchers
- Community members hired and trained to work on the project, which is important to build the tribe's research capacity
- Culturally specific methods that are aligned with community values
There has been a history of researchers coming into tribal communities and conducting research that did not benefit participants. Tribes may find CBPR useful as a way to prevent this type of encounter and to help shape research that does meet their members' needs. The National Congress of American Indians Policy Research Center 2010 report, Community-Based Participatory Research in American Indian and Alaska Native Communities, discusses CBPR in tribal communities and includes several case studies. The report also provides an overview of the tribal government approval and oversight required for conducting research with tribal communities. It discusses the importance of considering different cultures when a research project includes more than one tribe or tribal community.
A 2009 literature review in Pimatisiwin: A Journal of Aboriginal and Indigenous Community Health, Contextualizing CBPR: Key Principles of CBPR Meet the Indigenous Research Context, covers the key principles of CBPR in the context of tribal research. This article and the TPR article identify additional considerations:
- Recognize the historical research experiences of tribal communities and work to improve the image of research
- Understand tribal sovereignty
- Learn about tribal membership and community membership
- Understand the differences among tribes
- Expect a long review and approval process and allow time to build trust
- Be prepared to allocate a large portion of project funding to oversight and planning
- Identify key tribal members, such as tribal elders and tribal government representatives
- Anticipate turnover in tribal leadership
- Learn about and incorporate the tribe's spiritual and philosophical beliefs into the project
- Consider the tribal context when interpreting the data
- Obtain tribal approval for publication of study results
- Recognize increased tribal capacity as a valuable outcome of the project
What is comparative effectiveness research (CER) and how can it help us understand how well specific healthcare interventions work for rural residents?
Comparative effectiveness research (CER) helps inform healthcare decision-making by comparing different treatment options. It looks at how well a particular procedure, medication, test, or other healthcare intervention works compared to other options. CER may focus on how well a treatment works for a particular racial/ethnic group, for a specific age group, or for one gender. When focused on rural health, CER examines healthcare outcomes for a particular treatment or intervention used in a rural setting.
By gathering evidence about the effectiveness of different treatments for rural populations, CER can help support:
- rural providers and healthcare facilities deciding on what interventions to use
- rural patients and their families selecting among treatment options
- health insurance companies and other payors deciding what preventive care and treatments to reimburse and at what level
- policymakers supporting effective approaches in a rural setting
The National Academy for State Health Policy offers a step-by-step look at how CER can be used to inform health policy: A Roadmap for State Policymakers to Use Comparative Effectiveness and Patient-Centered Outcomes Research to Inform Decision Making.
The Affordable Care Act established the Patient-Centered Outcomes Research Institute (PCORI), which seeks to help patients and providers make informed healthcare decisions based on evidence of effective treatment. PCORI funds studies that address priority populations, including residents of rural areas, and summarizes findings from those studies. According to the FAQs for PCORI's engagement award program,
“PCORI recognizes that doing research differently will require input and involvement from all communities across the healthcare continuum. PCORI is interested in investing in stakeholders who believe in the promise of PCOR/CER but may not have had the opportunity to engage actively in research in the past.”
A November 2013 PCORI report, Notes from the Field: Improving Healthcare Outcomes in Rural Settings, describes a stakeholder meeting that identified rural CER research priorities, which include comparing interventions that address:
- Healthy aging in place
- Transitional care models for different populations
- Telemedicine and other innovative delivery models
- Health information communication and dissemination
The same report also summarizes the rural-focused PCORI studies funded through October 2013. A searchable list of all PCORI awards is also available.
What is the role of practice-based research networks (PBRNs) and what are some examples of rural PBRNs?
Practice-Based Research Networks (PBRNs) are networks of healthcare providers who seek to improve community health by translating research findings into clinical practice. Often PBRNs also include academic researchers as part of the network. The Agency for Healthcare Research and Quality provides an overview of primary care PBRNs.
Because a PBRN's research agenda is shaped by practicing clinicians, it can focus on with practical relevance for patient care. As discussed in the 2014 Journal of Rural Health article, Recruiting Rural Participants for a Telehealth Intervention on Diabetes Self-Management, academic researchers can develop stronger connections to rural communities and potential research participants by working with local healthcare providers via PBRNs. Involvement in a PBRN can also help rural clinicians build their local research capacity.
Some of the roles of PBRNs, according to AHRQ, include:
- Conducting comparative effectiveness research
- Supporting quality improvement within the member primary care practices
- Developing an evidence-based culture
AHRQ maintains of list of PBRNs including several rural PBRNs, such as:
- The Frontier Rural Innovations Network, a national PBRN focused on rural healthcare delivery.
- The High Plains Research Network, which serves eastern rural Colorado and undertakes research studies and quality improvement projects on many topics.
- The Oregon Rural Practice-based Research Network (ORPRN), a group of primary care practices and community organizations in Oregon focused on improving the health of rural residents. ORPRN conducts studies on a range a health topics and primary care delivery issues.
- The Texas A&M Health Science Center Rural and Community Health Institute, another state-based rural PBRN.
What are some different types of assessments relevant to rural health?
An assessment seeks to understand the current state of affairs in a community, organization, or program. Rural healthcare facilities and rural stakeholders conduct assessments as a first step in deciding how to strengthen rural health services. For example, you might assess:
- Comprehensive community health needs, as required for non-profit hospitals under the Affordable Care Act and for public health agencies seeking Public Health Accreditation Board (PHAB) accreditation
- A healthcare facility's attractiveness to potential healthcare providers by identifying facility and community strengths and weaknesses, for example through the Community Apgar Questionnaire
- The need for a particular healthcare service in a community
- The healthcare status of a particular rural population, such as a racial/ethnic group or the elderly
- Health needs related to a particular health condition in a rural community
- Rural healthcare workforce supply and demand, for one or many professions
What are the main steps in planning for and conducting an assessment?
Rural communities planning to conduct an assessment will benefit from thinking about the resources available in their particular community that can support the assessment activities. They can leverage existing relationships and networks, both formal and informal, to gain support and partners in the assessment process. They can make use of the hospital newsletter, local newspaper, radio station, and other local media to inform and engage community members. Already-scheduled local events, such as regular meetings of service organizations, are another opportunity for weaving the assessment activities into community life.
While the details of an assessment will vary depending upon its purpose, the broad steps are similar:
- Gather the interested parties or stakeholders. Think about and network with other organizations in the community or region. Who else could benefit from the information being gathered? Who should be at the table to address the suspected issues or concerns? Who would be willing to contribute resources?
- As a group, define what is being assessed. What geography? What types of services? Provided to whom?
- Identify the goals of the assessment and the purposes for which you anticipate the findings will be used.
- Locate sources of existing data that can answer questions suggested by the assessment goals.
- Gather information through focus groups, interviews, surveys, and public forums.
- Analyze the results, including the development of maps and other types of reports that show what has been learned. Consider including recommendations for next steps.
- Share the findings.
- Consider making assessment a recurring activity, for example repeating annually or every 3-5 years, rather than a one-time task.
The Community Toolbox, a free service from the Work Group for Community Health and Development at the University of Kansas, includes a section on Assessing Community Needs and Resources that describes specific steps in the assessment process.
What are the Affordable Care Act requirements for hospitals to conduct Community Health Needs Assessments (CHNAs)?
The Internal Revenue Service bulletin, Additional Requirements for Charitable Hospitals; Community Health Needs Assessments for Charitable Hospitals; Requirement of a Section 4959 Excise Tax Return and Time for Filing the Return, issued on February 2, 2015 contains the final regulations regarding ACA requirements for non-profit hospitals, which includes a requirement to "conduct a community health needs assessment (CHNA) and adopt an implementation strategy to meet the community health needs identified through the CHNA at least once every three years."
On an annual basis, hospitals must provide a description of the actions taken during that taxable year to address the needs. The implementation strategy should seek to address the needs identified in the CHNA. In setting forth how it plans to address identified needs, a hospital should:
- Explain what actions it will take and the anticipated impact of the actions,
- Identify the programs and resources it plans to commit, and
- Describe any planned collaboration between the hospital and other facilities or organizations
The IRS document begins with information on how the regulations evolved over time, identifying previous IRS publications that addressed the requirement. The final regulations appear at the end of the document in section §1.501(r)–3 and describe the CHNA requirements in detail.
What are the community needs assessment requirements for public health agencies related to the Public Health Accreditation Board (PHAB) accreditation standards?
One of the prerequisites for public health agencies interested in pursuing accreditation is the completion of a community health assessment (CHA) and a community health improvement plan (CHIP). A July 2012 guide from the Public Health Accreditation Board, National Public Health Department Accreditation Prerequisites, includes details on the CHA and CHIP requirements. Similar to the CHNA requirements, the CHIP should address the findings from the CHA.
How can rural hospitals and public health agencies work together to conduct community assessments?
The requirements rural hospitals face under the ACA and local public health agencies must meet to pursue accreditation are similar. In rural communities, the stakeholder groups for each assessment overlap and the financial resources to undertake a study are limited. It makes sense for the hospital and public health agency to work together.
A 2012 Public Health Institute report, Best Practices for Community Health Needs Assessment and Implementation Strategy Development: A Review of Scientific Methods, Current Practices, and Future Potential, identifies a number of reasons for rural hospitals and public health to work together, including:
- The large geographic service areas of many rural hospitals and public health agencies may overlap
- Each organization's influence with different groups of stakeholders
- Sharing staff expertise and in-kind resources
- Similar missions and responsibility to community health
- Commitment to collaborate on similar activities
- More likely to have institutional flexibility and history of working together
The PHI report also discusses how public health and non-profit hospitals can work together to assess community health and plan for improvement. Some of the issues highlighted include:
- The need to build "shared ownership of community health"
- Addressing differences in the services areas of the hospital and public health agency
- The benefit of collaborating in data collection, which can be resource-intensive
- The roles/involvement of the hospital, public health, and stakeholders in setting and addressing priorities
- Collaboration in evaluating community benefit program results, with hospitals potentially benefitting from the evaluation expertise of public health staff
- Jointly sharing findings and progress on improvement plans with the community
What CHNA tools and resources are available for rural facilities?
There are many different CHNA processes available, including examples that have worked well in rural areas. Rural hospitals should consider a range of options to identify an approach that is a good match for their community.
The National Center for Rural Health Works offers a Community Health Needs Assessment Toolkit developed using a national advisory team and tested in several rural communities. This toolkit, while designed for rural hospitals, can also be modified for use by other types of organizations conducting similar assessments. The introduction of the toolkit addresses the legislative requirements, the main sections walk through a step-by-step process, and the appendices include examples and key contacts. The toolkit is also available in sections, with additional background information on the CHNA.
The National Center for Rural Health Works also offers training and technical assistance related to CHNA. To request assistance, please contact the Center at 405-744-9823 or email Cheryl St. Clair at firstname.lastname@example.org and Gerald Doeksen at email@example.com.
Community Commons also offers resources for learning about the CHNA and a process for conducting one, Community Health Needs Assessment (CHNA). There are several video tutorials available to use the Community Commons CHNA tool, as well as an extensive list of CHNA resources and tools.
Other resources for learning about and conducting CHNAs:
- Resources for Implementing the Community Health Needs Assessment Process, Centers for Disease Control and Prevention
- Community Benefit / Community Health Needs Assessment, National Library of Medicine Health Services Research Information Central
- Assessing and Addressing Community Health Needs, Catholic Health Association
- Webinar recording: Understanding the Hospital Community Benefit Requirement and the Community Health Needs Assessment, County Health Rankings and Roadmaps
- Your State Office of Rural Health
- Your state hospital association
What are the different purposes evaluation can help address?
Program evaluations can serve many purposes, all with the goal of ensuring resources invested in the program and in future programs are well spent. Key purposes are:
- Program Improvement
Program improvement goes by many names, including process evaluation and formative evaluation. It is important a program is implemented as intended. If it isn’t, then there is less chance the program will be effective, meeting its goals and objectives. Each step of how the program is supposed to be delivered should be written down. The evaluation then monitors whether each step happened as planned and, if not, provides suggestions for improvement. Rural communities, in particular, often must adapt programs that have not been implemented in a rural area. The evaluation can help fine-tune those adaptations for the rural setting.
A funder often has some minimal data they would like the program to track, such as the number and type of participants and the number and type of activities being conducted. This information doesn’t say whether the program is being delivered according to plan or is having it intended impact, but does give the funder a general sense of whether the things they expect to be done are actually done.
- Impact or Effectiveness
Outcome evaluation focuses on the results the program achieved. The results of an outcome evaluation will help determine what difference the program makes to the participants; its impact. The information gathered can help an organization decide whether it is worthwhile to keep offering the program and help funders decide whether to support the program. The impact of a program will also determine whether other rural communities are likely to want to try a similar intervention.
Logic models are a way to ensure the program assumptions are linked to activities and measurement. For example, a project intended to address obesity through a walking club might start with this:
|Program assumption||physical activity reduces obesity|
|Measurement||minutes walked, BMI|
According to a 2002 American Journal of Evaluation article, A Three-Step Approach to Teaching Logic Models, when these three elements are linked there is an increased likelihood the program will be successful and the success can be measured. When done correctly, a logic model is a useful tool to illustrate how a program's activities are intended to impact the world.
There are many types and guides for developing logic models. This is typical logic model layout:
Source: State Heart Disease and Stroke Prevention Program Evaluation Guide: Developing and Using a Logic Model, Centers for Disease Control and Prevention
To learn more about logic models:
- The Community Toolbox includes a section on developing logic models.
- The presentation, Logic Model: Components and Implementation, by John Gale of the Maine Rural Health Research Center, while it focuses on the Flex Program, also provides a general overview of logic model development.
- The presentation Defining Focus and Measuring Performance: Logic Models Are Your Friends by Alisa Druzba of the New Hampshire State Office of Rural Health discusses the benefits of developing a logic model.
Why is it important to evaluate rural health programs?
Rural communities face many challenges related to healthcare delivery and population health but also have limited resources to address these challenges. Program evaluation can help ensure the investment of staff time, organizational will, and other resources are well directed. Program evaluation can also help demonstrate to funders that a program is a worthwhile investment.
Another important reason to evaluate rural interventions is to show what works in a rural setting. Many of the evidence-based approaches that federal programs or foundations may request applicants use were demonstrated to work in a non-rural setting. Seeing how well these interventions work, or what adjustments might be needed to make them effective and practical in a rural setting, is a key purpose for evaluating rural health programs.
How can rural programs plan for and conduct efficient and practical program evaluations?
Wilder Research, a non-profit organization that conducts program evaluations, in its factsheet Evaluation on a Shoestring Budget, offers guidance to any organization with limited resources. Another source of tips comes from Australian program evaluator Patricia Rogers, in her Evaluation on a Shoestring blog post from the BetterEvaluation.org website. Recommendations from these sources include:
- Plan your evaluation at the outset of the program
- Prioritize the evaluation's focus based on stakeholder needs
- Limit the scope of the evaluation
- Select approaches that are easy, appropriate to staff skills and time available, and fit your budget
- Collaborate on the evaluation with project partner organizations
- Consider seeking assistance from a local college or university
What tools are available to help rural grantees learn about program evaluation?
The National Rural Health Resource Center hosted several rural health program evaluation workshops and shares the program evaluation workshop materials, which cover topics like steps in the evaluation process and evaluation best practices. It also includes adaptable templates and a list of additional resources.
The State Offices of Rural Health offer program evaluation technical assistance to rural grantees. Depending on your funding source, you may also be able to get technical assistance through your grant program or funding organization.
Some other sources for learning about program evaluation include:
- The Non-Researcher's Guide to Evidence-Based Program Evaluation, National Registry of Evidence-based Programs and Practices (NREPP), Substance Abuse and Mental Health Services Administration
- The Community Toolbox section on Evaluating the Initiative
- Evaluating your Community-Based Program guide from the American Academy of Pediatrics
How can funders ensure their grantees' experiences help build the collective understanding of what is effective in addressing rural health issues?
Funders can encourage and support their grantees to share their project results and help other rural communities learn about what is effective. Requests for proposals can include requirements for program evaluation. Funders can offer financial support, learning opportunities, and technical assistance to grantees regarding program evaluation.
They can also include funding or provide encouragement to program staff to write up their program results for publication. Publication in a peer-reviewed journal is an important step in making a program or model part of the evidence base.
Funders are in a great position to identify projects that were particularly successful. They can recognize those programs themselves through an award or honor or recommend the projects to other sources that identify model programs. RAC's Rural Health Models & Innovations, for example, is a collection of rural project examples, including evidence-based approaches. RAC welcomes suggestions for additions to this collection.