MHSC Scan: January 1996

MHSC Cites Need for Outcome Evaluation

The ultimate goal of the extensive resources devoted to human services here in Frankin County is improving peoples' quality of life. At MHSC, we believe that measuring the effect these services have on people's conditions is an integral part of a quality community planning process that will, indeed, improve people's lives. Most recently, we have focused our attention on one aspect of that process, evaluating outcomes.

Outcome evaluation can tell us whether or not our resources are making changes in people's conditions. Often, our experiences and instincts tell us that the strategies we use improve the quality of life in Central Ohio. Infrequently, however, do we systematically measure and illustrate the impact of such strategies. Why is this?

All too often it is because evaluation costs money and doesn't get budgeted as part of a specific strategy. Some funders are unwilling to consistently allocate money for evaluation. Sometimes, it is because the program sets goals and objectives that are process or service oriented and thus are not suited for outcome evaluation. Or perhpas we want to see results in too short of a time period. Often, evaluation doesn't occur because it is difficult to isolate factors to be measured.

For example, the Juvenile Delinquency Task Force Implementation Committee has a truancy prevention program for children. It is difficult to isolate the program's impact from the multiple other factors that affect the child's school attendance. The role of a neighbor, teacher, minister or grandmother -- or the fact that Mom got a job -- may be responsible for changes in a child's truant behavior. While we can measure the changes in the behavior, we cannot be sure that it was the truancy program's strategy that brought about that change.

The fact that it is difficult work should not stop our efforts to conduct outcome evaluation. MHSC believes that outcome evaluation should be an intrinsic, funded and expected part of the human services system. Not only is it the appropriate thing to do, these days it is also the politically expedient thing to do. In a time of shrinking federal resources, the call for documented accountability is heard clearly everywhere. And, whether you are a public entity or a business with a foundation that funds human services, you and the community can benefit from knowing if your efforts improved people's conditions. Human services professionals could benefit from knowing that their day-to-day efforts do make a difference for people.


WHAT ARE OUTCOMES?

Outcomes assess the extent to which methods and strategies result in the desired changes in people and the community. It makes sense, then, that we understand outcomes as needing to talk about people and their conditions. Outcomes do not measure inputs, processes or methods. Instead, they measure the extent to which a strategy is reaching its objective.

Outcomes, to be useful, should tell:

Focusing on a specific example may best illuminate both the process and data involved in outcome evaluation. Many different systems in the community address the goal of enhancing the health of teens. They begin by defining what they mean by "healthy" -- drug free, enrolled in school, living in loving homes and not having children. For example, an individual system, like the Columbus Health Department, might set a goal of reducing the number of teens who have babies, based on its findings that pregnant teens are more likely to receive inadequate prenatal care, drop out of high school and live in poverty as a result of their pregnancy.

The focus on teen pregnancy is a good example, as the percentage of Franklin County teenagers giving birth has been increasing since the late 1980s. During 1993, the latest year for which figures are available, 12.8% of Franklin County births, or 2,104 babies, were to teens. 1,811 of these births were to unwed teens.

Given this context, what might be an appropriate measure to show how the system's strategies affected the number of teens who give birth?

By the year 2000, teen births will represent 7.8% of total births, as measured by the births to teens as a percent of total Franklin County births.

This outcome includes all the components of a good outcome statement; it focuses on who is going to do what, by when, and how it will be measured.

MHSC fully realizes that there are administrative, process and financial aspects of conducting evaluation. However, from a community-wide perspective, the ultimate question is whether or not people's conditions improve. This is, of course, central to MHSC's mission and why we believe evaluation is so important. MHSC continues to encourage and support agencies and organizations which try to incorporate outcome evaluation in their planning and service provision processes. Changing behaviors around outcome evaluation can occur if we value it, train for it, and fund it. And, the results will be seen in the improved quality of life we and our neighbors experience in our community.

Staff member Pat Deering contributed to this article.

OUTCOME EVALUATION and QUALITY COMMUNITY PLANNING

Outcome evaluation is one of four elements that MHSC identifies as key to quality community planning. Funders and providers of human services will be engaging in quality community planning when they:

Viewed within the context provided by quality community planning, programs have people-centered goals and objectives which are connected to a vision focused on people and their conditions. These goals and objectives can then be translated into measurable outcomes which identify desired changes in people and conditions.

A number of organizations have focused efforts to incorporate outcome evaluation in their planning processes. Here are just a few samples of those efforts.

United Way's Work on Outcomes

A United Way draft report suggests that good outcome statements include the following items:

intent or expected change;

magnitude of the expected change;

target of the intervention;

time frame in which the intervention will occur;

measure(s) of the expected change.

Objectives of Franklin County ADAMH Board's Prevention Efforts

Franklin County Children's Cabinet

In its Member Agreement dated 9/6/95, the Franklin County Children's Cabinet cites its first objective to be:

"To encourage organizations to develop and track measurable outcomes toward achieving that vision."

The Cabinet has begun work on establishing a vision for all children in Franklin County.

Franklin County Board of Mental Retardation and Developmental Disabilities

The MR/DD system establishes an Individualized Education Plan (IEP) for each of the students receiving services. The IEP contains specific goals, objectives, strategies and measures, as shown here:

MHSC INVITES NOMINEES FOR HUMAN SERVICES AWARDS

The Metropolitan Human Services Commission invites you to submit nominations for its Barry Mastrine and Grace Kindig Awards. Both awards honor people involved in human services in the Franklin County community.

The Mastrine Award is being given for the first time. At its October 1995 meeting, the MHSC Board established the award to honor Mr. Mastrine, who was MHSC's first executive director. Mr. Mastrine died last July. The award honors professionals whose work promotes cooperation and coordination among human services organizations in order to promote the dignity, well-being and self-sufficiency of Franklin County residents.

The award reflects Mr. Mastrine's commitment to the community and his leadership style. His numerous community activities focused on improving services for children, persons with mental illness and homebound individuals. Professionally, as President of The Davon Group, he provided consulting services to community organizations involved in strategic planning projects.

The Kindig Award has been given to 30 individuals and organizations since its creation in 1979. It honors volunteer individuals and organizations whose work promotes cooperation and coordination in human services.

Mrs. Kindig was a member of MHSC's first Board of Trustees from February 1977 until declining health forced her resignation in November of 1978. Her professional and volunteer involvement with issues around health planning, mental health and cerebral palsy made her a well-known and well-respected advocate of human services.

You may obtain a nomination form by calling MHSC at 224-1336. Nominators and their nominees will be informed of their selection and presentation details during early 1996.

NANCY COLLEY JOINS MHSC BOARD

The Metropolitan Human Services Commission appointed Nancy Colley to the Board at its September meeting. Ms. Colley is Human Resources Chief for the Ohio Deparment of Aging. She was designated for appointment by the Franklin County Board of Commissioners. Ms. Colley is an active community volunteer with several organizations, including the Hannah Neil Center for Children, Jefferson Music Academy, Hillsdale College and Columbus Symphony.

MHSC Expects ADAMH and FCCS To Go To Voters in '96

The MHSC Standing Committee for Levy-Funded Human Services will gear up early this spring to review levy requests by the Franklin County Alcohol, Drug Addiction and Mental Health Services Board (ADAMH) and Franklin County Children Services (FCCS). Failure of ADAMH's 2.4 mill levy last November necessitates another request in 1996. At its Board meeting on December 13, ADAMH decided to forego the March primary date. The current FCCS levy expires in December of 1996. Voters last approved its request for a 3 year, .8 mill levy in November of 1993.

According to Committee chair Jim Bowman, the Committee will immediately begin gearing up for the reviews. "It will be extremely important that we have a schedule set for meetings with both agencies, that we do a thorough review of their requests and have our own analysis and recommendations ready for the MHSC Board by late July," he commented at a recent meeting.

As a result of its levy failure, ADAMH is cutting $3 million dollars from the first six months of its 1996 budget. It proposes to do so by not making about $800,000 worth of performance awards earned in 1995 by its contract agencies and by delaying implementation of its centralized intake process. Other cuts in administrative costs and agency funding will make up the rest of the $3 million in cuts. ADAMH says additional cuts may be necessary in the second half of 1996. ADAMH is currently considering its options for next year's levy, particularly in light of its current budget cuts and service needs.

FCCS Board and staff are also considering a levy request for voters next year. At the November Standing Committee meeting, FCCS Executive Director John Saros presented a report detailing the status of the agency, its current caseload numbers and financial statements. The agency has faced substantial budgetary woes this year, which FCCS staff attribute to higher costs necessitated by larger numbers of teenagers being ordered into care by the Franklin County Court system. FCCS had 1415 children in paid care during the first nine months of 1995. The entries of young children into the system declined (from 39% to 29%) while the entries of teenagers increased (from 37% to 48%). Current figures show that it placement costs for teens are higher than for other age groups. The average cost per day, for the first ten months of 1995, are $25 for toddlers and $91 for teens. The FCCS Board will consider these costs and service needs as it develops its levy request for 1996. It anticipates making a specific levy request proposal during the spring.

The Scan is published by the Metropolitan Human Services Commission. MHSC is a comprehensive human services planning agency that provides data, assessment and decision making assistance. Questions or comments can be directed via email to claudia@pie.mhsc.org (Claudia Herrold) or by telephone at 224-1336.