Caring for Kids: Expanding Dental and Mental Health Services through School-Based Health Centers

Expanding Mental And Dental Health Services
within School-Based Health Centers:
Operations and Evaluation Challenges


Mental Health Challenges

The eight participants in Caring for Kids initiative that are developing mental health interventions met in two workshops to discuss their projects and exchange views on operational and evaluation challenges.

Project plans

Lynn Community Health Center (Lynn, MA). Lili Silva, Director, School-Based Health Centers. The Lynn Community Health Center operates eight school-based health centers. All our centers have a mental health provider who is on site about 20 hours a week. Our proposal looks to expand our services by bringing on a psychiatric nurse specialist and a pediatric psychiatrist to help us address the issue of medication management. We also plan to provide consultation services and support to the mental health providers in the other school-based health centers. The psychiatric nurse specialist will be based at one of our new school-based health centers that opened this past year. She will be the primary behavioral health provider at that site as well as a consultant to providers at the other seven centers. Her job will be to see what issues come up in medication assessment and medication management. Our goals are to do a better job of staying in touch with our students, providing continuity of care, but not contributing to duplication of services.

Buncombe County Health Center (Asheville, NC). Nelle Gregory, School Health Supervisor, Buncombe County Health Department. The Buncombe County Health Center, a part of the Buncombe County Health Department, sponsors three school-based health centers -- two urban and one rural. We will expand mental health services in the centers by hiring two licensed clinical social workers to join the one licensed clinical social worker (LCSW) who currently serves the three centers. In the long run, we hope to partially fund the mental health program with Medicaid and NC-CHIP money. In North Carolina we can receive Medicaid and NC-CHIP funding for up to six mental health visits that do not have mental health (DSM-IV) diagnoses. Thus we can use v-codes for these initial visits. We hope to do in-depth evaluation of the v-codes we see on those first six visits.

University of New Mexico Center for Health Promotion and Disease Prevention (Albuquerque, NM). Nissa Patterson, Health Education Manager, ACL Teen Centers.

This grant, along with another, will support a master's level mental health professional 30 hours a week at our two ACL Teen Centers. Our focus is on catching and helping kids before they're really in trouble. Pediatric and psychology interns will also participate in this phase of our work. The project psychiatrist will provide consultation to staff on-site one day a week. Because this is a rural site, we also want to increase the capacity of community providers. Therefore, the project psychiatrist will provide quarterly in-service sessions for staff and community providers together. We also plan to increase our capacity and effectiveness in intervening on substance abuse by training staff and community providers in substance abuse interventions developed at the UNM Center on Alcoholism, Substance Abuse, and Addiction (CASAA).

Seattle-King County Public Health (Seattle, WA). Linda St. Clair, Manager, Youth Health Services, Seattle-King County Department of Public Health. We manage a system of 13 school-based health centers in Seattle Public Schools. All our school-based health centers currently have mental health counselors. This grant enables us to add psychiatric fellows from the University of Washington to provide consultation on site to staff at seven of the centers. On a limited basis the fellows will also provide direct care to students referred to them.

Children's Hospital of Los Angeles(CHLA) (Los Angeles, CA). TuLynn Smylie, Director, School-Based Health Program, CHLA. Children's Hospital sponsors three school-based health centers in LA High, Hollywood High and Manual Arts High School. Our challenge is to figure out how to sustain mental health services. We have provided mental health care on and off through the years but nailing down the funding has eluded us. With this grant we are expanding services by reviving a program of psychology interns. A new project coordinator will enable us to document performance improvement and outcomes. We will also focus on maximizing third-party revenues by employing a part-time bilingual enrollment specialist to facilitate utilization of Medi-Cal and EPSDT funding.

Children's Aid Society (New York, NY). Scott Bloom, Director, Mental Health, School-Based Health Clinics, Children's Aid Society. Children's Aid Society has five comprehensive school-based health centers, ten dental clinics, and a number of mental health clinics in schools that do not offer comprehensive services. Our grant will do a number of things. We want to expand services, particularly at one of our schools, IS-218, which has more than 1,700 kids. We will do this by hiring a Senior Social Worker. We also added another staff social worker, so now we have three staff social workers plus the senior social worker. In addition to increasing capacity, we want to sharpen diagnosis and treatment approaches. We will use the computerized Diagnostic Interview Schedule for Children (DISC) tool to identify children with problems. We also plan more outreach to teachers, especially to do training around appropriate referrals. And we will do workshops with parents on adolescent development, how to talk with your teens, etc. Our plate is pretty full.

Schneider Children's Hospital, North Shore-LIJ Health System (New York, NY). Martha Arden, Director, School-Based Health Services, Schneider Children's Hospital. Schneider Children's Hospital sponsors three urban school-based health centers: two are high schools and one is a middle school. With the grant we will expand and reorganize the mental health program at the school-based health center at Franklin Lane High School. 3400 students attend this school and 90% of them are enrolled in our clinic. Our current mental health program includes a social worker supervisor and a staff social worker, as well as another staff social worker and program coordinator funded by a sexual abstinence support grant. With CFK funding, we will institute standardized assessments and treatment planning conferences in order to assign patients to mental health services in a structured fashion, distinguishing those who would respond to social and emotional support from those who have diagnosable mental illness. Support will be provided in a group format, while individual therapy will be provided using evidence-based, outcome-oriented therapy methods. A psychologist will be added to our mental health team, providing expertise in these treatment methods and making our program eligible to bill for Medicaid for mental health services, as social work visits do not qualify for reimbursement in NewYork State. Social work graduate students will increase our program capacity by administering computerized assessments, assisting with groups, and helping families obtain insurance coverage.

Dorchester County Health Department, (Cambridge, MD). Kathleen Wise, Program Manager, Healthy Families, Dorchester County Health Department. Dorchester County, Maryland's largest county at low-tide, has four school-based health centers sponsored by the Health Department. We are using some of our grant to hire an additional social worker so that we now have four licensed clinical social workers (LCSW) at the two high school and two middle school health centers. Three additional LCSWs provide care at the elementary schools. In Maryland, LCSWs are reimbursed by Medicaid IF there is a DSM-IV diagnosis. We will use the grant period to explore the possibility of billing for commercial insurance payments for these services as well. We plan to secure psychiatric consultation not only to work with the staff but also to see children for medication evaluation. Finally, it is our hope to secure some intra-office coordination with the Health Departments addictions program so that we might bring some of their services into the schools.

Operational Issues

Bruce Guernsey, group facilitator. As a person trained in clinical social work, one of the reasons I left personal health care and went into public health was that I was looking for a more holistic, interdisciplinary approach to health. But I didn't find what I was looking for in public health either. In our state, when it comes to health services, public health is nursing-dominated just as personal health care is physician-dominated. So, I kept looking. And I stumbled into school-based health centers. And while the people I met in the field initially saw school-based health centers as a promising means to address teen pregnancy, the first thing that dawned on me was, this is a promising way to deliver mental health care and I got very excited.

What this session is about are the operational issues confronting your programs. Where do you want to end up in three years? And what do you need to do now to make sure that is accomplished?

Issue 1. Introducing change: How do we gather support for what's new?

Scott Bloom, Children's Aid Society: In our Caring for Kids project, we are adding new structures and procedures to more clearly identify who needs services and to make sure we match a student's need to the most appropriate type of care. We are going to use new in-take forms and use more time-limited therapies. And I can see it will be difficult to make that transition because our staff members are used to the earlier, "looser" way. So our question is: Since we've been in operation for awhile, how do we get people comfortable with and accepting of these new procedures?

And the changes affect not just our staff. They affect student users, teachers who refer to us, and the community-based providers to whom we refer.

The role of vision and mission

Guernsey: This may be where vision and mission fit in. Maybe this is when people need to get together and articulate what the changes will be and try to develop buy-in.

Beverly Colon, Children's Aid Society: Our program is doing a Retreat Day before school begins. We are going to use the summer to get our staff on the same page. We hope that when September gets here, we can then take the program to the next level. This should work well for us because our whole school-based health center has a couple of days before school when no one can take leave and we all -- medical, dental, mental health -- the whole team comes together, puts on t-shirts, and has a full-scale retreat.

Our task is to get everyone -- both in the mental health team and in the health center at large -- used to the fact that we are no longer going to be unstructured. It used to be that referrals came in without anything written down. We need to make sure that the right response is developed. We have peer mediation. I like peer mediation but I need to know that it is right for a specific kid in a specific situation. But however this plays out, we have to create boundaries and structures in the school. We cannot continue with our current pattern which is that the social workers believe they must do whatever the teacher asks.

Guernsey: So the retreat will be an opportunity to define vision and mission and introduce changes in policies and procedures. And it will help you articulate yourselves to the broader school community

Challenges and Responses in
SBHC Mental Health Service Development

Accustoming staff to new structures and procedures

  • Use staff retreats to get all members on same page
  • Develop marketing plan for SBHC to explain and promote new arrangements
  • Introduce mental health program or program changes at back-to-school faculty meetings, with more extended discussion to follow at breakfast or lunch meetings at the center
  • Plan informational activities with the school population as a whole; provide feedback on new activities or changes to school staff

Developing a cross-disciplinary mental health practice

  • Develop a SBHC program report card to share service and other data with staff
  • Work to develop a team practice with a vision and mission statement that is backed up with policies and procedures

Defining the roles of medical professionals in providing mental health services

  • Formalize the case conferencing process with regularly scheduled meetings and agendas. Include mental health and medical staff together.
  • Clarify roles and responsibilities of all staff

Building relationships with parent organization

Participants agreed this topic included major challenges that were not always under their control, whose solutions were not readily apparent. Thus this topic remains just that - challenges to be addressed with responses to be developed as the programs mature.

  • Challenge of building a positive relationship and figuring out how to address specificissues inherent in small program/large institution dynamics
  • Challenge of sponsoring organization moving to paperless record. How do you make sure that new record is usable at the school-based site?


Using strategic communication tools

Unknown: One of the things that has helped us is that we took advantage of The Robert Wood Johnson Foundation offer for strategic communications training. That training was very useful in thinking about these issues -- communicating strategically with teachers, community providers, our sponsoring institution, etc. The training has helped us stop and say, "Where are our target audiences? What do we want them to know? What do we want them to do?" So we've created a communication action team that is going to help develop messages and think about how they get delivered.

Guernsey: How we deliver these messages really matters. Even if the basic words are "You've been a great faculty and staff and we really appreciate your connecting us with students. We have some plans to make it work even better." But you know not every group will bite or bite as quickly on this message. What are you doing that works?

Mona Carey, Dorchester County: We feed them. We invite people for lunch. We invite teachers for lunch. We invite other agencies for lunch. Space is always a problem but we make it work because we have to. Sitting down and breaking bread together seems to mean we can talk about some of these things in a non-threatening way.

Guernsey: One lesson from my experience is that lunches are good places for discussion and negotiation; the annual faculty meeting at the first of the school year is not. The back-to-school meeting is good to find out what is going on and what the school's current concerns are. But it's not a good time for extended conversation. The school has a lengthy agenda and our information can get lost or distorted.

Unknown: Sometimes there is an opportunity if your school does workshops for CEUs at the beginning of the school year. Then the school may want people who can teach those things. And the faculty members that come will be interested. But it is a difficult time of year to get people's attention.

Colon: One thing we are doing this year is to invite faculty to our school-based health center retreat. We will let you know how it works out.

Unknown: We're testing moving from the sound bite at the big meeting to a more extended discussion at the departmental meetings. At the middle and high school level the schools have departments and they have meeting times. They're real intimate. Also the kinds of referrals you get from departments vary so this makes different conversations more helpful.

Maintaining changes

Unknown: Another angle on this problem of introducing change is how to maintain it. I can see us doing real well in September and October and then beginning to backslide. How do you maintain momentum?

One thing I learned from past experiences in school is that that environment is used to seeing new initiatives come and go. The prevailing view is "This too shall pass." So we should plan for backsliding to happen. Knowing it will occur, how do we re-energize our efforts periodically?

Guernsey: Would it be possible to provide feedback to the school on how the changes are going? Let faculty and staff know whether the new procedures are making a difference? Maybe some case studies or data on how many kids you've seen? How many faculty referrals have been handled? Some schools have e-mail and that may be a vehicle for providing feedback.

Julia Lear: One school-based health center I visited sent out a teacher thank-you that included a report and a small gift. I don't remember whether they got donated movie tickets or some other gift, but they had something they could tie a ribbon around to accompany their report. It is so uncommon in the school setting for teachers to receive tangible appreciation that the small gesture meant a lot.

Unknown: One of the concepts we've used is to think about faculty and staff as one set of our mental health clients. Just as with our clients, we try to be aware of what is going on in their lives -- the cycle of report cards, planning, the end of the year. Then we are in better shape to anticipate their needs and respond.

Colon: In our school, we surveyed both teachers and parents regarding topics they would like to hear about from us. When the Board of Education sets aside training days for teachers, we are able to participate and provide training on topics the teachers want.

Issue 2. Blending mental health disciplines

Martha Arden, LIJ-Northshore: We have a different kind of challenge. We are adding a psychologist to our mental health program that has historically been staffed by social workers. It may be quite difficult for those two disciplines to agree on how to describe mental health issues, how to approach therapy, and how to address a host of other professional issues. Developing a common language and common program will be a challenge. How do we create a workable, cross-disciplinary practice?

TJ Cosgrove, Seattle-King County, WA: Let me put an upside to that. In Seattle, we're adding psychiatry to our clinical social work services. Everyone is thrilled and sees this as a win-win. Of course, the psychiatry consultation won't be there on a daily basis. However, the psychiatrist does build a bridge between mental health and medicine for us.

Case-conferencing: A tool for bridging disciplines

Guernsey: What I'm also hearing is that how well cross-disciplinary mental health practice works may be a function of how you "team" it. Adding a new piece to an existing team may throw the old arrangements out-of-balance. How can a new balance be created?

While we're talking about interdisciplinary practice, it seems that you're saying that your case conferencing is done on a catch-as-catch-can basis. Why is this? What are the barriers to routine, scheduled case conferences?

Unknown: We're a small staff; we find informal works for us.

Unknown: What I've found is that even in well functioning teams, people may talk less than they think.

TuLynn Smylie, Children's Hospital of Los Angeles: We found that over time, case-conferencing became more systematized. As we got busier, we had to schedule talk time to make sure it happened. The longer you are open, the busier you get, the more you "move into your office" and the informal case conferencing is reduced.

Trainees are the other program element that pushes scheduling case conferences. At the LA Free Clinic where I spent time initially, there were trainees from all the disciplines. But part of everyone's weekly business was a one-hour lunchtime case conference. I was sold then and I can't imagine wanting to practice without that component. It just seems to strengthen everyone.

Guernsey: I'd have to second that. I find teams do less informal consultation than they think. I also think we believe we're catching the important stuff and that may not be the case.

Smylie: But to make it happen, there has to be leadership from the top. What struck me was that the busiest people who were also teaching at Children's Hospital thought this was important enough to come do.

Another benefit of the case conference was that for our highest risk patients who frequently were also the highest users more than one person was familiar with their case. When their regular provider was not at the health center, another person was somewhat familiar with the case. It really helped continuity.

A related question I'd like views on is the participation of non-clinical staff at case conferences. Sometimes we include the front-desk person because that person has knowledge and a caring relationship with certain students. But we want to make sure we have a clear view of what that person's role is.

Unknown: We have a health educator but she does more population-focused activity and little individual counseling. She would not be included in a case review.

Unknown: We had help from health educators who are working individually, especially as part of HIV prevention work. They interview and meet with students. We would see them as appropriate participants with certain cases.

Unknown: My background is in youth development. I really think that the front-desk person is a youth worker. I think she is a valuable team member and I encourage the clinicians to integrate her more into their case discussions.

Guernsey: These issues sound like our biggest, most immediate challenge. You are all expanding. Your program will become different. How do you get your staff to behave differently? How do you get your referring sources to behave differently? How do you integrate a broader teaming that includes components that you did not have before.

Issue 3. Building relationships between the medical staff and mental health staff

Guernsey: How do mental health professionals react to the idea of a physician or nurse practitioner having some part in the mental health treatment plan?

Unknown: In our center, the nurse practitioner is so busy with medical stuff that I can't imagine her handling much of the mental health piece. But she's done a nice job of transitioning kids with whom she's developed a relationship around their physical issues. Though some might say that successfully transitioning a young person from a physical problem to an understanding that they might benefit from counseling is a therapeutic function.

Unknown: One thing we noticed when we went from one social worker to two and thereby made someone available for crisis intervention, we saw the nurse practitioner's medical visits increase. In think as we made resources more available to the medical side, they could hand-off those problems to us and, as a result, strengthening the mental health program strengthened medical services as well.

Cosgrove: The nurse practitioners in Washington State just got prescriptive authority. We are very curious about the communications among the medical providers who may be involved with patients around psychotropic drugs and the mental health professionals who may or may not know this is going on.

Guernsey: That brings us back to the discussion we had about how we handle this kind of issue on a team basis and how we assure that the mental health services are coordinated and support each other.

Unknown: A couple of ways to handle this come to mind. One approach would be to print out a list of kids who have been to the center 15 times. Maybe that number of visits is necessary but maybe an interdisciplinary plan might get the kid's needs addressed more effectively.

Issue 4. Defining the relationship between the SBHC mental health providers and the school's academic program

Kathleen Wise, Dorchester County: I would like to raise another issue: the relationship of mental health staff to academics. This business of interfacing with the academic environment and supporting students who are failing academically is not easy. Last year our school-based health centers had the experience of securing a small grant to develop a parent/child homework club. But the teachers and guidance counselors have not wanted to be involved in this project. So how far can we go in addressing academic issues that are key.

Let me add that the student who won the school's "most improved student" award went from straight E's to A's and B's as a result of his involvement with his mother in the homework plan.

Unknown: Even if you are not directly involved in providing academic support, what is our role in terms of advocating for students around the special education program? And how do we define our advocacy role?

Scott Bloom, Children's Aid Society: That's a big issue for us. We can become involved in Board of Education special education processes - filling out forms, participating in IEP conferences. And we get very frustrated when the school staff who are supposed to show up don't show. We're not used to that in the health care environment I come from. But we understand that re-organizing school services is not what we are supposed to do.

One of the Children's Aid Society mental health professionals was speaking with an assistant principal and it turned out he thought our job was to be the link to the Special Education evaluation. It was a total misunderstanding.

Unknown: The model that is useful for me is to think of myself as a peak performance coach. My job is not to be the person pointing their finger and labeling people as unable to perform. And the coaching mindset is consistent with that of school personnel.

Issue 5. Negotiating the relationship between health-center based mental health professionals and school mental health professionals

Unknown: In our school, another challenge is the relationship with the school psychologists who primarily test students for placement in Special Education programs. Sometimes, under pressure from the school system, the school psychologist may say that a student doesn't need special education when our clinicians can demonstrate the student does.

Wise: As a former school board member, one of the things I am always aware of is that education has to pay for kids that need special education services. It is no surprise that education is reluctant to put a lot of kids into special education.

We really need to recognize the school system's needs and sometimes that may mean that we need to take longer to solve a problem if that's what it takes to allow the school system to solve problems. For example, the tutoring problem that Mona mentioned. We could probably find a tutor more quickly than the school. But maybe we have to give them an opportunity to solve the problem or work with us to solve the problem.

Unknown: We also have to pick our battles. I've certainly had to balance kids' needs against reality. One response is trying to coach parents to advocate for their children. When we get a parent involved and the parent responds to coaching, it is one of the best outcomes possible.

Issue 6. Maintaining patient confidentiality

Marco Martorano, LIJ-North Shore: There probably won't be time to discuss this issue now but a major issue for us is patient confidentiality and helping school staff understand the limits of what we say. I'm not sure if HIPAA is going to make this job easier or harder.

Unknown: Our confidentiality problem is a little bit different. We have case managers who serve both the medical and mental health sides of the shop. They do a lot of triage. The mental health professionals sometimes have tensions with this position because they say the kid needs counseling or some kind of therapeutic relationship with a provider. The case managers say: this kid isn't ready for that. He just needs someone to talk with about a breakup with a girlfriend or boyfriend. Could we develop some guidance around these positions that straddle mental health and case management?

Guernsey: So another question is: how do the counseling services provided by non-mental health trained providers figure into the overall mix? What do we need to do to make sure we know who is doing what pieces and that their practice is clinically sound.

Issue 7. Mental health billing and state financing policies

Guernsey: This is a big issue to leave to the end. It's not easy. At the state level we have to get the Departments of Education and Health as well as Medicaid to decide how they are going to funnel some resources to support mental health services for kids.

With all your different approaches to providing mental health care, it will be up to you to document what you are doing and whether current reimbursement or other state funding is providing adequate support. The increasing support for mental health parity will certainly help.

Concluding thoughts

Lear: What I have loved about this session is that we have been talking about how you can provide superb mental health services in a school-based clinic. That is the goal of this program. What you are doing and what you are concerned about is precisely what the whole school-based health center field needs to know. So keep doing what you are doing, please document, describe and share what you are learning. It will make a tremendous difference to the kids, their families, your colleagues, and the schools.

Guernsey: I agree with that completely. School-based health care is a simple idea that is extremely complex operationally. But from my experience, 100 percent worth the effort.

Evaluation Issues

With four of the eight mental health projects using the Clinical Fusion software package to collect data in their centers and the rest using other electronic management information systems, the basic information that will be required for management and research will be electronically stored and retrievable. The group focused its conversation on their collective need to define the core components of their services and to identify questions about their projects' performance that would be helpful to study (see below).

Collecting Data and Evaluating Mental Health Services in School-Based Health Centers: Questions to Answer

Definitions needed

  • Who is a mental health user?
  • Who is a mental health provider?
  • What is a mental health visit?
  • Within a mental health visit, how do we categorize and count mental health services?

Outcomes

  • Are we improving mental health?
  • What do our students/clients think about our care?
  • Have we improved students' access to mental health care, both on site and by referral?

Processes

  • How many students are being screened? How many are referred for services?
  • What treatment modalities are being used?
  • To what extent are school staff involved as measured by teacher consultations and teacher referrals/feedback?
  • To what extent is student academic risk assessed? (This might be measured by grade point average noted in medical record or by documented advocacy on behalf of student academic needs, eg seeking academic support or after-school program for student.)
 

The evaluation discussion reflected two concerns: (1) that we need to identify evaluation questions that will be asked by all sites, and (2) that we cannot identify evaluation questions until we resolve definitional issues fundamental to collecting common data elements. The conversation did not resolve the tension between these two points. The Program office will offer some suggestions during the 02 - 03 school year.

Linda Juszczak, facilitator: Here are some questions to think about. What kind of information or data do you want to gather? How do you need to collect it? What do you need to show? And to whom? Legislators? School officials? Parents? And how can we standardize our data or create commonalities among our data bases?

Marco Martorano: One of the questions we've discussed is how we define mental health users? Is this a person who uses a service provided by a mental health provider? Is it a person for whom a V-code is assigned. If a person is seen by a nurse practitioner who uses a V-code for the visit, does that person become a mental health user? Or, for purposes of data collection do you include only those visits provided by certified mental health providers?

Mona Carey: That's a great question because our nurse practitioners say that they do a lot of what they consider mental health counseling. But they can't bill; they can't make a mental health diagnosis. So, there's a large number of visits that would be considered mental health but we don't track those codes.

Martorano: We bill everyone - "counseling, not otherwise specified". And so the issue is you never actually know what the student's being seen for. The issues around labeling or giving a formal diagnosis are big issues.

Juszczak: May I suggest we take advantage of the opportunity to determine what data will be tracked and used to evaluate the effectiveness of the program. Some of the questions the dental health group are considering may be relevant. The first one that they are tackling is are we improving dental health? The second question tries to get at the unique aspects of providing dental services in a comprehensive setting: What is different about being part of an integrated team? What is different about school-based dental care for providing care at the community health center down the street or at the dental school's outpatient clinic?

Wise: It is difficult to talk about what we will evaluate before we are clear on what we value and what are the unanswered questions in relation to those values. This may be harder to do in mental health than in dental health.

One of the things I'm hearing is a question about use of V-codes, which identify mental health-related services as opposed to diagnoses, which define a mental health disorder. If we try to use billing data for evaluation, in Maryland we will only capture diagnosed mental illness because we can only bill for kids with a DSM-IV diagnosis. However, our program is also clear that we will not diagnose just to get paid. So we see a lot of kids for mental health-related services that are not diagnosed. What is the best way of describing that part of what we do and measuring its effect?

Juszczak: Good point. So create your own way to capture that information. Come up with a list of categories - don't call them diagnoses - that you want to count.

Nelle Gregory: I like breaking categories into prevention, early intervention, and treatment. For us, we look at a public health model in which we emphasize prevention and early intervention.

Juszczak: Back up - why are you counting people? The question on the table is evaluation, not program reporting. What is the question you are trying to answer? Why are you counting them? To what end?

Scott Bloom: Picking up on Kathleen's point of our need to explore basic issues before grappling with which ones apply to an evaluation structure, we probably don't even agree on what mental health services are. I think that is one of the questions we are struggling with. What is an intervention as opposed to treatment, and how do you define the difference? Where does risk assessment fit in.

Juszczak: You're right! Evaluation may be tougher in mental health than in dental. At least a cavity is a cavity.

Lili Silva: In terms of the relationship of mental health services to our centers' medical component, we need to look at the number of kids we screen and how many are referred.

Juszczak: Is that something you want to look at? Your relationship to the other providers? All your programs are providing mental health care in the context of a multi-faceted, school-based program.

Bloom: Our goal is to screen the entire student population so that we can do early intervention. Until then, we just try to take advantage of all our encounters to assess the students for mental health problems.

Juszczak: So to pull this conversation back toward evaluation & evaluation design, a basic question could be: What are we doing? To make the design more complex, the question could be: What are we doing compared to what other people are doing? This latter question should be achievable. Are there other places you can get the data? Maybe in your sponsoring institution also manages an outpatient clinic or a community-based program? Could you get information on how often teenagers or younger children access mental health services at the other facilities?

Arden: If we took your original question and plugged in the word "mental" instead of dental, then the question is: are we improving mental health? That is what the funders want to know. They don't really hear the prevention or early intervention distinctions. Their point is: If we are funding you to provide care in schools, are you improving mental health? That is a very difficult question to answer. You may not know if you are improving mental health for 10 to 20 years. The studies that answer that question are longitudinal studies.

Juszczak: Good point. But that brings us back to the question: what data do you want to collect now?

Colon: Marti's right. Outcomes are long term. We're looking at access. Without our services, would our kids get mental health services? What students do access those services and who doesn't? And what do we know about the kids who don't come in. For elementary school programs we might be more ambitious because you can track those kids into high school.

Unknown: You need an operational definition of access. It's more than just counting the students you see once. We are heavily engaged with our kids and we can do more than identify the problems. We are involved in their treatment as well. We can use chart reviews to document this 2nd part of our impact on access.

Unknown: Another aspect of expanding and enhancing services is raising the knowledge and skill level of the staff. We want to improve referrals -- both to school-based clinic providers and to community providers. We're also concerned about consumer satisfaction. And we should include the families in measuring that satisfaction.

Unknown: We've been involved in a study at our site, comparing routine therapies for depression. The problem with effectiveness research is that this kind of research requires Institutional Review Board approvals (IRB) and, basically, elaborate staff-intensive procedures. We need to keep in mind the resources required to pursue the evaluations we propose.

Juszczak: It might be helpful to remember there are things that you may want to tackle at your individual sites, but what we are talking about right now is what is the aggregate data we want to collect across all sites. And in order to determine what that is, you have to know WHY you want to collect them.

Unknown: Even describing what we do is tough. We need to ask what treatment modalities are being used? How much group work are we doing? How much are we working with individuals? How much are we working with families? I think it would be a tremendous contribution to the field to have this many programs provide information about how you operate and what you do.

Unknown: I'm particularly interested in educational outcomes and in the relationship between the mental health staff and school personnel. What kinds of contacts are made with teachers or other school personnel. Also to what degree does the school-based health center become involved in supporting a healthy school environment?

Next: Dental Health Challenges