The Supportive Housing Program (SHP) of the U.S. Department of Housing and Urban Development (HUD) provides funds for service providers who seek to stabilize and house homeless persons. Local and state government are responsible for organizing SHP applications to HUD and then for distributing the funds to successful applicants and monitoring their operations.
In 1995, the City of Boston helped 20 service programs to secure HUD SHP funding. The funded programs provided either housing placement and stabilization services, employment, training and job placement services, or outreach services. Each program was intended to improve the continuum of care for homeless persons in Boston. They focused on either homeless individuals or homeless families with children; some were designed ot meet the needs of specific subgroups, including persons suffering from mental illness, substance abuse, AIDS, and victims of domestic violence.
The UMass/Boston - City of Boston Supportive Housing Evaluation Project (SHEP) collected assessment and outcome data from the 17 Supportive Housing programs that enrolled clients between October 1995 and December 1996. Most programs submitted data on all these new clients to the SHEP system. This report uses these data to improve understanding of homeless persons’ needs and to demonstrate the value of an integrated information management system. Its primary goal is to use SHEP’s findings and experiences to improve service delivery to homeless persons.
Project Overview
The City of Boston has sought to improve the service system for homeless persons as well as the particular services provided by individual programs. In order to achieve this goal, comparable information about clients and service outcomes is needed across different programs. The Supportive Housing Evaluation Project was designed to provide this information for the programs funded by HUD’s 1995 SHP grant.
The SHEP effort had several specific goals: to give feedback on service system operations; to improve the client information used by service staff; to facilitate preparation of required reports; and to provide an option for other service systems seeking to improve their information processing.
The Data Collection System
The Supportive Housing Evaluation Project was designed to collect client data at four different points in the service delivery process: initial referral, intake assessment, service delivery, and follow-up. All SHP programs provided initial assessment data to the SHEP system and all but one submitted follow-up data on SHEP forms. Some programs also used the SHEP referral log to record decisions about potential clients or the SHEP service log to record the services provided to particular clients. This report is based almost entirely on data collected at initial assessment and the first follow-up. These data permit description of the clients entering SHP programs and analyses of their outcomes after this point.
Both the initial and follow-up assessment forms were designed to collect information required in HUD’s Annual Progress Reports (APRs). Several additional fields were included to meet perceived program information needs. The primary SHEP assessment form, the Client Assessment Summary and Evaluation Form, is completed by staff after an in-person interview with the client at intake. Basic information is collected on sociodemographic characteristics, family status, employment and benefits status, housing history, and service needs. Staff also record on the form whether the program accepted the individual and whether the individual decided to enter the program.
The SHEP Summary & Evaluation form was designed as a common tool for capturing key client information that might be collected with different types of intake interviews. In addition, the SHEP system offered a longer “Complete Client Assessment Interview.” Four Boston programs used this longer form to develop a more thorough understanding of client needs at intake. The Complete Assessment tool included multiple questions for clients about their social background, residential history and needs, employment history and needs, other service needs, income sources, physical health, substance use, mental health, social contacts, and legal issues. Questions for this comprehensive assessment were selected from instruments of proven value in studies of homeless persons (such as the CES-D depression index, health measures in the Medical Outcomes Study, the NIMH ACCESS evaluation instrument, and the Dartmouth substance abuse index). The Complete Assessment obtained all data required by the Summary and Evaluation forms. One job placement program used a modified form of the Complete Assessment that collected more detailed information on work history.
Four programs used a unique assessment interview, but they still recorded key client information on SHEP Summary and Evaluation forms.
The Staff Follow-up Assessment form is completed by staff on the basis of their knowledge of client status at a designated follow-up interval. In addition to program status, the form requests information on job and housing outcomes, benefits status, and substance abuse. In Boston’s 1995-1996 SHEP project, all but one program used the follow-up forms for at least some clients. Most programs completed the followup form at the end of the designated program year, although a few programs collected follow-up data on a more regular basis.
The SHEP system allows collection of comparable information across SHP programs in a form that can be used to satisfy HUD reporting requirements. It also can improve client information available to program service staff. However, the system requires full-time staff to collect data and monitor system operations. Because the resources available to the Boston project were insufficient for this purpose, client data collection efforts were focused in most programs on the initial assessment and follow-up forms. Complete Assessment forms, service logs and referral logs were collected from only a fraction of the SHP programs.
The database collected by the SHEP project includes data on the characteristics and needs of more than 1800 SHP clients and outcome data for more than half of them. This report reviews the SHEP data collection system; it describes the backgrounds and needs of the SHP clients; and it analyzes variation in outcomes across programs.
Figure 1
Report Design
The report first reviews the implementation of the SHEP information management system and evaluates the quality of the information obtained. The next section describes the flow of clients into SHP programs. Two sections then report the basic descriptive findings for the SHP programs as a whole. First, client characteristics and needs at the time of initial assessment are described. Then, service outcomes are identified as of the follow-up date. The next section presents outcome indicators in relation to program status at follow-up, so that the outcomes achieved by program graduates can be distinguished from those of program dropouts and those still in the programs.
Much of the report presents assessment and outcome data for the specific service programs, in separate sections corresponding to program type: family housing, adult individual housing and stabilization, employment and training, and outreach. Multiple measures are included to indicate housing outcomes, employment and benefits outcomes, and substance abuse at followup. Tables are also included to identify the relation between several key client characteristics--age, substance, and gender--and outcomes. This allows a preliminary consideration of the types of clients that derived the most benefit from each type of program.
Several special sections present more detailed data on clients and the services they received. Comprehensive assessment data are presented for four family programs and one employment and training program that used the SHEP Complete Assessment Interview. Longitudinal service usage data are reported for clients in three programs that used SHEP Service Logs. One employment program provided a second wave of followup data on SHEP Staff Follow-up Assessment forms. These data allow a preliminary analysis of change in outcomes over time. Each of these special sections indicate ways in which implementation of the full SHEP system can enhance understanding of client needs and program impact.
Methodology
Collecting comparable data from many hundreds of clients seen by diverse staff at multiple different agencies would be a challenge even with unlimited time and resources. To carry this out while at the same time designing an entire data collection system and meeting bureaucratic reporting deadlines multiplied the challenges facing both SHEP and program staff. But the combined efforts of committed program staff and talented SHEP assistants made it possible to develop within one year a working system and a large database on SHP clients. The goal of this section is to review basic indicators of the quality of information collected by this system.
Follow-up Coverage by Agency
Programs reported data on 1,833 SHP clients—1623 during 1995 and 1996 and another 210 between January and May of 1997 (all of the 1997 cases were from Project Impact). Follow-up forms were completed on over half of these clients (981, or 61% of the clients first assessed in 1995-1996).
Figure 2 displays the percentage of clients from whom follow-up data were collected at some time during the first year. It indicates that all but three programs succeeded in collecting follow-up data from at least 80% of their SHP clients; in fact, 13 programs achieved just about 100% follow-up coverage. However, because the three programs that lacked follow-up data for more than 20% of their cases were large, they had a substantial impact on the overall project follow-up rate.
Figure 2
The follow-up rate was substantially less than 100% due to particular circumstances at three programs, rather than because of a fundamental problem in the data collection system itself. GBLS was an outreach program that did not include in its service design any plan for systematic follow-up data collection. (Some follow-up housing and benefits information was recorded, but not on SHEP followup forms.) Impact Employment Services agreed to collect follow-up data every 4 months, but many of their clients had not reached their four-month follow-up period by the end of the first year. In addition, an oversight in assigning identification numbers resulted in the loss of the opportunity to collect follow-up data on clients assessed in the first several months of program operation. GBHI used its own data collection system and only reported follow-up data to SHEP for a small sample of clients used to test the SHEP system.
The consequences of missing follow-up data can partially be assessed by comparing the characteristics at entry of those clients at Impact and GBHI who did and did not receive assessments later in the year. As a group, the clients for whom follow-up forms were submitted, at both GBHI and Impact, were similar to those for whom follow-up forms were not submitted in terms of gender, race and ethnicity, and income. However, at GBHI, clients who were followed up were less likely to be substance abusers than those for whom follow-up forms were not completed (t-test, p<.001). Also, GBHI clients who were followed up were younger (p<.05). Those at Impact who were followed up were somewhat more likely to speak English and to be women, compared to clients who were not followed up. These discrepancies at intake suggest the likelihood of some bias in the sample of clients followed up at both of these agencies.
This experience indicates that collection of follow-up data was feasible for all the SHP programs but created some problems for several large programs. It may be necessary for these large programs to provide more staff support for data collection efforts. Basic service staffing needs must be also be considered, since adequate follow-up data cannot be collected in this staff-based system unless there are some service contacts between staff and clients after the initial assessment. Procedures for collection of follow-up data from outreach programs must be designed to take into account their unique service goals and their different staff procedures. In addition, it is critical to collect follow-up data for all clients, since in the absence of a total population reporting requirement staff may focus follow-up data collection on clients who differ in some respects from their total client pool. The system adopted at Impact should make this feasible for large programs: periodic collection of follow-up data, at 4-month intervals, reduced staff burden and gradually increased the proportion of clients followed up. An alternative would be to use a stringent random sampling procedure to eliminate bias, but this would require special research staff to design and monitor client selection for follow-up.
Question Completion by Agency
Collection of data forms representing many clients is itself an accomplishment, but these forms would be of little value if they did not contain the required information. The next two figures indicate the extent to which key data were recorded on the SHEP client data forms.
On average, data were missing for only 11% of the key questions on the SHEP Summary & Evaluation form. As indicated in Figure 3, several programs (JRI, New Victory, St. Mary’s, Downtown Crossing) had almost no missing data fields. CEEH and GBHI both had relatively high rates of missing data (20%), but both used a unique initial assessment instrument that did not correspond in all respects to the SHEP assessment instrument. SHEP received from both programs a file containing assessment data that were then imported into the SHEP database, rather than SHEP Summary/Evaluation forms themselves. Crittenton-Hastings, ABCD, and GBLS also had at least 10% missing values, but each program used its own assessment interview as the basis for completion of the SHEP form.
Figure 3
Data were missing more often on the SHEP Follow-up forms. On average, 22% of key questions were left unanswered and most programs were missing at least 10% of the answers (figure 4). Three programs were missing more than one-quarter of the values for the key variables (in addition to GBLS, which provided no follow-up forms but did send some benefits information on some clients). CEEH was engaged in outreach and had little contact with many clients after the initial assessment. GBHI only used the SHEP follow-up system on an experimental basis.
Figure 4
Collection of adequate follow-up data was impaired when staff contact was infrequent. Follow-up data show a positive correlation between the extent of missing follow-up data and the time since last staff contact with the client. In other words, the more recent their contact with program staff, the more adequate the client follow-up data recorded by staff. This suggests that collection of follow-up data more often than once a year would improve the quality of that data. More frequent follow-up data collection was planned for the SHEP system, but it proved to create a burden for program staff. The tradeoff between staff burden and data quality must be considered carefully when implementing such a system.
The absence of follow-up forms for some clients and the lack of response to some questions on the Summary/Evaluation and Staff Follow-up forms create variation in the number of cases with valid data for particular questions. The actual number of cases responding to each question is indicated in the detailed tables for each program. It is important to take into account these base Ns when determining whether the responses given are likely to represent all program participants.
Outcome Measures
The SHEP follow-up form collected information from program staff about client program status, housing experience, employment and benefits, and substance abuse. Most of the questions used to record this information paralleled the structure of HUD’s Annual Progress Report and so were not ideal for research purposes. In order to maximize the utility of this information for outcome analyses, outcome measures were constructed as composites of responses to several questions on the follow-up form (see table 1).
Table 1
Outcome Measures
Composite Measure/Component Indicators
Ever Homeless (indicated by any of the following):
The client is currently homeless.
Client stayed any weeks in the past 12 in shelters, outdoors, public buildings.
Client left the program and went to shelters or streets.
Client is homeless at year’s end.
Housed at Any Time (indicated by any of the following):
Client stayed in own apartment or room some weeks in past 12.
Client left the program and went to alternate housing, subsidized housing, unsubsidized rental housing, section 8 housing, public housing, homeownership.
Client in permanent housing at year’s end.
Housed Consistently and Not Homeless (not ever homeless & any of the following):
Client stayed in own apartment or room every week in past 12 (of weeks w/ info).
Client left the program and went to alternate housing, subsidized housing, unsubsidized rental housing, section 8 housing, public housing, homeownership.
Client in permanent housing at year’s end.
Employed at any Time (indicated by any of the following):
Client employed at time left program.
Client employed at follow-up.
Client worked some hours in past month.
Client had some earned income.
Benefits: Number of benefit sources, or of only government benefits.
Substance Abuse (indicated by any of the following):
Substance abuse disability at baseline (for baseline + follow-up compsite).
Client has resided at a detox facility.
Client referred to the SHP from substance abuse treatment (baseline + follow-up).
Client abused alcohol or drugs, indicated on follow-up.
Client left the program and went to a detox facility.
Client needs detox services.
Client has received detox services.
When these outcome indicators are used to indicate the status of current program participants, their interpretation varies with the type of program. A program that provides transitional housing to participants while they are in the program may still classify these participants as homeless, while participants classified as homeless in an employment program may still be living in emergency shelters. Program staff should be consulted before making these interpretations.
Comparison to Annual Progress Report (APR) Data
HUD requires submission of Annual Progress Reports by each program. The 1996 reports submitted by programs (with the assistance of SHEP) present data that are comparable in most respects to the data contained in this report. However, there are some differences, for several reasons.
In total, the numbers reported in 1996 program APRs indicated that 1659 clients enrolled in the SHP programs by December 1, 1996. The corresponding figure from the SHEP database is 1,567, or 94% of the clients listed on APRs as having entered a SHP program (SHEP clients are omitted from this comparison if the assessment form indicates that they were rejected or refused to enter). Almost all of the additional cases in program APRs came from ABCD, Bay Cove, and Impact Employment Services. These programs were able to identify cases for which assessment forms had not been submitted.
Discrepancies with APR figures have several other sources. The goal for this report was to provide comprehensive information on all clients assessed by programs by the end of December 1996. This period did not coincide with the APR year, which ended on December 1 for most programs and in October for some. In addition, follow-up data may differ in some cases because follow-up assessments for a fraction of clients were completed as early as September. Outcome information for these clients may have changed before year’s end. Also, prospective clients whose entry was postponed or whose program status was otherwise unresolved are included in this report, although follow-up data may indicate that they did not begin. Finally, the APR requires that some outcome data be reported for only a certain group of clients (such as program dropouts who were not expected to return). These same restrictions are not applied in this analysis.
A different approach to indicating client outcomes also creates a divergence with APR figures. In order to minimize data loss in the outcome analysis, most outcome indicators in this report are a composite of responses to multiple questions on the follow-up form. For example, staff are asked on the follow-up form if a client was employed when they left the SHP program as well as whether the client was employed at year's end. Rather than treating these as two separate measures, each of which would have substantial missing information, the answers are combined to indicate whether the client was known to have been employed at any time since program entry. In the same way, follow-up questions about housing status at different times are combined to indicate whether clients were known to be homeless at any time since program entry, or whether they were known to be housed and not known to be homeless at any point. The result of this measurement strategy is a higher proportion of clients with some outcome indicators, but at the cost of less specificity in terms of the time period reflected by the indicators.
Two large programs completed follow-ups for, respectively, 15% and 35% of their clients. For analyses of outcome data for the entire SHP client population, the data from these two programs is weighted so that the programs’ clients make up the proper portion of the total population. In program-specific analyses, no weighting is done. It is important to note that the weighting procedure probably identifies the upper bound of possible program impacts, since the programs whose numbers are increased by the weighting (GBHI and Impact) showed relatively high rates of housing and job placement for the cases they did follow up. For this reason, aggregate outcomes are also presented for the unweighted file. The actual outcomes of the SHP as a whole are probably between the estimates based on the weighted and unweighted statistics.
The Complete Assessments
Several SHP programs used the SHEP Complete Client Assessment Interview to gain a more comprehensive, in-depth understanding of their clients’ backgrounds and needs. Most of these programs were family housing programs; all were programs that worked with clients for a long period of time and had to make multiple support and placement decisions. Client answers to more than 100 questions and sub-questions in the Complete form provided these programs with a comprehensive record of client needs. Five programs submitted a total of 54 Complete forms in time for inclusion in this report. Analysis of these data indicates the value of comprehensive assessment and provides an in-depth portrait of clients in the family housing programs and in two transitional work programs.
The pattern of client coverage for those programs submitting Complete Assessment data suggests some caution in generalizing to the total numbers of clients in two of these programs. As indicated in table 2, coverage was only partial for two family housing programs; coverage was complete for the two employment and training programs using the Complete Assessment and for one of the three family housing programs.
Table 2
SHP Program N S-E Forms N Complete % C/S-E
Dennis McLaughlin 27 13 48%
Elizabeth Stone 39 13 31%
WISH 7 7 100%
Downtown Crossing 16 16 100%
Project Place 5 5 100%
Comparison of Summary-Evaluation data for the 26 clients with Complete assessments at Dennis McLaughlin and Elizabeth Stone with the total of 66 clients at these agencies indicates that the Complete sample is unrepresentative in some respects. More of the clients for whom Complete forms were available were still in the program at follow-up (75% compared to 14%) and fewer had been asked to leave (5% compared to 57%). On average, they had higher incomes at the initial assessment ($577 compared to $171) and had gained more income during the project ($107 compared to -$485). For these two agencies, therefore, the Complete data represent a subset of clients for whom outcomes were more favorable.
Summary
The SHEP data collection system provided an efficient means for collection of basic assessment and follow-up data. Initial SHEP assessment data were based on several different interview procedures, but the SHEP Summary and Evaluation form provided a convenient tool for standardizing this information. More complete comprehensive assessment data also were collected on some clients, although it was not possible to regularize collection of this type of information within the first year. Follow-up data were collected successfully by most programs.
Data coverage
is easier to evaluate than the accuracy of the data collected. Any effort to
collect client information from different staff in different programs,
particularly over an extended period of time, must be concerned with the
possibility of inconsistent interpretations of questions and response choices.
In the SHEP project, guidelines were distributed about the use of SHEP forms
and some training was conducted in order to improve consistency. As apparent
throughout the report, staff were careful in conducting assessments and
gathering follow-up data. However, the quality of these data could be expected
to improve in subsequent years if staff received more regular training in the
use of the SHEP forms.
Client Flow
The number of clients served by SHP programs ranged from 644 at Project Impact (210 of these were assessed in 1997) to 5-7 clients at JRI, Project Place, St. Mary’s and WISH House (figure 5). Five programs assessed at least 200 clients during the year. Few applicants for any program were judged inappropriate for the program or decided not to enter (overall, 1.1%), but no information was available on the acceptance decision for about 13% (233 cases). Most of these clients (134) were seen by the one large housing program (GBHI) that used its own system for data collection.
Figure 5
Cumulative Number of SHP Clients
Clients were assessed and then entered the SHP program at a steady rate throughout most of the first year of funding (figure 6). Only the first few months operated at a slightly reduced pace. The apparent decline in December 1996 reflects the end of the data collection period for all programs except Project Impact.
Figure 6
Assessments by Quarter and Program Type
Although the overall rate of client assessment was relatively stable across the program year (which ended on December 1, 1996), the rate for different types of program varied by quarter (figure 7). The adult housing and stabilization programs began accepting large numbers of clients first, reflecting the early start date of October 1995 for the GBHI program. Job and training programs (primarily Impact) began to enroll large numbers of clients in the second quarter, with the outreach programs reaching their peak enrollments in the third quarter.
Figure 7
The flow of client assessments by specific programs is reported in the next table. Eleven programs met or exceeded their annual goals.
Table 3
Number of Clients Assessed
|
|
<Dec |
Dec-Feb. |
Mar.-May |
June-Aug |
Sep.-Nov. |
Total |
Annual Goal |
|
Family Housing |
|
|
|
|
|
|
|
|
Crittenton Hastings |
|
|
|
|
|
|
|
|
Dennis McLaughlin |
|
|
|
|
|
|
|
|
WISH House |
|
|
|
|
|
|
|
|
Elizabeth Stone |
|
|
|
|
|
|
|
|
St. Mary’s |
|
|
|
|
|
|
|
|
Adult Housing |
|
|
|
|
|
|
|
|
ABCD
|
|
|
|
|
|
|
|
|
Bay Cove
|
|
|
|
|
|
|
|
|
GBHI
|
|
|
|
|
|
|
|
|
JRI
|
|
|
|
|
|
|
|
|
New Victory |
|
|
|
|
|
|
|
|
Pine St. Inn |
|
|
|
|
|
|
|
|
Employment & Training |
|
|
|
|
|
|
|
|
Center House |
|
|
|
|
|
|
|
|
Project IMPACT |
|
|
|
|
|
|
|
|
Project Place-Pepsi |
|
|
|
|
|
|
|
|
Downtown Crossing |
|
|
|
|
|
|
|
|
Outreach |
|
|
|
|
|
|
|
|
GBLS |
|
|
|
|
|
|
|
|
Committee to End Elder |
|
|
|
|
|
|
|
|
Totals |
|
|
|
|
|
|
|
*Does not include clients who were rejected or did not enroll.
** Includes clients to be served by other programs.
***F=Families; I=Individuals. (Distinguished only for family housing programs.)
****Includes 19 clients assessed after 12/1/96, who are not included in the next report section on client characteristics. Impact counselors assessed 210 additional clients in 1997.
SHP Clients in 1995-1996: Characteristics and Needs at Intake
The SHP programs served a diverse mix of homeless persons in 1995-1996.
Client Characteristics
Gender
More than half of the Supportive Housing clients were male; 44% were female.
Figure 8
Age
The average age of clients was 39, with more than half of the clients in their 20s and 30s (figure 9). Only 22% were 50 or older and just 4% were in their teens.
Figure 9
Race/Ethnicity
About equal proportions of clients were African-American (43%) and white (38%), while an additional 17% were Hispanic (figure 10). Only one percent were from Asian backgrounds and fewer than that were identified as Native Americans. Ten percent of the clients could not speak English.
Figure 10
In general, the diversity of client characteristics attests to the broad range of programs funded and the multiple segments of the homeless population that they serve.
Client Needs
SHEP Summary & Evaluation forms give some indications of client needs. Prior living situation, sources of benefits and levels of income, as well as disability status each help to suggest what service options should be emphasized.
Prior Living Situation
Half the clients had previously lived in shelters (figure 11). Transitional or rental housing, substance abuse treatment facilities, and family or friends’ places each accounted for the prior living situation of another 10%.
Figure 11
Benefits
SSI and AFDC were the two most common sources of income benefits, each received by about one in five clients (figure 12). Employment income was about as common. No other income sources were identified by as many as 10% of clients.
Figure 12
Income
Income was recorded for 1320 clients by the end of the 1996 project year. More than one-third (38%) of these received $250 or less per month in benefits or employment income, and only 10% received as much as $1000 per month (figure 13). The average (mean) monthly income for the total sample by December 1, 1996 was $433.
Figure 13
Disability Status
Four in ten SHP clients were thought at program entry to have no disability (figure 14). Substance abuse was the only frequently mentioned disability among the rest, with almost one-third identified as substance abusers, either alone (25%) or in combination with a mental illness (6%). About one in five were identified as mentally ill (including those also identified as substance abusers). The listed categories were based on HUD reporting requirements and therefore did not include physical illness, but written comments about “other” disabilities identified at least 6% as having a disabling physical illness.
Figure 14
Summary
SHP clients were diverse in terms of their social backgrounds and their service needs. They were almost equally likely to be male as female and black as white, with Hispanics also represented. Although they tended to be young, more than one in five were at least 50. Most previously had been living as homeless and had a very limited income. Substance abuse and mental illness were identified as problems for almost half.
SHP Clients: Outcomes
The first outcome examined is program status—the proportion of clients who had graduated, were still in the program, or had dropped out or been asked to leave. Since programs had accepted clients at different rates over the course of the year, and clients had themselves been in SHP programs for varying lengths of time at year’s end, it is inappropriate with these data to compare the distribution of client status between programs. However, evaluation of all other client outcomes must take into account program status. In many programs, clients are not expected to attain such outcomes as job or housing placement until they have graduated from the program. It also is important to take into account the proportion of clients who did not complete a program before comparing housing placement and other outcomes between programs. For these distributions, outcome data are weighted to compensate for the partial follow-up rate at two large programs (cases at GBLS are not included, since no outcome data were collected with SHEP follow-up forms).
Program Status
About two-thirds of the clients had dropped out (or been asked to leave) or had graduated (figure 15). About one-quarter were still in their SHP program at the time of follow-up. A small number had an unknown status. Six percent had never actually started the SHP program and so are omitted from the outcome analyses in subsequent sections.
Figure 15
Reason for Leaving
The SHP dropouts, about one in every five program entrants, most often left the program on a voluntary basis (48%) (figure 16). The reason for leaving was not known for one in three of the other clients. Substance abuse (15%) and noncompliance with housing requirements (5%) were the other most often mentioned reasons for leaving.
Figure 16
Destination after Leaving
Staff often did not know the destination of one-third of program dropouts (figure 17). About one in five moved into some type of housing, but 12% left for the streets or an emergency shelter. Family or friends was the destination for 9%.
Figure 17
Aggregate Outcomes
Several client outcomes were measured by staff with the SHEP Staff Follow-up Form: residential experience, employment situation, benefits, income, and substance abuse status. Service needs were also indicated. This section describes these outcomes for clients who had left an SHP program, either by graduating or dropping out. These data are weighted to compensate for the missing follow-up data from GBHI and Impact Employment Services, but unweighted statistics are presented in the text [in brackets] when they differ by more than 10 percentage points from the weighted statistics.
Gains in housing, employment, and income were impressive for those who began an SHP program and were no longer in it (figure 18). Almost one in six had been housed at some time since follow-up and for 36%, there was no indication of homelessness since they began the SHP program. Fifty percent had been homeless at some time in the follow-up period. Staff reported 11% as having been institutionalized and 17% as living in housing that was not their own.
Figure 18
The average gain in monthly income for those who had been in an SHP program was $432 [$298 with the unweighted data] (figure 19). Only a small number of those who had left an SHP program (9%) reported any income loss, although about one-third had experienced no income change [45% with the unweighted data].
Figure 19
More than half of those who had left SHP programs had secured a job at some point, although they had not necessarily been employed continuously (figure 20). No employment information was available for 4.6%.
Figure 20
Increases in number of government benefits received were modest for those leaving SHP programs (figure 21). More than half had no change in benefits; just over one-quarter reported an increase in the number of benefits received [39% with the unweighted data] while 16% reported a decrease.
Figure 21
The increased average income for former SHP participants was explained primarily by employment, not by securing more government benefits (figure 22). Those who had secured jobs at some time in the follow-up period had increased their monthly incomes by almost $700, while incomes were unchanged for those who had not secured employment.
Figure 22
The increase in income occurred primarily for those who were receiving fewer government benefits (who gained $608 per month) or had experienced no change in number of government benefits (monthly income gain was $471), not for those who were receiving more types of government benefits (who gained only $141 per month).
Figure 23
Program Status and Outcomes
This section describes outcomes for the entire population of clients served by SHP programs. Outcomes are described in relation to detailed program status. The statistics in this section serve as a point of comparison for the program-specific presentation of client outcomes in a subsequent section. These data are weighted to compensate for the missing follow-up data from GBHI and Impact Employment Services, but whenever the weighted and unweighted outcome statistics differ by more than 10 percentage points, the unweighted statistics are mentioned in the text.
Residential Experience by Program Status
The differences in residential history for groups differing in program status were dramatic (figure 24). Eight in ten program graduates (79%) had been housed at some time during the year and for half of the graduates there was no indication of homelessness at any time in the year. On the other hand, 60% of the program dropouts had been homeless at some point during the year and only 16% appeared to have been housed without experiencing homelessness. About three-quarters of current program participants were still rated as homeless.
Figure 24
Income Change by Program Status
Monthly income was also substantially higher among program graduates than among dropouts or current participants (figure 25). Monthly income was higher by $627 for program graduates compared to their income at initial assessment (a 179% increase) [the unweighted followup increase was $442, a 110% increase]. Incomes also rose for those who dropped out of an SHP program (by $157, a 61% increase) [the unweighted increase for this group was 38%], but remained relatively stable for those still participating. The small number of clients who had been asked to leave an SHP program had experienced a monthly income decline.
Figure 25
Employment Status by Program Status
For program graduates, the likelihood of earning income from a job increased by 2.9 times, to 69%, compared to the initial assessment [the unweighted increase was to 52%]. The likelihood of earning job income had also increased for the other groups defined by program status, but less markedly (except for the small group [26 persons] with an “other” program status). When all indications of employment at follow-up are considered, it seems that almost 80% of the program graduates had secured a job at some point [64% without weighting], compared to no more than 42% of the other groups [37% without weighting].
Figure 26
Benefits Change by Program Status
The percentage of clients who received government benefits declined for SHP graduates (from 46% to 37%) but remained relatively constant for the others (figure 27). Current program participants were most likely to be receiving income from government sources at followup (64%).
Figure 27
Substance Abuse by Program Status
Indications of substance abuse had declined for graduates and continuing SHP participants, from about 37% to 25% of these groups (figure 28). Almost half of the SHP clients who were asked to leave their program were identified as abusing substances, as were about one-third of the program dropouts.
Figure 28
Service Needs by Program Status
Staff rated SHP graduates as having fewer service needs than either current program participants or program dropouts, but they were receiving about as many services as these two groups (figure 29). As a result, program graduates needed, on average, only one service that they were not receiving. Program participants had an average service needs-received deficit of 1.5, while dropouts were lacking, on average, 3.6 needed services.
Figure 29
Summary
Within their first year, SHP programs graduated about one-third of those they had admitted, although about as many had dropped out or been asked to leave. Considered as a whole, many of these persons who had been in an SHP program were housed or employed. More than one-third were housed after they had started the program and there was no indication of their having been homeless; 60% had been housed for some of this time. Half had secured a job at some point and average incomes had increased.
Outcomes were impressive for program graduates. Eight in ten had been housed at some point and a majority did not appear to experience homelessness after leaving the program. Program graduates were much more likely to secure employment than others and their incomes increased substantially, even though they were less likely to rely on government benefits. Staff rated program graduates as having fewer unmet needs at follow-up than was the case for current program participants or dropouts.
In spite of the
success of graduates, program dropout was an important concern: about as many
participants dropped out or were terminated as graduated and few dropouts for
whom information was available left their SHP program for a more acceptable
living situation. A substantial fraction were identified as having a substance
abuse problem.
Family Housing Programs
The five family housing programs each intended to serve somewhat different client populations and to use somewhat different service approaches. However, the service core of each program was similar: transitional living in a house that also offered support services. Each program also assisted clients with housing search and placement after they graduated.
Program Descriptions
The primary service of the Transitional Residence at Crittenton (TRAC) Program at the Crittenton Hastings House is housing provision for young mothers who need support services. Ten new transitional housing units were developed to assist homeless single mothers with demonstrated readiness to move successfully towards economic independence and self-sufficiency. Women with major substance abuse or mental health problems are not encouraged to participate. Particular attention is paid to strengthening the family structure by providing women with the skills necessary to balance work and family and by meeting the family’s emotional and economic needs. Case managers assist in the housing search during what is expected to be a year in the program.
The Dennis McLaughlin House of Charlestown, Inc. provides housing to women referred from approximately thirty agencies in Boston. The program expects clients to remain for 6 - 16 months on average, and up to 24 months total. The program seeks to stabilize lives, to improve the health and general well being of each program participant, to address the short-term and long range needs of participants and to secure necessary services, including: financial, educational, training, job placement and career development, childcare, medical, psychiatric, basic life skills, family, social, cultural, recreational, and housing issues. It is expected that many clients have had substance abuse problems or have suffered domestic violence. The Dennis McLaughlin House assists participants in applying for subsidies and locating, securing and maintaining permanent housing in such a way that they feel empowered and that their families are kept intact or reunited.
The Pathways to Independence Program at St. Mary’s Women’s and Infants’ Center also provides transitional housing, although with an emphasis on support services for women and their families who are at serious risk. The program was designed to run from
6 to 9 months per person with the end goal of permanent housing for each. Participants must be either pregnant or parenting; some are expected to have problems with alcohol and others will have a serious mental illness.
WISH House, Inc. provides housing and services to homeless women disabled with HIV/AIDS and to their children. Applicants are expected to predominantly be African American women, many of whom may be victims of domestic violence. The long term goal for these individuals or families is attainment of safe permanent housing in a supportive environment. In the short-term, the foci are residential stability, health and substance abuse recovery, maximizing support through the course of illness, and realization of individual and family goals. No alcohol or drugs are allowed on the premises and periodic urine tests are conducted. Some clients have been asked to leave due to a failure to comply with rules, and a three-strikes-you’re-out policy is followed.
The primary service offered by the Transitional Housing Program at Elizabeth Stone House is housing provision with an emphasis on fostering client independence and self reliance. The program provides women and their children a 12- to 18- month program to stabilize the family in a safe and supportive living environment while defining and working toward practical goals of achieving autonomy when they leave the program. Clients are drawn from families with children in custody of the Department of Social Services. Services include violence and on-site addictions support groups, self-help advocacy and support groups, and information and education. Particular efforts are made to reach out and educate the Latina community about violence and its effect on mental health, and to offer services in Spanish to women who do not speak English. All clients accepted by the Stone House must ultimately be self motivated, and women currently using drugs or having major emotional or physical issues are not accepted.
Clients
These five programs assessed a total of 100 adult individuals--6% of the total number of SHP clients during the year. All served women, most of whom had children with them.
Characteristics
The McKinney programs at Crittenton-Hastings House and St. Marys both served younger women, with Crittenton-Hastings focused on African-American women and St. Marys serving many Hispanic women as well (as did Elizabeth Stone House). The other three programs served women with an average age in their 30s. Dennis McLaughlin was the only one of these programs where a majority of clients were white. At all three programs with many Hispanic clients, at least 10% of the clients could not speak English.
Table 4
No
Agency Mean Age %Female % AfroAm Hispanic English
(base N) (6) (6) (5) (5) (6) The different prior living situations of their clients reflected the varying service foci of the five family housing programs (table 5). Dennis McLaughlin and WISH clients had most often been in a substance abuse treatment facility, while friends’ and relatives’ homes predominated among the prior living situations of Crittenton-Hastings and St. Mary’s clients. More than one-third of Elizabeth Stone clients had been in situations involving domestic violence.
Table 5
Needs
Clients at Crittenton, WISH, and St. Marys each had average monthly incomes at initial assessment of about $200-$300 (table 6). However, the sources of income differed between the shelters, with St. Mary’s clients relying on TAFDC (or EAEDC), and WISH clients more dependent on SSI. Elizabeth Stone clients, who were most likely to be employed, received the highest average monthly incomes. Dennis McLaughlin clients received benefits from more sources than at the other programs and also reported a relatively high average monthly income.
Table 6
Mean Mo Job Food
Agency Income SSI TAFDC Income Stamps
(base N) (15) (22) (22) (22) (22)
Dennis McLaughlin 464 38.5% 34.6% 19.2% .0%
(base N) (20) (26) (26) (26) (26)
WISH House 289 50.0% .0% .0% .0%
(base N) (4) (4) (4) (4) (4)
Elizabeth Stone Hse 713 10.5% 36.8% 26.3% 5.3%
(base N) (32) (38) (38) (38) (38)
St. Mary’s 319 .0% 100.0% .0% .0%
(base N) (6) (6) (6) (6) (6)
Four houses focused on clients defined by a particular primary disability: substance abuse at Dennis McLaughlin, AIDS-related problems at WISH, and, to a lesser extent, victims of abuse at Elizabeth Stone (table 7). Substantial percentages of WISH (57%) and Elizabeth Stone clients (23%) were also identified as substance abusers through other questions. Crittenton-Hastings clients were generally not identified at the initial assessment as being mentally ill or having a substance abuse problem. No St. Mary’s clients were identified at intake as having a disability.
Table 7
Family & Children Information
Data provided on the Summary & Evaluation forms indicate that the family housing providers served a total of 100 children of their SHP clients. These children tended to be young, between the ages of 1 and 12 (figure 30).
Figure 30
The Comprehensive Assessment data provide greater insight into the family situation and needs of some clients in these programs.
More than half of the family housing program participants surveyed with the Comprehensive form had never been married (figure 31). On average, they had custody of 80% of their children and were seeking help for 60% of their children (figure not shown).
Figure 31
Help with child care was the most highly rated concern in terms of children’s needs, with getting help for problems in family relations a close second (figure 32). Help with counseling and schooling for their children were rated as somewhat less important.
Figure 32
Comprehensive Assessment in Family Programs
The comprehensive assessment data also provide more detailed general background information on some of the participants in three of the five family programs.
Most family housing program participants had long histories in Boston.
· About 40% were born in Boston and another 28% had been born elsewhere in Massachusetts.
· Just 6% were from another country.
· Two-thirds had lived in Boston for more than 5 years.
· 75% reported having been homeless at some time since they were 18 years old.
Educational levels were low for the sample of family housing program participants.
· 9% had a GED.
· One-quarter reported specialized technical or vocational training.
Homelessness was not entrenched for these participants.
· 74% reported having been homeless at some time since they were 18 years old.
· They had been homeless, on average, for only 9% of the last 90 days (figure 33).
Figure 33
The service orientations of family housing program participants were diverse.
· Half had sought help for their housing needs in the last year, although only one-third had received these services (figure 34). More than half reported dissatisfaction with these services.
Figure 34
· Only 22% had sought help for their employment needs in the preceding year (figure 35). Most had received this help and few were still receiving it. They were mostly satisfied with this help.
Figure 35 
Many services were rated as at least "very important": finding housing, help with drug or alcohol problems, mental health services, safety, and making more social contacts.
Figure 36 
· Just one in five was working, although nine in ten had ever worked full-time. Only 10% were actively looking for work, although 40% reported they would like job training. One-third expressed interest in help with finding a job.
Figure 37
Health problems were common (figure 38).
· 80% reported a current health problem, although only 20% rated their health as not good.
· 60% had been prescribed medication.
· Few reported needing medical support devices or suffering from functional impairments.
· About one-third had some type of health insurance or related benefits.
Figure 38
· Just under 40% reported some symptoms of substance abuse (figure 39).
· 30% had had treatment experiences or feelings that indicate the possibility of serious mental illness.
· Legal problems were rare, although one-quarter had had some legal problems in the past.
Figure 39
Family members were the most important source of social support for the family housing program participants (figure 40). Service agency staff were least highly rated as sources of social support.
Figure 40
Program Status at Follow-up
Program status at follow-up is the first outcome indicator presented on a program-specific basis. Program dropouts receive special attention, in tables that show their reasons for leaving the SHP program and their destination after leaving. Other outcomes achieved for clients who had graduated or dropped out of specific programs are then broken down by program status at follow-up.
Program Status
Just 15% of the clients accepted had graduated from these programs, and all of these were from Crittenton-Hastings House, Dennis McLaughlin, and Elizabeth Stone House (table 8). About one-third of the clients at these programs had dropped out or been asked to leave and about 40% were still program participants. More than 80% of the WISH clients still remained in the program, as did half of those at St. Mary’s.
Table 8
Dropping Out
The reasons for clients leaving before graduation varied between the family housing programs. All premature departures from the Crittenton-Hastings program were due to noncompliance with housing requirements (table 9). Substance abuse led to most departures from the Dennis McLaughlin program, while the few clients who left WISH and Elizabeth Stone House did so of their own accord. However, the number of dropouts in each of these programs was so small that there is little basis for expecting these to be enduring differences between the programs.
Table 9
About half the clients leaving family housing programs went to live with family or friends (table 10). No other destination was named for even as many as one-fifth of the clients.
Table 10
Outcomes for Graduates and Dropouts
Several indicators are used to describe the residential, employment and benefits outcomes of graduates and dropouts of the family programs. In addition, indicators of substance abuse at follow-up are presented.
Residential Status
One in five dropouts had been housed with no indication of homelessness and just 17% had had an episode of homelessness that was known to staff (table 11). On average, dropouts whose recent residential status was known to staff had spent 20% of the preceding three months in their own homes. There were too few dropouts within any one program to allow meaningful inter-program comparisons.
Table 11
The small numbers of graduates from these programs limit the value of the outcome indicators. Overall, housing outcomes for the graduates were much better than for dropouts. At the three programs with some graduates, all clients had avoided homelessness and at least three-quarters had been housed without an episode of homelessness (table 11). The graduates had spent eighty-three percent of the preceding three months in their own housing.
Table 12
Employment and Benefits
One-quarter of the family program dropouts had worked at some time since they started the program and about one in five had worked full-time (table 13). Overall, there had been few changes for family program dropouts in income or number of government benefits received. Work experience was more common for dropouts from Dennis McLaughlin and Elizabeth Stone, but the total income and number of government benefits received by these former clients tended to decline.
Table 13
Employment and benefits outcomes were much better for family housing program graduates than for those who had dropped out (table 14). Almost half had worked at some point, their average incomes had risen, although only slightly, and they were receiving government benefits from more sources. However, income changes were available for only six of the ten graduates and the amount of income change varied markedly between the two programs that provided this information, although the number of graduates was too small to have confidence that these results represented true program differences.
Table 14
Substance Abuse
Substance abuse appeared to occur for about half of the family program dropouts, with indications of a decline in substance abuse problems compared to the initial assessment at Dennis McLaughlin and Elizabeth Stone House (table 15). Staff reported no indications of substance abuse at Crittenton Hastings House.
Table 15
Substance abuse appeared to be as common among family housing program graduates as among dropouts and there were no indications of any change in its prevalence since the initial assessment.
Table 16
Gender, Age, Substance Abuse and Outcomes
Since all but two of the family housing program dropouts and graduates were women, family housing program outcomes are not classified by gender.
Older women were more likely to graduate from family programs than were women younger than 36 (table 17). Older women were also much more likely to work full-time, but they experienced a smaller increase in number of government benefits and had lost income, on average.
Table 17
Family program clients who seemed to be abusing substances were doing more poorly than those who did not seem to be substance abusers in terms of number of unmet service needs and income gains. However, substance abuse status had little association with the likelihood of graduation from family housing programs or with subsequent residential status.
Table 18
Outcome Summary
In their proposals for SHP funding, these programs had set ambitious goals for themselves. Crittenton Hastings sought to house 90% of program graduates and to place in jobs 60% of clients who completed their GED or job training (expected to be 75% of their clients). Of those program participants leaving the Elizabeth Stone program, 85% were to have permanent housing in place and 50% were to have increased family income due to wages. 75% of program participants were to improve their work/vocational situation. All of the Dennis McLaughlin clients were to obtain permanent housing within 18 to 24 months after beginning the program and 50% were to obtain employment within their first year.
The outcomes achieved represent substantial progress toward achieving these goals, in what was actually less than one full year of operation for most programs. No graduates seemed to have experienced homelessness and at least three-quarters had been housed. However, less progress was made toward meeting job goals.
Participant Status
Of the clients who were still in the family housing programs at the time of followup, more than half were rated by staff as experiencing homelessness at some time while in the program (table 19). On average, housing program clients had spent about one-third of their days homeless during the 3 months preceding follow-up—about three times the days they had spent housed. However, the marked differences between programs in ratings of the housing experience of their participants suggests inconsistencies in the definition of homelessness and housing while in the program.
Table 19
Family housing program participants were unlikely to be employed while they were still in the program (table 20). However, their average number of government benefits had increased (by one source, on average) as had their total average incomes (by $187), compared to their benefits and incomes at the initial assessment.
Table 20
Substance abuse was identified as a problem at the initial assessment for about one-third of the family program participants who remained in their program at followup (table 21). There had been little change in the numbers who seemed to have a substance abuse problem. The distribution of substance abuse problems varied markedly between programs, with substance abusing clients the focus particularly at Dennis McLaughlin and WISH House.
Table 21
Summary
The family
housing programs accepted the types of families they were intended to serve.
Most graduates were housed consistently. Rates of program termination were
relatively high for some programs. This requires further investigation, but
may be related to the expectations for conformity to rules in long-term
housing.
Adult Housing and Stabilization Programs
These programs sought either to place clients directly into permanent housing or to provide a transitional living experience that would prepare clients for permanent housing. All offered some support services to facilitate achievement of these housing goals.
Program Descriptions
The primary service offered by Action for Boston Community Development is housing outreach. The intent of the program is to move consumers out of shelters and to prevent recurring homelessness. The program emphasizes building self-sufficiency and provides housing search and post-placement stabilization services, including information, referral, and advocacy services. Referrals were to be made from provider networks serving the disabled elders, and the single, disabled individual target groups, although some non-AFDC-eligible couples and families will also be served.
Housing provision is the primary service offered by Bay Cove’s Transition to Independent Living (TIL) program. TIL is designed to move adult male clients from the emergency shelter to independent living in the community, to assist them in maintaining permanent housing, to increase their skills and/or income levels, and to help them achieve greater self determination. Vocational goals include employment for non-disabled, and volunteering for the disabled. The central educational goal of the program is attaining the ability to read and write well enough to acquire and retain a job. Clients participate in the program for 6 to 12 months, during which they move through four phases: Phase I, emphasizing case management and support groups; Phase II, with job seeking skill development and job search assistance; Phase III, with job retention groups and more community-based agency referrals; and Phase IV - active housing search, complete shift of supports to community, and development of termination and follow-up plans. Program graduation requires completion of a service plan, current employment, continued drug and alcohol abstinence, securing appropriate housing, and completing a personalized follow-up plan.
The Greater Boston Housing Initiative’s Project Link provides housing search, placement and stabilization services for homeless adults in shelters and transitional programs.
Clients are met once a week if they are active searchers, and those on waiting lists are contacted by phone to check their progress. Stabilization services are available following placement.
The primary service offered by Justice Resource Institute’s Assisted Living Program is housing outreach for AIDS- or HIV-infected homeless individuals and their families. JRI’s primary goal is to provide services for persons with AIDS and substance abuse problems while they live in scattered-site apartments receiving Section 8 tenant assistance. Another service goal is enhancing the dignity and quality of life for those served. JRI assists participants to obtain and/or remain in permanent housing, to increase skills and/or income, and to achieve greater self-determination. JRI has found that supportive services must accompany subsidies if housing maintenance is to succeed. After a housing placement is made, the case manager makes provisions for the implementation of a service plan based on a needs assessment.
The New Victory program provides transitional housing and extensive supportive services for homeless men and women who are dually diagnosed with substance abuse and HIV disease. Support services assist residents in recovery to obtain the skills and resources needed to stabilize their lives while they enhance their ability to live independently and substance-free. Intensive counseling and educational programs are available, as well as staff who enable the residences to work with people who are newly in recovery. The usual length of enrollment was anticipated to be from 3 to 9 months, 7 days per week for each individual before permanent housing would be located.
Pine Street Inn’s Project Engage assists very long term homeless, treatment resistant, chronically mentally ill men to obtain and/or remain in permanent housing, to obtain skills and income, and to achieve greater self-determination and sociability as well as sobriety. Project Engage is a slow but persistent 3 phase plan of progressive intensive engagement comprised of a lunch program with supervised activities, community based activities, case management, development of daily living skills, part time employment, vocational training, health care and housing counseling. The plan provides these services to participants for up to two years.
Clients
The adult housing and
stabilization programs assessed 568 clients, representing 35% of the total
population served by the 1995-1996 McKinney programs.
Characteristics
Most of the programs served men who were, on average, in their late 30s or early forties (table 22). Pine Street served an older clientele and about one-third of clients at both ABCD and GBHI were women. Between 40 and 50% of clients at ABCD, GBHI, JRI, and New Victory were African-American and from 9-20% were Hispanic. At Bay Cove and Pine Street, just over half of the clients were African-American and none were Hispanic. Only at GBHI were a substantial fraction of clients (22%) not English speakers.
Table 22
No
Agency Mean Age %Female % AfroAm Hispanic English
These programs drew most of their clients from emergency shelters and the streets (table 23). Most of the rest, at Bay Cove and New Victory, came from substance abuse treatment facilities. JRI was the only program to draw many clients (50% of their total) from transitional housing.
Table 23
Needs
Average monthly incomes at the time of initial assessment ranged from $409 at Bay Cove to $874 at Pine Street, with the average for the other programs between about $520 and $620 (table 24). The Bay Cove clients did not tend to receive income from one source in particular, while clients at ABCD, JRI, New Victory and Pine Street primarily received SSI or SSDI income. GBHI was the only one of these programs where job earnings were common.
Table 24
Mean Mo SSI or Job Food
Agency Income SSDI TAFDC Income Stamps
_________________ _________ _________ _________ _________ _________
The variation in the most common primary client disabilities between the agencies reflected their different service foci (table 24). Bay Cove served primarily clients with substance abuse problems, while JRI focused on those with AIDS or AIDS-related disorders (43% of whom were identified by other questions as also having a substance abuse problem) and Victory Programs served clients diagnosed with both AIDS and substance abuse. All Project Engage clients at Pine Street Inn were identified as mentally ill, and more than half also abused substances. By contrast, almost half of the ABCD clients were judged as having no disability; about one-quarter were identified as mentally ill.
Table 24
*All Victory clients were diagnosed with both AIDS and substance abuse problems.
Program Status at Follow-up
Program status at follow-up is the first outcome indicator presented on a program-specific basis. Program dropouts receive special attention, in tables that show their reasons for leaving the SHP program and their destination after leaving. Other outcomes achieved for specific programs are then broken down by program status at follow-up.
Program Status
Just under half of the adult housing program participants were still in their program at followup (table 25). The graduation rate of 30% exceeded the rate of dropping out or being asked to leave the program (24%). All JRI participants were still in the program at their last assessment in 1996, as were about two-thirds at Bay Cove and Pine Street. Program retention was lower at New Victory, the other housing program. At New Victory almost half of those assessed were still in the program but one-third had dropped out or been asked to leave. The two programs that provided housing placement services, ABCD and GBHI, had both graduated about one-third of the clients assessed within the year. GBHI also reported that another one-third had dropped out, so that fewer of those clients initially assessed were still participating at GBHI (29%) than at ABCD (43%).
Table 25
Dropping Out
About half of the dropouts from adult housing programs were classified as voluntary and just 8% as due to failing to comply with housing requirements—this being a major issue only at New Victory. The reason for leaving was unknown for almost one-third.
Table 26
Adult housing dropouts tended to move in with family or friends (17%) or become homeless (32%). However, destination was not reported for few dropouts at all programs except ABCD.
Table 27
Outcomes for Graduates and Dropouts
Several
indicators are used to describe the residential, employment and benefits
outcomes of graduates and dropouts of the adult housing programs. In addition,
indicators of substance abuse at follow-up are presented.
Residential Status
Half of the adult housing program dropouts had been homeless at some time since starting the program and just one in twenty had been housed at some point (table 28). These housing experiences varied between the programs, with GBHI dropouts being the most likely to have been housed. ABCD dropouts seemed to have fared poorly: half had been homeless and among those for whom information was available, an average of half of the preceding three months had been spent homeless.
Table 28
Housing outcomes were positive for the programs that had graduates (table 29). Of the four programs with more than two graduates for whom information was available, only Pine Street and GBHI reported more than ten percent had re-experienced homelessness (one-third at Pine Street). Pine Street graduates had the lowest rate of housing (two-thirds); more than 80% of the graduates of the other programs were identified as having been housed and were not known to have been homeless since program intake. At ABCD and GBHI, the only programs with more than three graduates followed up, graduates had spent between 80% and 90% of the preceding three months housed.
Table 29
Employment and Benefits
Employment and benefit outcomes varied between the programs (table 30). Most of the Bay Cove and GBHI dropouts had worked at some point, compared to one-third of the ABCD dropouts and none at the other two small programs. Incomes declined for dropouts at ABCD, the only program with valid information for many dropouts at Bay Cove and New Victory.
Table 30
Employment and benefits were only marginally better for adult housing program graduates than for dropouts, although, unlike the dropouts, the incomes of graduates had increased (table 31). Most GBHI graduates were working, as were about one-third of the ABCD graduates. The likelihood of having worked full-time was substantially lower at ABCD, but not much lower at GBHI than it was for working at all. Income had risen substantially for GBHI graduates, but this was not apparent for the other programs. Graduates tended to have more government benefits than when they were assessed initially, although there had been little change in this at GBHI.
Table 31
Substance Abuse
About half of the adult housing program dropouts appeared to have had a substance abuse problem at their initial assessment or at followup, with about one in three identified as substance abusers at each time. Only ABCD reported more than a few dropouts.
Table 32
Proportionately fewer adult housing program graduates were identified as substance abusers—31%--than was the case for dropouts (table 33). In addition, indications of substance abuse were less common at followup than they had been at the initial assessment, largely due to a decline for GBHI graduates.
Table 33
Gender, Age, Substance Abuse and Outcomes
Associations with outcomes are presented for ABCD, the only adult housing program with more than a few graduates and dropouts.
Men were more likely than women to have had graduated from ABCD but they also were more likely to have experienced homelessness (table 34). Men had added fewer sources of government benefits and had experienced a smaller income gain than women.
Table 34
Age only had a weak association with outcome for clients who had left the ABCD program (table 35). Older former ABCD clients were somewhat more likely to have experienced homelessness, were rated as having more unmet service needs, and had not experienced the income gain reported by dropouts and graduates under the age of 36.
Table 35
Those rated as substance abusers were less likely to have graduated at ABCD, more likely to have experienced homelessness, and had more unmet service needs than did nonabusers. They did not differ appreciably from nonabusers in their employment or benefits status.
Table 36
Service Logs at Center House
Staff at Center House recorded service usage information on logs for 53 clients between July and December, 1996. Most (38) of these clients were classified as continuing participants at the time of follow-up. These data indicate that clients who received more special education services were more likely to be housed and not homeless (r=.43, p<.001), while those who received more supported employment services were much more likely to be employed at some point (r=.71, p<.001). Although these associations do not establish a causal effect of service delivery on client outcomes, they suggest that more staff service effort is associated with improved client outcomes.
Outcome Summary
Outcome standards must reflect the different goals of these programs. GBHI sought to place clients in housing, while other programs sought to provide a transitional residential experience. These outcomes were generally consistent with housing and employment goals set in the SHP proposals submitted by these programs. ABCD exceeded its proposed goals of 75% housing placement and 20% employment. GBHI housed most graduates as expected and exceeded its employment goal; it also made substantial gains in client incomes, as planned. The total number of graduates is small for both Victory and Pine Street (N=3 for both programs), so outcome figures are not reliable. Nevertheless, the programs appeared to be achieving the desired housing outcomes. New Victory exceeded its goal of 50% housing. Employment and income were not yet improving for Victory and Pine Street graduates at this point.
Participant Status
Adult housing program clients who were still in their program at follow-up were generally rated as still being homeless (table 37). JRI participants were the only exception, with more than half having been housed without any indication of homelessness.
Table 37
Overall, one-third of
the adult housing program participants who were still in the program at
followup had worked since the initial assessment, but fewer than one in ten had
worked full-time. However, these figures varied markedly between programs,
with work experience concentrated at Bay Cove and GBHI. Monthly incomes had
tended to increase for participants at these two programs, but not for
participants at the other
programs. In contrast, participants at the other programs tended to be
receiving more government benefits, compared to those still at Bay Cove and
GBHI.
Table 38
Substance abuse was identified for about half of the adult housing program continuing participants, but indications of this problem had declined since the initial assessment for clients at all the programs except ABCD (table 39).
Table 39
Summary
Adult housing and stabilization programs enrolled the type of clients they were designed to serve and were succeeding in placing many of them in housing. However, rates of program dropout were high for several programs. Since several programs were providing long-term stabilization services in a transitional setting, low graduation rates in the first program year do not indicate a problem, but longer-term follow-up is essential.
Employment and Training Programs
The largest employment and training program, Impact Employment Services, placed clients directly into jobs or, if necessary, provided some support services until they were job ready. The other programs employed clients directly, either in an agency setting (Center House) or in the community (Project Place and Downtown Crossing).
Program Descriptions
The Friends of Shattuck Shelter (IMPACT Employment Services) offers Boston -wide assessment and employment assistance. Success is defined as assisting homeless individuals in achieving economic self-sufficiency through employment, not just getting a job, and helping clients to live in the community of their choosing. Specific services include employment workshops, development and retention, employer liaisons, and upgrading, as well as referrals for child care, substance abuse support services, and housing. Counselors determine whether a client is employment ready or ready for pre-employment training and counseling.
The primary service offered by Center House, Inc. is vocational outreach. Center House, Inc., accepts clients regardless of age, gender, race or ethnicity. Some clients have the secondary disability of substance abuse. Center House does not have a cut-off point for clients who are offered and may require a variety of vocational and supportive services. Center House aims to provide supportive employment services and if necessary mental illness, substance abuse and housing information programs. The program is completely drug free, although it does not deny education or employment services to those in active treatment. Violations of physical safety codes result in immediate expulsion.
The primary service of the Interseminarian Project Place Economic Development Partnership Program is supported employment and training. The program enables participants to become socially and economically self-sufficient through permanent employment in a Pepsi-Cola/Project Place vending machine business. Clients are adult men and women who have resided previously in Boston- based shelters or transitional programs. Program eligibility requires that applicants and clients alike possess the physical and emotional stability necessary to meet the challenges of the job to be performed and have commitment to a self-improvement plan.
The Downtown Crossing Association/Project Place Clean Corners…Bright Hopes collaboration provides employment through a public facilities management program in Boston business districts. Employment and competitive wage status are intended to enable clients to secure housing, health and dental benefits, without specific time limitations. Client characteristics and eligibility requirements are the same as those for participants of Interseminarian Project Place’s Project Pepsi.
Clients
The employment and training programs assessed a total of 511 individuals in the project year—32% of the total SHP population.
Characteristics
Clients were in their mid-30s, on average and most were men (table 40). Only Center House had as many as one-third female clients. African-Americans represented at least four in ten of those assessed by each program.
Table 40
No
Agency Mean Age %Female % AfroAm Hispanic English
_______________ _________ _________ _________ _________ _________
Shelters were the most common prior living situation for these clients. Transitional housing was also reported for at least 16%, at all these programs except for Impact Employment Services. More of Impact Employment Services’ clients came from substance abuse treatment facilities or had been living with friends or relatives.
Table 41
Needs
Monthly incomes tended to be high in three of these four programs at intake, but for different reasons (table 42). More than half of the Center House clients received SSI or SSDI benefits. Project Place and Downtown Crossing clients were already employed at the time of assessment, as indicated by the fact that they all received job income. Impact Employment Services clients, on the other hand, had few benefit sources other than food stamps, and were receiving an average monthly income of only $163 at the time of assessment.
Table 42
Mean Mo SSI or Job Food
Agency Income SSDI TAFDC Income Stamps
________________ _________ _________ _________ _________ _________
All of the Center House clients were identified as mentally ill, with just 8% of these also having a substance abuse problem (table 43). A majority of clients at the other three programs were identified as not having a disability. Substance abuse was the reported disability for almost all the rest.
Table 43
Comprehensive Assessment at Project Place
Most Project Place participants had long histories in Boston.
· About 40% of the Project Place participants were born in Boston, and another 19% had been born elsewhere in Massachusetts.
· Just 14% were from another country. These three participants had green cards.
· Eight in ten had lived in Boston for more than 5 years.
Educational levels were relatively high for the Project Place participants.
· 24% had earned a GED.
· Almost half reported having had specialized technical or vocational training.
· Few had had stable residential histories. 86% reported having been homeless at some time since they were 18 years old.
· On average, they had been homeless for 25% of the preceding 90 days (figure 41).
Figure 41
· Almost 60% had sought help for their housing needs in the past year. 40% had received and were still receiving these services (figure 42).
Figure 42
The Project Place participants were strongly job oriented.
· Almost 90% had sought help for their employment needs in the preceding year and these clients had all received that help (figure 43). Almost 70% reported that they were still receiving help with getting a job. Satisfaction with job help was very high, expressed by almost 90%.
Figure 43
· Service interests were dominated by employment-related issues, in addition to finding housing (figure 44). Finding a job, improving job skills, and taking courses were each rated, on average, as very important. Help with substance abuse and mental health problems were the next most important service concerns, but received average ratings of only "a little important."
Figure 44
· All were working at the time of the interview (figure 45). Few were actively looking for work or sought job training or help with finding a job.
Figure 45
Physical health appeared to be good among Project Place participants, but substance abuse was a major concern (figure 46).
·
None reported that their health was not good and only 20%
reported any current health problems or had been prescribed medication. Almost
none reported use of health devices or any functional limitations.
· 60% had access to some type of health benefits.
Figure 46
· All Project Place participants reported symptoms of substance abuse.
· 40% reported indications of serious mental health problems.
Legal problems were common (figure 47).
· 40% reported current legal problems; three-quarters had been arrested, convicted, or jailed in the past.
Figure 47
Project Place participants relied on friends for social support, even more than they relied on family (figure 48). Agency staff were rated as much less important as a source of social support.
Figure 48
Program Status at Follow-up
Several indicators are used to describe the residential, employment and benefits outcomes of graduates and dropouts of the adult housing programs. In addition, indicators of substance abuse at follow-up are presented.
Program Status
Few of the clients who had been assessed by Impact counselors were still in the program at follow-up (table 44). Rates of current participation were much higher at the other three programs, with Center House and Project Place still retaining at least two-thirds of the clients they had first assessed during the year. Impact was the only program with a substantial graduation rate (44%), but both Impact and Downtown Crossing had experienced a substantial dropout rate (above 40%).
Table 44
Dropping Out
Reasons for leaving were recorded primarily for Impact dropouts. These clients had most often left for voluntary reasons. Substance abuse was identified as a reason for leaving by more than 10% of the dropouts for whom information was available at Center House, Project Impact, and Downtown Crossing.
Table 45
Home ownership was the destination of one-quarter of the Impact dropouts, while alternative supportive housing was the destination of two of the five Center House dropouts (table 46).
Table 46
Outcomes for Graduates and Dropouts
Outcomes differed markedly between Impact, with its goal of rapid job placement, and the other programs, that provided jobs within their program or support services to working clients.
Residential Status
Although just 14% of the employment program dropouts had been housed without having experienced homelessness, more than one-third of those from Impact had been housed at some point (table 47). However, the rate of having experienced homelessness was high at Impact and substantial at the other programs.
Table 47
Housing outcomes were better for graduates of Project Impact than for dropouts; sufficient data were not available from other programs (table 48). More than one-quarter had been housed without any indication of homelessness and almost two-thirds had been housed at some point. However, many of the graduates had experienced homelessness at some point and they had been housed for just 21% of the 3 months preceding the followup assessment.
Table 48
Employment and Benefits
Fewer than half of the dropouts from Center House and Impact had worked since the initial assessment (table 49). Of course, all those leaving Downtown Crossing and Project Place had been employed, since in these programs all clients were employed when they began. The dropouts from Center House and Impact for whom data were available were still receiving more income than they had at the initial assessment, although they had not gained more government benefits.
Table 49
Among the employment programs, only Impact reported a number of graduates. All had had jobs and their average monthly income had increased since the initial assessment by $870, on average. They also had reduced the average number of government benefits they were receiving.
Table 50
Substance Abuse
Substance abuse was identified for half of the employment program dropouts, although it was less common for those from Center House (table 51). Indicators of substance abuse had not changed appreciably since the initial assessment.
Table 51
Substance abuse was as prevalent among Impact graduates as it was among their dropouts. There were insufficient Center House graduates to support any conclusion about the prevalence of substance abuse.
Table 52
Gender, Age, Substance Abuse and Outcomes
Women who had left Impact were less likely to have graduated or to have worked full-time and had achieved less of an income gain than did men. They also were reported to have more unmet service needs than did men.
Table 53
Age had no more than a negligible association with outcomes for Impact clients, although younger clients were somewhat more likely to have worked full-time.
Table 54
Outcomes for Impact clients differed little for those who were identified as substance abusers and others. The risk of homelessness was somewhat higher for substance abusers.
Table 55
Repeat Follow-up at Impact
Seventy-nine Impact clients were followed up for a second time between January and May, 1997, approximately four months after their first follow-up. All but five of these clients had already graduated from or dropped out of the Impact program by the time of their first follow-up, but many continued to maintain contact with Impact staff. In fact, the frequency of staff contact with these clients had actually increased between the first and second follow-ups.
In general, outcomes improved for these Impact clients. One-quarter of the program drop-outs who were not working at the first follow-up had had some work experience by the time of their second follow-up. One in five of the Impact clients who had experienced homelessness prior to their first follow-up avoided this experience before their second follow-up, as far as program staff knew. The number of government benefits they were receiving tended to decline during this period. Indications of substance abuse decreased in frequency. There were no appreciable changes in service needs or income for these clients during this period.
Outcome Summary
The employment and training programs tended to serve young and middle-aged men, many of whom were black or Hispanic and most of whom, except at Center House, had a substance abuse problem. Rates of client dropout were a problem at two programs. Using a direct job placement approach, Impact Employment Services was successful in securing jobs for all of its graduates as well as increasing substantially their monthly income. However, housing status was often not improved by the time of the follow-up assessment.
The two Center House graduates do not provide a sufficient basis for generalizations about outcomes. Impact appeared to be exceeding its job placement goal (64% of those judged to be job ready), but was not reaching its proposed goal to place 55% of employed participants into housing. Female clients were less likely to have graduated or to gain in monthly income. Clients with substance abuse problems were somewhat more likely to have experienced homelessness since the initial assessment. Older clients were less likely to have worked full-time. There were no differences in outcomes by race identified for Impact clients.
Participant Status
About one-quarter of the clients who were still in employment programs at follow-up had been housed without experiencing homelessness; about half had experienced homelessness (table 56). Homelessness tended to have occurred for a longer period of time for clients at Impact.
Table 56
Those who were still in the Project Place and Downtown Crossing programs at follow-up were employed, while no more than one-third were at Center House or Project Impact. Average monthly incomes had increased at each program other than Center House, with no change in number of government benefits received.
Table 57
Substance abuse was indicated among about one-third of the participants at the initial assessment, but except at Center House indications substance abuse declined among continuing participants.
Table 58
Summary
The contrast between the three types of employment and training program represented by Center House, Impact, and Project Place/Downtown Crossing show how important it is to take into account program goals and clientele when comparing outcome statistics between programs. Center House provided services to homeless mentally ill persons, so it had longer periods of engagement with clients and tended to have relatively few graduates. Project Place also engaged clients for long periods, but this because it actually provided jobs to participants. Impact, on the other hand, placed homeless job seekers rapidly into jobs and thus tended to have much less outcome data.
Outreach Programs
Two very different programs provided outreach in shelters. The Committee to End Elder Homelessness focused on connecting elders with housing opportunities. Greater Boston Legal Services helped families in shelters to identify additional benefits and services to which they were entitled (including nutrition, cash benefits, education and training, child support and child care). GBLS also helped clients properly determine all the preferences and priorities of the public and subsidized housing to which they were entitled, thereby increasing clients’ likelihood of obtaining affordable housing. Substantial legal advocacy was provided to many of these families.
Program Descriptions
Housing provision and housing outreach are the primary services offered by The Committee to End Elder Homelessness’ Special Initiative to End Elder Homelessness. The program identifies and assesses homeless elders, places them in housing and stabilizes them in safe, affordable and permanent housing (including specialized sites such as mental health facilities, nursing homes, chronic care hospitals, and hospices). There is a strong effort to link elders to existing housing and services. Case managers provide outreach, assessment, individual case management, housing search and stabilization services, including transitional housing, respite and specialized treatment facilities which prepare individuals for permanent housing and independent living, as well as a benefits entitlement analysis.
The primary service offered by Greater Boston Legal Services’ Project New Beginnings is legal advocacy. In addition to assisting participants to obtain additional benefits, services, and permanent housing, Project New Beginnings also aims to help participants achieve greater self-determination. Prospective clients are not restricted on the basis of age, gender, race, ethnicity or disability. Referrals to the project are made by shelter staff.
Clients
These two programs served markedly different client populations. Altogether, they assessed 426 adults—26% of the total SHP population.
Characteristics
GBLS clients were young women, most of whom were either African-American or Hispanic (table 59). CEEH clients, by contrast, had an average age of 65 and two-thirds were men. A majority were white and just 8% were Hispanic.
Table 59
No
Agency Mean Age %Female % AfroAm Hispanic English
__________________ _________ _________ _________ _________ _______
Almost all GBLS clients were living in an emergency shelter. Those seen by CEEH staff were most often in shelters (45%), but also were identified on the streets, in transitional housing, with friends or relatives, and in rental housing.
Table 60
Needs
Average monthly incomes for both GBLS and CEEH clients at the initial assessment were above $450 (table 61). For GBLS clients, this income was primarily from AFDC. Benefit sources were available on just 81 of the 200 clients in the CEEH database. SSI and SSDI were a source of benefits for 59% of these clients.
Table 61
Mean Mo SSI or Job Food
Agency Income SSDI TAFDC Income Stamps
______________________ ______ _________ _________ _________ _______
In general, GBLS staff did not identify disabilities among their clients, except for about 10% thought to be mentally ill. Physical illness was a problem for about a third of the CEEH clients and mental illness was also common. Substance abuse was identified as a problem for about one-quarter (including, for 6%, a dual diagnosis with mental illness).
Table 62
Outreach Programs
Family & Children Information
Summary & Evaluation data indicated that the clients assessed by GBLS staff had a total of 405 children. These were mostly toddlers (age 1-5) and young children (6-12).
Figure 49
Program Status at Follow-up
Some information on
benefits and program status were available from GBLS for a limited number of
the persons who were assessed, but these data were not collected with the SHEP
follow-up instrument. As a result, these data for GBLS are not presented here
in tabular form. CEEH completed followup forms for all the homeless persons
they had assessed.
Program Status
Many (42%) of the persons assessed by CEEH did not begin the CEEH program. Just 15% had graduated and about the same percentage (20%) had dropped out after enrollment.
Table 63
Dropping Out
Almost half of those who were classified as Committee dropouts had left for voluntary reasons; the reason for leaving was unknown for most of the rest.
Table 64
About one-quarter of the Committee dropouts went to the streets or a shelter, while more than one in five became psychiatric hospital patients.
Table 65
Outcomes for Graduates and Dropouts
Outcomes are presented only for those who left the Committee program.
Residential Status
About one-quarter of the CEEH dropouts had been housed without experiencing homelessness; more than half had been homeless (table 66). For those for whom the data were available, about half of the preceding three months had been spent homeless, compared to just under one-third of the time that had been spent housed.
Table 66
Committee graduates were much more likely to have been housed than were dropouts. More than 60% had been housed without experiencing homelessness and almost all had been housed for some time. They had spent more of the preceding three months housed.
Table 67
Employment and Benefits
CEEH dropouts were unlikely to have worked and had achieved only small gains in income and number of benefits.
Table 68
Although few CEEH graduates had worked, they were receiving, on average, $143 more dollars per month than they had at the initial assessment and they had about one more source of government benefits.
Table 69
Substance Abuse
Substance abuse was noted for about one in five CEEH dropouts, with no indication of change from the initial assessment to the followup.
Table 70
Substance abuse was at least as common among CEEH graduates as was reported for dropouts, again with no change indicated from the initial assessment to the followup.
Table 71
Gender, Age, Substance Abuse and Outcomes
In general, outcomes did not vary for CEEH clients in relation to gender, age, and substance abuse.
Outcome Summary
The housing outcomes exceeded the proposed CEEH goal of 50%
housing placement. The increased income and receipt of more government
benefits are consistent with proposed goals (86% to have access to public
benefits programs). There were no important relationships between
sociodemographic characteristics of CEEH clients and their program outcomes.
Participant Status
Those who remained in the CEEH program were more likely to have experienced homelessness than to have been housed without the experience of homelessness.
Table 72
Working was uncommon among continuing CEEH clients, but these clients had increased their monthly incomes and receipt of government benefits since the initial assessment.
Table 73
Indications of substance abuse were more common among continuing CEEH participants at followup than they had been at the initial assessment, although the estimate of substance abuse based on the combination of these two assessments was similar to that obtained for the CEEH dropouts and graduates.
Table 74
Summary
These two outreach programs had both been able to reach out and assess the type of clients they had been designed to serve. CEEH assessed a large number of clients and placed many graduates in housing. GBLS also assessed a large number of clients who were caring for hundreds of children.
Conclusions
The set of 17 McKinney SHP programs seemed to be filling multiple gaps in the continuum of care. They served a large number of clients and provided, collectively, a mix of services. Their clients were diverse in terms of social background and service needs. They were generally very impoverished, with one-third having no income at all upon initial assessment, and they were shelter-poor. However, the prevalence of disabilities and resources varied both between and within program types.
Many SHP clients achieved impressive gains in residential and employment status, so that the programs often achieved most of the goals they had set out for themselves in their SHP proposals. Most graduates obtained housing and many secured jobs and received more income. Income gains were most often due to jobs, rather than to receipt of more types of government benefits (although it should be noted that being employed can require more expenditures than receiving benefits, thus lowering the value of the added income). Although a much longer and more consistent follow-up effort is needed, it appears at this early point that the SHP programs are helping clients to move toward greater independence and residential stability.
Of course it cannot be assumed that these client gains were a consequence of enrollment in the SHP programs. It is possible that the gains would have occurred for the type of clients who graduated from SHP programs, even if they had not entered the program. Without a control group, it is not possible to rule this out.
Program dropouts were common and identifying the sources of dropping out and collecting more adequate follow-up data for dropouts is an important next step for data collection efforts. Analysis of the 1996 SHEP data identifies substance abuse as a common correlate of poor outcomes, with age and gender also being consequential for some types of program. More data like that collected with the SHEP comprehensive assessment tool will be required to improve predictions of the types of clients that will have the most problems achieving successful outcomes in particular types of programs.
The analysis also indicates some of the different influences on housing and job outcomes. Substance abuse was often associated with poorer housing outcomes but it had little relationship to employment outcomes. The influence of gender also varied across these outcomes, with women less successful in achieving employment. Coordination between programs focused on housing and employment seems essential for maximizing overall outcomes.
The SHEP system succeeded in collecting comparable data from more than 1600 clients served by 17 different programs, with most programs providing follow-up data on almost all clients. This achievement demonstrates conclusively the feasibility of system-wide data collection, data-base management, and client tracking efforts. By implication, it also establishes the readiness of program staff to participate in such efforts. The value of this effort for increasing understanding of client needs and gaps in service provision is apparent in many of the findings presented in this report.
In spite of this positive outcome, the SHEP project also demonstrates the need for significant resources to develop and maintain an adequate system. The demands on program and SHEP staff during its first year exceeded what was feasible given SHEP resources. As a result, contact with program staff was not consistent enough to maintain a regular data flow and staff training was too limited to ensure data quality. The importance of enforcing high standards for data collection and providing the necessary resources is apparent in the differences between the clients who were followed up (or assessed comprehensively) and those who were not followed up (or assessed comprehensively) at the programs that achieved only partial coverage in their follow-up or comprehensive assessment efforts. These differences could result in either over-estimating or under-estimating the outcomes achieved by the total number of clients served by these programs. For the same reasons, it also is important for staff to minimize the rate of missing information on the forms that they complete.
A shift from a planned triennial to an annual period for collection of follow-up data at most programs reduced the data collection burden on staff, but it also led to less complete follow-up data, since the proportion of missing data on follow-up forms declined as the time since the last client contact increased.
It is not reasonable to simply impose a standardized data collection system on all programs without consideration of their unique circumstances and needs, but any deviation from standardization can create problems for data quality. For example, the SHEP system was designed to allow programs to use their own assessment instruments and then summarize client information on the standard SHEP Summary and Evaluation form. While this arrangement worked as intended, those programs who used their own assessment tool tended to provide only partial summary information to SHEP.
Data provided by programs using the SHEP Comprehensive Assessment tool were much more thorough. As the numbers of clients assessed with this form increases, these data will support much more sophisticated analyses of client needs and correlates of client outcomes. However, the use of this form is restricted to programs that can benefit from comprehensive client information and that are willing to devote more staff time to assessment. Simply put, there is a tradeoff between the burden imposed on program staff by data collection requirements and the quality of the data that are likely to be obtained. This tradeoff must be evaluated carefully by all participants in the system in order to achieve an appropriate balance.
Implementation of the SHEP system permits for the first time comparison of program operation with client characteristics and outcomes. Such detailed comparison should be used to guide discussion between program directors and municipal officials about needed services. In addition, this information may identify the value of system-wide service agreements, with a goal of providing clients with more appropriate service placements after they are first assessed by any program in the system.
It is in documenting the contributions of the SHP service programs to the reduction of homelessness that the value of the SHEP system is most apparent. High rates of housing after graduation, job acquisition, and improved income, without reliance on government benefit programs, each signal an important contribution to Boston’s continuum of care for homeless persons.
Appendix
SHEP Summary/Evaluation Form
SHEP Staff Follow-up Form