Housing Counseling Process Evaluation and Design of ... - HUD User

parent organization operating nationally, HUD’s administrative information did not include contact
information at the local agency level. As a result, we attempted to work through the parent
organization to obtain responses for their branches. Unfortunately, despite repeated efforts, we were
unable to get a response from this intermediary. This one organization represents 8 percent of the
total group of agencies targeted by the survey. Given the importance of this one organization among
HUD-approved agencies, in the examination of response rates by organizational characteristics
presented below we provide information on the distribution of agency characteristics both including
and excluding these branches.
While the response rates obtained were close to the goal of an 80 percent rate, there were a number of
challenges in conducting the survey reflecting the nature of the population surveyed. Many of the
agencies surveyed are small non-profit organizations. These agencies reported both limited staff
availability to respond to the survey and limited technical capability for navigating the web survey
itself. Many agencies reported a lack of willingness to complete the survey due to lack of
significant—or any—financial support from HUD coupled with already extensive reporting and
oversight requirements associated with being HUD-approved. Nonetheless, as shown in Exhibit B-1,
excluding the non-responding intermediary branches and the agencies that are no longer HUD
approved, the survey obtained a 79 percent completion rate, including both those formally submitted
and those not submitted but essentially completed. Thus, with the notable exception of the response
from the one national intermediary, the survey response rate was very close to the goal of an 80
percent response rate.
The possibility of any bias in survey response rates can be assessed using HUD administrative data
that is available for all or most HUD-approved agencies. For all HUD-approved agencies we have
information on the state where the agency is located, so we can assess response rates for any state or
regional geographical bias. For most HUD-approved agencies (87 percent) we also have 9902 data
for FY 2006 to examine response bias associated with an agency’s client volumes or mix of client
types.
Exhibit B-2 summarizes the response rates by agency size in terms of the number of clients served in
fiscal 2006. The lowest response rate was among agencies that did not report 9902 data for fiscal
2006, with only 62 percent of these agencies responding compared to an overall response rate of 72
percent. There were slightly lower response rates for smaller agencies (less than 500 total clients in
FY 2006), but this largely reflects the fact that the branches of the non-responding national
intermediary generally had small volumes of housing counseling clients. The last column shows
response rates excluding this intermediary’s branches, which shows very little variation in response
rates by agency size once these branches are excluded.
170
Appendix B. Survey of HUD-Approved Counseling Agencies