Contribution Accountability Framework
Evaluation Framework for
Citizenship and Immigration Canada's
>> Adobe® Acrobat format, 847 K 60 pp
Dr. Gillian Kerr
May 3, 2004
This Settlement Evaluation Framework is the product of several research activities. The current version of the document was prepared by RealWorld Systems, based closely on a draft Settlement Evaluation Framework that was prepared by Goss Gilroy Inc. (GGI) and submitted to CIC in January 2004. Most of Section 2 and large parts of other sections, along with the evaluation framework and logic model for LINC, were retained from the GGI version with little or no alteration. The revised sections incorporate findings from a series of 14 workshops across the country carried out by RealWorld Systems with representatives of service provider organizations delivering the Immigrant Settlement and Adaptation Program (ISAP) and the Host Program. RealWorld Systems also consulted recent literature on settlement research as well as several key informants, including CIC representatives.
The Settlement Evaluation Framework prepared by GGI in January 2004 was itself based on a number of activities. Firstly, GGI carried out a series of workshops with representatives of service provider organizations delivering the Language Instruction for Newcomers to Canada (LINC) Program. Secondly, GGI interviewed several LINC, ISAP and Host program officials at CIC Headquarters in Ottawa to update the descriptive program information for those programs. Thirdly, the Framework drew on information in the Evaluation Frameworks prepared by GGI for each of the settlement programs (LINC, ISAP and Host) in 1999 and 2000.
Goss Gilroy Inc. and RealWorld Systems would like to thank all the individuals who participated in these projects for their time and valuable contributions. As well, the assistance of CIC Headquarter and Regional representatives, who took the time to review the draft deliverables and provide suggestions for improvement, was greatly appreciated.
3.1. PROGRAM RESOURCES
3.2.1. Settlement Division
3.2.2. CIC Regional Offices
3.2.3. CIC Local Offices
3.2.4. Service Providers
3.3. PROGRAM DESCRIPTION
3.3.2. Support of Service Providers
3.3.6. Host Program
4.2. ATTRIBUTION PROBLEMS
4.4.3. Host Program
Citizenship and Immigration Canada (CIC) is in the process of evaluating all of its national settlement programs. This report lays out an evaluation framework and logic models for CIC's programs, and defines some key questions that will guide the program evaluations.
The framework aims to clarify the desired results of CIC's settlement programs for their many stakeholders – CIC staff, Service Provider Organizations (service providers), evaluators, and senior members of the federal government. Like all frameworks, it should be revised and updated as new research expands current knowledge of what interventions lead to successful settlement and integration.
Stakeholder groups will use this framework in different ways:
CIC staff at National Headquarters will use the framework to define the requirements for evaluations for the settlement programs in 2004. In addition, the output variables and activities contained in the framework may lead to revisions of some iCAMS variables to reflect changes in activities and recommended outputs.
CIC staff in the Regions can use the framework to guide contribution negotiations with service providers. For example, service providers who can demonstrate success in key immediate outcomes may be able to make a case for increased funding by meeting CIC objectives for greater service effectiveness.
Service Providers can use the framework to help them develop data collection processes that will allow them to demonstrate success in meeting program outcomes. Service providers can also use the program logic models to review their service delivery models so that their activities lead logically to desired immediate outcomes.
The first draft of the evaluation framework for CIC's settlement programs was written in 1999/2000. Since then the framework has been significantly revised, based on dozens of meetings, workshops and conversations with service providers and CIC staff. Interviews with key informants and a review of the literature on settlement and labour market integration also contributed to the revisions. The evaluation framework for LINC was discussed in workshops with service providers in the fall of 2003, and a similar exercise (involving 14 workshops across the country) was carried out for the ISAP and Host programs in February and March 2004. The program logic models in Section 5 distill hundreds of comments and suggestions into four highly simplified diagrams.
Following are some key issues that should be kept in mind while reviewing this document.
Section 5 of this paper – Settlement Program Logic Models – is intended to be used as a stand-alone summary of the evaluation framework. It contains the logic models for all three programs, the logic model for CIC's settlement programs as a whole, and a few notes on implementation.