Archived - Evaluation of the Research and Policy Initiatives Assistance Program - Finance Canada

Archived information

Archived information is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Prepared for:  Finance Canada
(Program Evaluation and Performance Measurement Services)
March 2009

Table of Contents

Executive Summary

1.0 Introduction

1.1 Background
1.1.1 Budget

2.0 Evaluation Methodology

2.1 Evaluation Issues and Questions
2.2 Methodology
2.2.1 File Reviews
2.2.2 Stakeholder Interviews
2.3 Limitations of Methodology

3.0 Evaluation Findings
3.1 Program Relevance
3.2 Program Success
3.3 Program Cost-Effectiveness and Alternatives

4.0 Summary of Conclusions and Recommendations

Appendix A - List of RPIA-Approved Projects

Appendix B - Evaluation Matrix

Appendix C - FC RPIA Logic Model

Appendix D - Interview Guide

Executive Summary

This report presents the findings and conclusions of the evaluation of the Research and Policy Initiatives Assistance (RPIA) Program for Finance Canada (FC). The evaluation was undertaken to respond to the requirements for an evaluation of the Program for program renewal, and in accordance with the Transfer Payment Policy and Treasury Board requirements. The evaluation findings will also serve program managers by providing information to improve program delivery and performance. TBS Internal Audit and Evaluation Branch undertook a Stewardship function over the course of the evaluation. The evaluation focused on three main areas: relevance, success, and cost-effectiveness.

Description of the RPIA

The RPIA Program in FC is a class contribution program used for small ad hoc initiatives that contribute to FC objectives but do not deliver a clear good or service. A maximum of $25,000 per year can be paid to a single recipient and up to $100,000 per recipient over a five year period, in exceptional cases, such as multi-year projects. One hundred thousand dollars ($100,000) per year has been allocated for this Program for the department as a whole. Use of this authority requires the recommendation of a Branch Head, the sign-off of the department’s Senior Full-time Financial Officer and the approval of the Deputy Minister.

Over the six-year period of the evaluation (i.e., 2003/4 to 2008/9), a total of $173,700 was approved through the FC RPIA Program.

Evaluation Methodology and Limitations

The evaluation methodology was designed to ensure that the cost of the evaluation was commensurate with the relatively small amount of funding provided through this program. As a result, the evaluation integrated the use of only two lines of evidence:  file review and key stakeholder interviews.

As with any evaluation, there are limitations to the evaluation methodologies. Firstly, the interviews and file review did not include all projects that received funding through the FC RPIA Program. However, efforts were made to select those projects that accounted for the largest percentage of Program spending over the evaluation period. GCS considers the files selected to be a good sample, and believe they represent FC projects under the RPIA.

A second limitation of the evaluation relates to a lack of quantitative information. The interviews and file review conducted for the evaluation are qualitative sources of information. Little quantitative information was available about the Program, particularly with regards to benefits or outcomes of the Program.

Program Relevance

RPIA projects are aligned with Program and departmental objectives. RPIA projects reviewed were related to conducting research and supporting conferences on topics relevant to the objectives of FC and the Program. Topics examined through the RPIA projects reviewed included: the evolution of the financial services sector and the implications for Canadian providers; the improvement, advancement and promotion of sound corporate governance practices; the advancement of knowledge in the area of private pensions; and the development of private sector expertise in tax policy issues.

The RPIA Program is meeting the needs of FC sectors by providing funding for projects that may not otherwise have been possible. Interviewees noted that there were no in-house resources available to conduct the work done through the RPIA projects. As well, generally, interviewees said that the RPIA project activities would not have been carried-out by recipient organizations without the provided funding. According to interviewees, FC would not have had the same benefits had they not provided the funding. For example, organizations would not have been able to undertake as much research; FC would not have had as much influence on the research agenda; and researchers would not get involved in tax policy research. As well, in the absence of FC funding, projects probably would have been done on a smaller scale, with lower involvement by others.

Program Success

The file review conducted for this evaluation highlighted a lack of evidence in FC files regarding the benefits of the projects. For the most part, the only information available from the file review, with regards to benefits of the projects, was information about anticipated benefits, as opposed to actual benefits realized. The RPIA Program does not have a requirement to provide any kind of post-project summary for the project file. Thus, actual benefits to FC as a result of these projects are not routinely documented.

Recommendation #1:  The program area should implement a mechanism to ensure that the impact of the Program can be measured with respect to actual benefits, rather than anticipated benefits.  For example, a post-project report could be a requirement for all RPIA projects. This could be as simple as a one-page template that is filled out for each project.

Interviewees believe that the RPIA Program has contributed to the outcomes of the RPIA Program (i.e., enhancing policies or programs, increasing knowledge of emerging policy issues, and building capacity for analysis).

According to interviewees, the RPIA Program has helped to enhance policies or programs for which their sector is responsible because it allows FC to support research that enables them to offer better policy advice. Interviewees also agreed that the RPIA Program has helped to increase knowledge of emerging policy issues.  Finally, interviewees noted that the RPIA Program has helped to build capacity for analysis. Because the research is influenced by FC, it is tailored to FC’s needs and thus increases FC’s capacity to produce thorough and relevant analyses. RPIA-funded research also increases the amount of information available that can be used in analyses.

Program Cost-Effectiveness and Alternatives There appear to be few options for alternatives to the Program. Interviewees pointed to insufficient resources (i.e., human and financial) for FC to be able to conduct the work in-house. Issues were identified with regards to the only other potential alternative suggested (i.e., contracting the work) mainly due to the fact that they lacked the appropriate legal framework or contracting was considered to be less efficient than a contribution program.

Interviewees believe the Program is cost-effective because of the benefits of partnering with other organizations (i.e., leveraging). In most cases, FC provided only a portion of the funding to carry-out projects. For the files reviewed, FC contributed anywhere between 12 percent and 100 percent of the total cost of the projects.

Because of the nature of the Program and its relatively small size, program efficiency was not seen as a relevant issue on which to focus in this evaluation. However, some information was gathered during interviews that related to efficiency. Interviewees were generally positive in their views on program administration. Most interviewees noted that the application process was straightforward and easy, and approvals were timely.

Few program improvements were identified, although the file review undertaken for this evaluation highlighted the fact that not all project-related documents (i.e., the submission/ recommendation forms) were available through the RPIA Program Manager, and since post-project reports are not required on file, the actual benefits of projects are not routinely documented.

Recommendation #2: Project documentation should be collected and retained more systematically and stored in a central location.

1.0 Introduction

This report presents the findings and conclusions of the evaluation of the Research and Policy Initiatives Assistance (RPIA) Program in Finance Canada. The evaluation was undertaken to respond to the requirements for an evaluation of the Program for its renewal, and in accordance with the Transfer Payment Policy and Treasury Board requirements. The evaluation findings will also serve program managers by providing information to improve program delivery and performance. Finance Canada (FC) and Treasury Board Secretariat (TBS) engaged Government Consulting Services (GCS) to undertake the evaluation. TBS Internal Audit and Evaluation Branch undertook a Stewardship function over the course of the evaluation.

The objective of the study was to evaluate the Program in terms of relevance, success, and cost-effectiveness/alternatives. The research for this evaluation was conducted between December 2008 and January 2009.

The evaluation report is organized as follows:

  • Section 1 presents a description of the RPIA Program;
  • Section 2 presents the methodology for the evaluation;
  • Section 3 presents findings by evaluation issue and question; and
  • Section 4 presents the conclusions and recommendations.

1.1 Background

The RPIA Program in FC is a class contribution program used for small ad hoc initiatives that contribute to FC objectives but do not deliver a clear good or service. A maximum of $25,000 per year can be paid to a single recipient and up to $100,000 per recipient over a five year period, in exceptional cases, such as multi-year projects. One hundred thousand dollars ($100,000) per year has been allocated for this Program for the department as a whole. Use of this authority requires the recommendation of a Branch Head, the sign-off of the department’s Senior Full-time Financial Officer and the approval of the Deputy Minister.

The Terms and Conditions of the program were set to expire on March 31, 2008 but were extended for one year under the terms of the Transfer Payment Policy in effect at the time. A summative evaluation of the program is required to renew the program for another five-year period.

1.1.1 Budget

Over the six-year period of the evaluation (i.e., 2003/4 to 2008/9), a total of $173,700 was approved through the FC RPIA Program. (Note: $5,000 has been approved for spending in 2009/10.)  This represents an average of $28,950 per year allocated for projects. The breakdown of spending by year is provided in Table 1.

Table 1: 
FC RPIA Approved Funding
Fiscal Year Amount
2003/4 $ 25,000
2004/5 $ 73,700
2005/6 $ 14,500
2006/7 $ 0
2007/8 $ 10,000
2008/9 $ 50,500
Total $ 173,700

The RPIA funding was allocated to a total of eleven projects that primarily supported policy research and conferences (5 projects in each of these areas). Note that some of these projects were multi-year. See Appendix A for a full list of RPIA-approved projects during the six-year evaluation period (2003/4 to 2008/9).

2.0 Evaluation Methodology

2.1 Evaluation Issues and Questions

This evaluation was undertaken to respond to the evaluation issues and questions, as articulated in the evaluation matrix developed for the evaluation, and as included in the Memorandum of Understanding between TBS/FC and GCS.

The evaluation questions examined are included below in Table 2. The complete evaluation matrix is presented in Appendix B. Appendix C contains the logic model for the RPIA Program.

Table 2 :
Evaluation Questions
Evaluation Issue Evaluation Question
Relevance Is the RPIA meeting the needs of programs within FC?
Success Has the program achieved the intermediate outcomes identified in the logic model?
Cost-effectiveness What ‘leverage’ did the program generate (i.e., proportion of project funding provided by other organizations)?

2.2 Methodology

The evaluation methodology was designed to ensure that the cost of the evaluation was commensurate with the relatively small amount of funding provided through this program. As a result, the evaluation integrated the use of only two lines of evidence:  file review and key stakeholder interviews.

Complementary research methods were used as a means to enhance the reliability and validity of information and data to be collected. In particular, individuals interviewed for the evaluation were those associated with the projects selected for the file review.  

2.2.1 File Reviews

Selection of the files to be included in the file review was driven by the dollar value of the files. In particular, GCS attempted to select those files that accounted for the largest proportion of the money spent by the program during the evaluation period. However, selection of the files was influenced slightly by the availability of file documentation. One file which would have been included in the top five dollar value files was for an amount of $12,500. Because this project just took place in the last quarter of 2008, not all project documentation would have been in place. Therefore, the project was replaced by the next largest project, which was valued at $10,000.

Five projects were selected for inclusion in the file review. The value of these projects amounted to $133,000, or approximately 77 percent of total program spending over the five-year evaluation period. These five projects also corresponded to four different recipient organizations, and included project spending in five of the six years included in the evaluation. Therefore, these projects provide a reasonable sample from the existing FC projects under RPIA, since the total sample of these selected FC projects represent the majority of the funding.

A file review template was created to gather the relevant information from the project files. Missing or additional information needed was gathered through the interviews conducted with key stakeholders.

2.2.2 Stakeholder Interviews

A total of four interviews were conducted with representatives of the projects selected for the file review1. Interviews were conducted in person2. Interviewees were contacted in advance of the interview to schedule an appropriate time; and all interviewees received the interview guide in advance of the interview (see Appendix D for the interview guide).

The results of interviews were summarized in a template and then analyzed according to evaluation questions and indicators.

2.3 Limitations of Methodology

As with any evaluation, there are limitations to the evaluation methodologies. Firstly, the interviews and file review did not include all projects that received funding through the FC RPIA, and cannot be considered to be statistically representative of all program spending. However, as previously mentioned, efforts were made to select those projects that accounted for the largest percentage of program spending over the evaluation period. GCS considers the files selected to be a good sample, and believe they represent FC projects under the RPIA. Results are also based on interviews with four individuals, who represented five RPIA projects. While the number of interviewees is limited, they represent a very large proportion of total project spending (as discussed in section 2.2.1).

A second limitation of the evaluation relates to a lack of quantitative information. The interviews and file review conducted for the evaluation are qualitative sources of information. Little quantitative information was available about the program, particularly with regards to benefits or outcomes of the Program.

3.0 Evaluation Findings

This section of the report presents a summary of the evaluation findings, which are organized by evaluation question.

3.1 Program Relevance

Conclusion: RPIA projects are aligned with program and departmental objectives. The Program is meeting needs of sectors by providing funding for projects that otherwise may not have been possible. Some sectors would not have had access to the project benefits if the funding had not been provided.

Findings:

RPIA projects are aligned with Program and departmental objectives. According to the RPIA Program terms and conditions, program objectives are:

  • to improve, advance and promote fiscal policies and programs by supporting research initiatives that contribute to the advancement of fiscal, economic, social and financial sector knowledge as it relates to the financial affaires of Canada; and
  • to complement and enhance the Departmental agenda relative to the fiscal, economic, social and financial responsibilities of the Department in managing the financial affaires of Canada3.

FC’s strategic objective, as articulated in its Report on Plans and Priorities, 2008-2009, is:

  • to create a fiscal, economic, social, and global advantage for Canada by providing appropriate policies and sound advice with respect to economic, social, and financial conditions and to the government's overall agenda4.

RPIA projects reviewed were related to conducting research and supporting conferences on topics relevant to the objectives of FC and the Program. Topics examined through the RPIA projects reviewed included:

  • the evolution of the financial services sector and the implications for Canadian providers and users of financial services, financial markets, policy makers, regulators and deposit insurers;
  • the improvement, advancement and promotion of sound corporate governance practices, consistent with the objective of the Government of Canada, to foster investor confidence in Canada's capital markets;
  • the advancement of knowledge regarding the challenges facing Canada in the area of private pensions; and
  • the development of private sector expertise in tax policy issues, which in turn contributes to more informed public policy discussions.

As well, RPIA project documentation noted that the Program supported the development of intellectual capacity by encouraging more academic economists to work on topics that are relevant to policy issues of interest to FC sectors.

Interviewees also believe that RPIA projects are contributing to Program and departmental objectives. For example, one interviewee noted that the department’s PAA calls for fair, effective tax policy. This interviewee stated that the RPIA feeds into this objective since it helps develop fair and effective policy. As well, it encourages work in the tax policy area and increases the importance of tax policy in the research community. Another interviewee noted that the RPIA contributed to the support and dissemination of research initiatives in the financial sector, an area in which further research was needed to enrich public policy assessment.

The RPIA Program is meeting the needs of FC sectors by providing funding for projects that may not otherwise have been possible. Interviewees noted that there were no in-house resources available to conduct the work done through the RPIA projects. As well, generally, interviewees said that the RPIA project activities would not have been carried-out by recipient organizations without the provided funding.

In addition, FC would not have realized the same benefits had they not provided the funding. One interviewee suggested that if funding did not come from FC for tax policy research activities, then organizations, such as the CD Howe Institute would not undertake as much research or activities in this area. As well, according to this interviewee, funding provided through the RPIA for these research projects allows FC to influence the research agenda. Funding provided by FC encourages researchers to get involved in tax policy research, which they would not otherwise do. As well, in the absence of FC funding, projects would probably have been done on a smaller scale, with lower attendance and participation by others.

3.2 Program Success

Conclusion: Interviewees believe that the RPIA program has enhanced policies or programs, helped to increase knowledge, and helped to build capacity. There is a lack of evidence on file regarding the benefits of the projects.

Findings:

Interviewees believe that the RPIA program has contributed towards the outcomes of the RPIA Program (i.e., enhancing policies or programs, increasing knowledge of emerging policy issues, and building capacity for analysis).

According to interviewees, the RPIA Program has helped to enhance policies or programs for which their sector is responsible. One interviewee noted that the RPIA project allows FC to offer better policy advice. This interviewee noted that FC employees involved in policy development participated in the conference funded by the RPIA Program, benefited from the discussions that took place, and the department has access to papers produced for the conference.

Another interviewee noted that the Financial Sector Policy Branch is specifically responsible for developing policies, legislation and regulation to enhance the efficiency, safety and stability of the financial sector. The RPIA program supported themes for research that were suggested by contributing parties, including this Branch, and there were discussions on the usefulness of the research proposals before funding decisions were made in order to ensure the applicability of the research.

Interviewees also agreed that the RPIA Program has helped to increase knowledge of emerging policy issues. As mentioned previously, one interviewee noted the influence FC is able to exert, as a result of providing funding, on the type of research and activities being undertaken. As a result, the RPIA is able to increase knowledge in emerging policy issues of interest to FC. A second interviewee provided the example of a project funded through the RPIA that provided FC with research identifying structural issues in existence in the regulations for pension and the impacts associated with those issues. The interviewee noted that it is very important for FC to know these things, and having practitioners identify the impacts was very valuable to FC.

Interviewees also agreed that the RPIA Program has helped to build capacity for analysis. This was done primarily through research projects funded through the RPIA. As mentioned previously, by providing funding, FC is able to influence to some extent the research that is being conducted, to ensure it meets FC needs. This, then, increases FC’s capacity to produce thorough and relevant analyses. As well, as noted by one interviewee, RPIA funded projects increase the amount of information available that can be used in analyses. Finally, it was noted that the RPIA Program increased the capacity of researchers who were funded through the program, by contributing to doctoral theses and academic papers relevant to financial sector policy interests.

There is a lack of evidence on file regarding the benefits of the projects. For the most part, the only information available from the file review, with regards to benefits of the projects, was information about anticipated benefits, as opposed to actual benefits realized. These anticipated benefits came from the submission/recommendation form on the file. The RPIA Program does not have a requirement to provide any kind of post-project summary for the project file. In one case, information was available regarding actual project benefits to FC. However, for the majority for projects, the actual benefits of these projects to FC are not known to the RPIA program staff. In addition, while some project reports were gathered during the course of the evaluation, they often did not provide information about the benefits of the project to FC.

Recommendation #1:  The program area should implement a mechanism to ensure that the impact of the program can be measured with respect to actual benefits, rather than anticipated benefits.  For example, a post-project report could be a requirement for all RPIA projects. This could be as simple as a one-page template that is filled out for each project.

3.3 Program Cost-Effectiveness and Alternatives

Conclusion: The RPIA program appears to be cost-effective. Interviewees believe that alternative methods of undertaking the projects (e.g., contracting, in house) would be either too costly or would not be feasible due to resource issues. In most cases, FC provided a portion of the funding to carry-out projects. Few program improvements were identified, although better documentation/records management would be beneficial.

Findings:

Interviewees believe that that RPIA Program is cost-effective. While interviewees were not able to establish a numeric return on investment, all indicated that the Program is cost-effective.

There appear to be few options for alternatives to the Program. When asked if there are alternative means to achieve the objectives of the RPIA Program, interviewees suggested that doing the work in-house would be difficult due to resource constraints (both human and financial); and contracting the work, if it would even be possible, would be much more expensive. According to one interviewee, at the time of the RPIA’s introduction, “the Department examined ad hoc contribution agreements and service contracts, and contributions, which were determined to either lack the appropriate legal framework or to be less efficient than a contribution program.”

Cost-effectiveness was enhanced by the fact that FC typically provided only a portion of the funding to carry-out projects. The Department partnered with other organizations to undertake the projects, thus leveraging other resources. For the files reviewed, FC contributed anywhere between 12 percent and 100 percent of the total cost of the project. In 4 of 5 cases, the department contributed less than 50 percent of the total cost of the project.

Because of the nature of the Program and its relatively small size, program efficiency was not seen as a relevant issue on which to focus in this evaluation. However, some information was gathered during interviews that related to efficiency. Interviewees were generally positive in their views on program administration. Most interviewees noted that the application process was straightforward and easy, and approvals were timely.

Few program improvements were identified, although better documentation/records management would be beneficial. The file review undertaken for this evaluation highlighted a couple of issues relating to project documentation. Firstly, not all project-related documents (i.e., the submission/ recommendation forms) were available through the RPIA Program Manager. Secondly, as mentioned above, post-project reports are not required on file, and the actual benefits of projects are not known to RPIA Program staff.

Recommendation #2: Project documentation should be collected and retained more systematically and stored in a central location.

In addition, interviewees identified a couple of areas where they would like to see changes made. One interviewee suggested that DM sign-off can be a cumbersome process and that the authority should be delegated to a lower level. Another interviewee noted that he would like to be able to use the RPIA Program to make contributions to international projects, but that the requirements of the RPIA make this difficult. Specifically, the audit requirements are too onerous, and the funding limit ($25,000) is too low. This interviewee suggested that audit information from other governments should be acceptable, rather than asking international governments to submit to additional audit requirements.

4.0 Summary of Conclusions and Recommendations

Program Relevance

  • RPIA projects are aligned with program and departmental objectives. The Program is meeting needs of sectors by providing funding for projects that otherwise may not have been possible.  Some sectors would not have had access to the project benefits if the funding had not been provided

Program Success

  • Interviewees believe that the RPIA program has enhanced policies or programs, helped to increase knowledge, and helped to build capacity. Evidence for success is largely qualitative. There is a lack of evidence on file regarding the benefits of the projects, as only anticipated benefits are on file.

Recommendation #1:  The program area should implement a mechanism to ensure that the impact of the program can be measured with respect to actual benefits, rather than anticipated benefits.  For example, a post-project report could be a requirement for all RPIA projects. This could be as simple as a one-page template that is filled out for each project.

Program Cost-Effectiveness and Alternatives

  • The RPIA program appears to be cost-effective. Interviewees believe that alternative methods of undertaking the projects (e.g., contracting, in house) would be either too costly or would not be feasible due to resource issues. In most cases, FC provided a portion of the funding to carry-out projects. Few program improvements were identified, although better documentation/records management would be beneficial.

Recommendation #2:  Project documentation should be collected and retained more systematically and stored in a central location.

Appendix A - List of RPIA-Approved Projects

Project Sector FC Contribution
Policy Research Financial Sector Policy $ 25,000
2003-2004 TOTAL   $ 25,000
Policy Research Financial Sector Policy $ 25,000
Conference - Canadian Public
 Economics Group
Tax Policy $ 4,500
Departmental History Project DDCC $ 25,000
Research Project on Institutional
 Investor Activism and Market Confidence
Financial Sector Policy $ 10,000
Project (untitled) EFP $ 4,200
Institute of Public Administration
 of Canada National Conference
Deputy Minister's Office $ 5,000
2004-2005 TOTAL   $ 73,700
Policy Research Financial Sector Policy $ 10,000
Conference - Canadian
  Public Economics Group
Tax Policy $ 4,500
2005-2006 TOTAL   $ 14,500
No projects funded    
2006-2007 TOTAL   $   -
Conference Financial Sector Policy $ 5,000
Pension Research Series Financial Sector Policy $ 5,000
2007-2008 TOTAL   $ 10,000
Pension Research Series Financial Sector Policy $ 5,000
Policy Conference Tax Policy $ 25,000
Conference - Canadian Public Economics Group Tax Policy $ 3,000
Queen's International Institute on Social Policy Tax Policy / Federal-Provincial $ 5,000
Policy Conference on Sovereign Wealth Funds Financial Sector Policy $ 12,500
2008-2009 TOTAL   $ 50,500
TOTAL TBS RPIA PAYMENTS TO DATE $ 173,700

Appendix B - Evaluation Matrix

Issue Question Indicator Data Source / Methodology
Relevance Is the RPIA meeting the needs of programs within FC? internal stakeholder opinion internal stakeholder interviews
Success Has the program achieved the intermediate outcomes identified in the logic model? internal stakeholder opinion internal stakeholder interviews
achievements of 50% sample of contribution agreements (agreements with largest dollar values) file review
internal stakeholder interviews
Cost-effectiveness What ‘leverage’ did the program generate (i.e., proportion of project funding provided by other organizations)? value of financial and ‘in kind’ investments of 50% sample of contribution recipients (agreements with largest dollar values) file review
internal stakeholder interviews

Appendix C - FC RPIA Logic Model

FC RPIA Logic Model

Appendix D - Interview Guide

Evaluation of the Research and Policy Initiative Assistance Contributions Programs (RPIA)

Interview Guide for Finance Canada (FC)

Government Consulting Services (GCS) has been engaged by Treasury Board Secretariat/ Finance Canada (TBS/FC) to conduct evaluations of the Research and Policy Initiative Assistance Contributions Programs (RPIAs). The purpose of these evaluations is to examine the relevance, success, and cost-effectiveness of the programs.

As part of the evaluations, GCS is conducting interviews with key internal stakeholders involved in the programs. The goal of the interviews is to gain a better understanding of the programs, collect information to assess their success and identify possible improvements.

The following questions will serve as a guide for our interview. Please note that the responses you provide will not be attributed to you in the evaluation report (only aggregate information will be released) or in any documentation provided to the responsible department.

Background

1. Can you please briefly describe your role and involvement with the RPIA Program?

Relevance

2. What are the objectives of the RPIA Program in Finance Canada (FC)?

3. Is the RPIA Program meeting the needs of FC and of your sector? Please explain.

4. Has the RPIA program contributed to the achievement of FC strategic objectives and government wide priorities? If so, in what way?

5. To what extent would organizations receiving contributions through the RPIA Program have been able to carry out the activities funded if they had not obtained a RPIA Program contribution?
a. What would have been the impact on FC if funding had not been provided?
   (i.e., would FC still have had access to the benefits/outcomes of the activities?)

Success

6. Has the RPIA Program contributed to enhancing policies and programs for which your sector is responsible?  If yes, please provide specific examples (e.g., research findings and use of these findings; post-conference opinions recorded and used, etc.) If no, why not?

7. In your sector, has the RPIA Program contributed to increasing knowledge of emerging policy issues of interest to FC?  If yes, please provide specific examples (e.g., research findings, post-conference opinions recorded and used, etc.)  If no, why not?

8. Has the RPIA Program contributed to increasing the capacity of your sector to produce more thorough and relevant analyses?  If yes, please explain and provide specific examples. If no, why not?

Cost-Effectiveness/Alternatives

9. Is the RPIA Program cost-effective?

10. Are there alternative means to achieve the objectives of the RPIA Program?  If yes, what are these alternatives and would they be more or less cost-effective than the existing RPIA Program?

11. Do you have any suggestions for improving the RPIA Program? (i.e., to make it more efficient or cost-effective) 

Other

12. Has the RPIA Program had any impacts (either positive or negative) that were unintended? Please explain.

13. Do you have any other comments you would like to make, regarding the RPIA Program in FC, which may be relevant to our evaluation?

Thank you very much for participation in our evaluation study.

Management Response and Action Plan in Response to Evaluation Recommendations for the Research and Policy Initiatives Assistance Program
Recommendation Management Response Planned Action Lead Target Date Comments
A mechanism should be implemented to ensure that the impact of the program can be measured with respect to actual benefits, rather than anticipated benefits.   The Evaluation recommendations will be implemented. The contribution submission and approval process for proposed RPIA contributions will be amended to ensure that the impact of the program can be measured with respect to actual benefits, rather than anticipated benefits. . Financial Management Division, Corporate Services Branch July 31st, 2009 Implemented.
Project documentation should be collected and retained more systematically in a central location. The Evaluation recommendations will be implemented. The contribution submission and approval process for proposed RPIA contributions will be amended to ensure the project documentation is collected and retained more systematically and stored in a central location Financial Management Division, Corporate Services Branch July 31st, 2009 Implemented.

1One interviewee was the representative for two projects.

2Due to scheduling difficulties, one interviewee provided written responses.

3Finance Canada. Terms and Conditions for the RPIA Program.

4Finance Canada. Report on Plans and Priorities, 2007-2008.

5Note that the RPIA is a small ad hoc program with limited spending (maximum of $25K per year paid to a single recipient and up to $100K per recipient over a five year period in exceptional cases).