# Archived - Evaluation of the Federal-Provincial Relations Division

Archived information

Archived information is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Final Report

Prepared by
Internal Audit and Evaluation

Approved by the Deputy Minister of Finance on the recommendation of
Audit and Evaluation Committee on January 4, 2012

Executive Summary

1. Background

4. Evaluation Findings

Appendix A: Management Response and Action Plan

Appendix B: References

## 2.Evaluation Objective and Scope

The objective of the evaluation was to assess the relevance and performance (effectiveness, efficiency and economy) of FPRD, with emphasis on its two key functions: (1) the administration of the transfer payment programs and (2) the policy research, analysis and advice. The scope of the evaluation covered about 3 million in annual direct program spending and focused primarily on FPRD’s operations between 2005 and 2010. ## 3. Methodology ### 3.1 Evaluation Approach The methodological approach for this evaluation was designed to address two key evaluation issues: (1) relevance, which is an assessment of the continued appropriateness of the current approach to the administration and the policy function related to the transfer payment programs, including alignment and consistency with the government’s priorities, and roles and responsibilities; and (2) performance, which is an assessment of the effectiveness, efficiency and economy of FPRD’s functions. FPRD devotes over two-thirds of its resources to policy research, analysis and advice in support of ongoing development and renewal of the transfer payment programs. Thus, an evaluation of the policy function is an essential component of this evaluation. Evaluations of policy functions have rarely been conducted in the Canadian Government or in other governments. This is primarily because of the methodological challenges in establishing a causal link between policy advice and policy decisions. Policy decisions may be informed or influenced by research, analysis and advice provided by policy functions, but these decisions ultimately rest with elected officials. This presents an attribution challenge for an evaluation whereby policy decisions cannot be directly attributed to the policy function. To overcome this challenge to the extent possible, Internal Audit and Evaluation conducted a thorough review of existing international and domestic practices, and engaged external experts to develop an approach to evaluating the policy function that would yield informative and useful results.[4] Under this approach, the evaluation examined the quality of FPRD’s policy function and the appropriateness and adequacy of its processes and resources. The premise for this approach is that if government policy makers have access to high-quality, timely and relevant policy advice for their decision making, then it would be reasonable to suggest that the policy function has contributed, to the extent possible, to the achievement of Departmental outcomes. The quality assessment of the policy function was based on a set of criteria for high-quality policy advice identified by Schacter.[5] The evaluation approach also incorporated elements of organizational assessment inspired by the Balanced Scorecard. In particular, the evaluation assessed FPRD’s relations with its clients and stakeholders, the appropriateness of its organizational structure and internal processes, and the level of its financial and human resources. This aspect of the methodology was based on the generally accepted premise that for an organization to achieve its goals cost-effectively, the needs of clients and stakeholders should be well-understood, and employees should have the required skills and resources to implement appropriate processes to meet client and stakeholder needs. Internal Audit and Evaluation calibrated the evaluation approach and level of effort based on the risks associated with the different components of the policy advice function. For example, due to the substantial financial value of the transfer payments, FPRD had already undergone a number of audits and studies. The evaluation leveraged the existing oversight work to avoid duplication. The evaluation relied on Internal Audit and Evaluation’s recent internal audit of the management of transfer payments, along with relevant annual work done by the Office of the Auditor General of Canada and external consultants, to reduce the evaluation work in areas already assessed. ### 3.2 Evaluation Issues Table 2 presents the evaluation issues and the relevant data sources. To maximize the level of rigour and to ensure that the conclusions are supported by multiple lines of evidence, the evaluation used the following four lines of evidence: (1) document and literature review; (2) interviews of key informants (officials from the Department of Finance Canada, representatives from the provinces and territories, and experts in fiscal federalism); (3) audits and other studies; and (4) a service survey. Table 2. Evaluation Issues and Lines of Evidence Note: “X” indicates that this line of evidence was used. Evaluation Issues Line of Evidence Document and literature review Interviews Audits and other studies Service survey Relevance 1a. Does the current approach to administering transfer payments remain appropriate? X X 1b. Is the policy analysis function of FPRD relevant? X X Performance: Achievement of Expected Outcomes 2a. Are the payments delivered accurately and on time? Is the existing transfer payments delivery system efficient and economical? Are there any concerns related to the delivery process or the delivery model (e.g., long-term viability)? X X X X 2b. Are there alternative payment systems that would be better designed for a more efficient distribution of funds? X X 2c. Are the formulas, simulations, and the data used to calculate each payment an accurate representation of the objective of the payment? X X X 2d. Is FPRD’s policy analysis high-quality, timely, efficient, economic and effective? X X X 2e. Would an alternative organization of the policy analysis function of FPRD be more efficient and effective? Are the organizational structure, processes, resources and tools appropriate for an effective and efficient administration of transfer payment programs? X X X ### 3.3 Data Collectionand Analysis The bulk of the data utilized in this evaluation came from document and literature review and interviews. These were supplemented by a service survey that provided information on the relative importance of FPRD’s outputs and the level of satisfaction of staff and senior management with these outputs. FPRD’s financial data for the period 2005–10 were also used in assessing efficiency and economy of its operations. #### 3.3.1 Data collection methods Document and literature review This line of evidence enabled the evaluators to better understand FPRD’s activities and the context in which it operates. Documents included Reports on Plans and Priorities, Departmental Performance Reports; backgrounders, financial and other budget-related documents; various literature and commentaries on or about various transfer payments; domestic and international literature on fiscal federalism; the proceedings and recommendations of the Expert Panel on Equalization and Territorial Formula Financing; and the proceedings and recommendations of the Advisory Panel on Fiscal Imbalance. Key informant interviews The main purpose of this line of evidence was to collect information on the extent to which FPRD was able to achieve its objectives and to deliver its outputs efficiently and economically. These interviews were conducted between October 2010 and April 2011. Different questionnaires were administered to each group of key informants. Table 3 shows the categories and the number of key informants interviewed. Table 3. Interviews by Category and Number of Key Informants Category Number Department of Finance Canada officials 13 Provincial and territorial representatives 11 Academia and other experts 7 Total 31 Audits and other studies Relevant information was reviewed and extracted from four audits and studies: the Internal Audit and Evaluation Audit of the Management of Transfer Payments to Provinces and TerritoriesFinal Report(2010), a review of internal control documentation over the transfer payment process conducted by consulting firm Ernst & Young (2010), the audit of the Equalization program in the 1997 April Report of the Auditor General of Canada, and the study of federal transfers to provinces and territories in the 2008 December Report of the Auditor General of Canada. Service survey A survey was conducted to find out how FPRD’s staff and senior management assessed FPRD’s policy outputs using the quality assessment criteria identified in subsection 3.1. The survey was used on an experimental basis, but was intended to give an assessment of the quality and utility of FPRD’s policy function from the perspective of the staff and management at the Department of Finance Canada. In total, there were twelve responses. The results of this survey were used in conjunction with the information collected through interviews to develop an overall assessment of the quality of FPRD’s policy function. #### 3.3.2 Data analysis The data collected from various sources were analyzed using the following approach. The evaluation team analyzed the findings from interviews for thematic content. Themes identified by two primary evaluators were compared and then verified by other evaluators within Internal Audit and Evaluation. Any difference in interpretation of interview findings was resolved through deliberation among the evaluators. Information collected through various lines of evidence was compared to confirm key findings, which in turn led to the formulation of conclusions and recommendations. ### 3.4 Methodology Limitations The limitations of the methodology relate to the non-random sample for key informant interviews, the small sample of interviewees, the omission of administration of transfer payment programs from the service survey, and a lack of comparable benchmark data for determining relative economy and efficiency of FPRD activities. In summary: • The evaluation drew samples from three different populations: Department of Finance officials, provincial and territorial representatives, and external experts in the area of fiscal federalism. In selecting provincial and territorial interviewees, the target group was limited to officials who had been interacting actively with FPRD as representatives of their respective provinces and territories. The drawing of a sample with desirable sampling characteristics was made difficult by the small size of the target group. Furthermore, the evaluation intended to interview at least one representative from each province and territory. Although all provinces and territories were contacted for interviews, a few chose not to participate. This lack of participation, combined with the province-specific responses of most participants, resulted in a limited commonality in responses (although some broad common themes were identified). • It was also challenging to find external interviewees (provincial and territorial or experts) who had, in addition to subject-matter expertise, an in-depth knowledge of the activities and outputs of FPRD, and its interactions with provinces and territories. • During the evaluation, it was brought to the evaluators’ attention that the administration of transfer payments should have been included as one of the divisional outputs in the service survey. Other lines of evidence were used to compensate for this deficiency. • Finally, a thorough assessment of the economy and efficiency of the key activities of FPRD was made difficult by the lack of comparable benchmark data. FPRD activities are unique in the Government of Canada as FPRD is responsible for managing all aspects, from conception to design to consultation and administration, of the federal government’s major transfer payment programs to provinces and territories. The administration also requires implementation of complex formulas. To compensate for the lack of benchmark data, the FPRD financial data was collected from 2005 onwards so that year-over-year comparisons could be made. Notwithstanding the limitations stated above, the conclusions of this evaluation were based on multiple lines of evidence and the weight of the evidence with respect to the key issues was strong enough to establish reasonably reliable conclusions and recommendations. ## 4. Evaluation Findings ### 4.1 Relevance of theAdministration and the Policy Function Key Findings The evidence suggests that the current approach to the administration of transfer payments and the organization of the policy function within FPRD is appropriate. The approach aligns well with the federal government’s priorities and roles and responsibilities. Overall, the current approach is preferable to other alternatives. In this section, the evaluation examines whether the current approach to the administration of the transfer payment programs and the conduct of the policy function within FPRD continue to be appropriate and aligned with the federal government’s priorities, as well as its roles and responsibilities. Under the current approach, FPRD manages the design, development, renewal, and administration of the transfer payment programs. Within this context, this section also explores the viability of alternative approaches that may be considered more effective and efficient. #### 4.1.1Continued need for and appropriateness of the current approach and the policy function The authority to administer the transfer payment programs by the Department of Finance Canada is derived from the Federal-Provincial Fiscal Arrangements Act and various other Acts. The Minister of Finance is responsible for the provisions of these Acts. FPRD supports the Minister in exercising his responsibilities. Although the federal government is responsible for the transfer payment programs, strong emphasis is placed on intergovernmental and public consultations to reach decisions on intergovernmental transfers and the current formulas for the major transfer payment programs reflect this reality. For example, consultations that mostly affected the Canada Health Transfer formula include the 2000 First Ministers’ meeting on Health Renewal and Early Childhood Development; the 2002 Royal Commission on the Future of Health Care; the 2002 Standing Senate Committee on Social Affairs, Science and Technology’s Report on The Health of Canadians – the Federal Role; the 2003 First Ministers’ Accord on Health Care Renewal; the 2004 10-Year Plan to Strengthen Health Care, and the Government’s 2007 Budget commitments. In its 2006 Speech from the Throne, the Government of Canada committed to restoring fiscal balance. Budget 2007 sought to restore fiscal balance by adding more than39 billion in new funding and revising the formulas for the major transfer payment programs, reflecting, to a large extent, the implementation of the 2006 recommendations of the Expert Panel on Equalization and Territorial Formula Financing.

Transfer payments from the federal government continue to provide important financial support to the provinces and territories, allowing them to deliver various social programs to Canadians. FPRD’s activities in administering these payments, and further developing the transfer payment programs, are aligned with federal government priorities. Furthermore, the majority of the interviewees from all categories indicated that the current approach to administering the transfer payment programs, which emphasizes intergovernmental and public consultations, was appropriate and consistent with the federal government’s roles and responsibilities.

#### 4.1.2 Feasibility of alternative approaches

While the ultimate responsibility for the transfer payment programs resides with the federal government, intergovernmental consultations, through a number of formally structured committees, play a pivotal role in the design of the transfer payment programs.

A review of relevant literature led the evaluators to focus on Shah and Blöchliger and Charbit, who identify three major types of institutional arrangements for intergovernmental transfers: (1) central/national agency model, (2) independent agency model, and (3) intergovernmental forum model. The central/national agency model, where intergovernmental transfers are managed by the central government, is often criticized for having a natural propensity for the central (federal) government to be heavily involved in sub-central (provincial) decision making, thus potentially reducing the benefits of decentralization. In contrast, economists Blöchliger and Charbit from the Organisation for Economic Co-operation and Development[6] point out that an independent agency model, which would assign the responsibility of intergovernmental transfers to a separate independent body, could lack the necessary flexibility in reacting to legitimate sub-central (provincial) needs, and the independent agency model is prone to greater complexity in its design of intergovernmental transfers.

According to World Bank economist Shah,[7] intergovernmental forums offer the best alternative to both the central/national government agency model and an independent agency model, since intergovernmental consultations allow for all relevant stakeholders to be involved in decisions on transfer programs.

The Expert Panel on Equalization and Territorial Formula Financing also examined alternative models for the administration of Equalization Payments. The Expert Panel specifically assessed whether the federal government should establish a permanent, independent body to provide ongoing advice on Equalization and Territorial Formula Financing, and conduct periodic reviews of provincial disparities. The Expert Panel examined the views of a large number of experts, including provincial and territorial representatives. In the consultation process, the majority of the participants rejected the idea of replacing the current intergovernmental process with an arms-length independent agency. The majority of the provinces also contended that “accountability for the program must continue to reside with the federal government rather than being transferred to a body that is not directly accountable to Canadians”.[8] The Expert Panel also noted that independent agencies are prone to increased complexity, and concluded that there was no merit in replacing the current approach to the management of Equalization and Territorial Formula Financing payments.

In the course of this evaluation, a minority of interviewees suggested that establishing an organization with an arms-length relationship with the government to administer the transfer payment programs would be more appropriate for Canada. Yet, the majority argued that administering and managing the transfer payment programs required a high degree of interaction between the government and the administrator. Accordingly, they stated that FPRD, as part of the Department of Finance Canada, was in a good position to facilitate such interactions. Furthermore, given the characteristics and requirements of Canada, the existing structure and arrangement was argued to be appropriate, and no radical change to the existing structure was found to be necessary.

In addition, most of the Department of Finance Canada interviewees indicated that the policy function was an integral part of FPRD’s activities, without which FPRD would not be able to carry out its mandate. The administration of transfer payments and the policy function were considered complementary, and any alteration of the existing structure was suggested to be counterproductive.

Although no alternative approach was advocated, the interviewees from outside the Department of Finance Canada suggested improvements to the existing approach, in particular to the processes surrounding the interactions between the provinces/territories and FPRD. This is discussed further in section 4.2.1.2.

### 4.2 Performance

In this section, the evaluation examines the extent to which FPRD has been effective in achieving its objectives and delivering its outputs efficiently and economically. It starts with an assessment of the extent to which FPRD has been able to achieve its objective of accurate and timely calculations and disbursements of transfer payments. An assessment of the appropriateness of the implementation of transfer payments formulas follows. Next, the quality and effectiveness of the policy research, analysis and advice produced by FPRD is examined. Finally, financial and human resources data are reviewed to assess efficiency and economy.

#### 4.2.1 Effectiveness

Key Findings

All evidence indicates that transfer payments have been accurate and timely. No concern was raised with respect to the payment processes.

The current methodologies are, in general, appropriate for the achievement of the programs’ objectives. However, some issues with respect to the transparency of methodologies were raised that require additional clarification and attention.

Overall, the policy function is generally perceived as high quality and effective. However, obtaining further feedback from the provinces and territories, as well as external sources, would improve and strengthen the quality of the research and analysis conducted by FPRD.

##### 4.2.1.1 Accuracy and timeliness in the calculation and delivery of transfer payments

Federal Acts and Regulations provide the basis for the specific processes that must be followed for the calculation and disbursements of transfer payments.[9] In this section, the evaluation examines the extent to which those processes were followed, and to what extent the calculation of the entitlements and delivery of the payments were accurate and timely.

The FPRD Business Process Narrative[10] outlines a comprehensive procedure aimed at “complete and accurate processing, disbursing, and recording of transfer payments in departmental financial statements.” The document also describes the roles of key players in both FPRD and the Financial Management Directorate of the Department of Finance Canada in ensuring that calculations of transfer payment entitlements comply with the various Acts, Regulations and subsequent amendments and that the data sources are appropriate.

The accuracy and timeliness of the calculation and delivery of transfer payments has been the subject of a number of internal and external audits and studies in recent years. For example, in 2010, Internal Audit and Evaluation conducted an audit of the management of transfer payments to provinces and territories. As part of this audit, a sample of 43 transactions that included payments for each of the four main transfer payments to all provinces and territories (valued at approximately $3.3 billion) was tested and analyzed. The audit found, among other things, that transactions were recorded accurately, and disbursed on time.[11] Ernst & Young, a consulting firm, conducted a brief review in 2010 of the existing internal controls of the transfer payment process. The main objective of the review was to test the key controls to determine their effectiveness. The consultants concluded that although certain controls could be improved, overall, the key controls were in place and had been internally validated. Interviews with the departmental officials included a question specifically aimed at finding out the extent to which the processes and steps, as described in the above-mentioned Business Process Narrative, were being consistently followed. Interviewees who were directly involved with payments and estimates attested to consistent adherence to the steps and processes described in the Narrative. The majority of the provincial and territorial interviewees expressed their satisfaction with the accuracy and timeliness of the delivery of transfer payments. ##### 4.2.1.2 Appropriateness of the implementation of transfer payments formulas FPRD is responsible for implementing the transfer payment formulas. The formulas have evolved into their present forms through decades of intergovernmental and public consultations. The transfer payment methodologies are elaborated principally in the Federal-Provincial Fiscal Arrangements Act and supporting Federal-Provincial Fiscal Arrangements Regulations. The data used to calculate transfer payments are obtained from Statistics Canada, the Canada Revenue Agency and other sources. None of the interviewees expressed any notable concerns with respect to the implementation of Canada Health Transfer and Canada Social Transfer formulas. The majority stated that the methodologies and formulas used were straightforward. Since almost all interviewees focused their responses on the methodology for the Equalization and Territorial Formula Financing Payments, this section primarily discusses the methodologies for these two transfer payments. The majority of interviewees from all categories indicated that the current methodologies were appropriate and contributed to the achievement of the objectives of the transfer payment programs. However, a large number of the provincial and territorial and external interviewees had issues with the way changes were made to the Equalization and Territorial Formula Financing programs without adequate consultations with provinces and territories. The interviewees characterized this as a “lack of transparency.” As expressed by one of the interviewees, “we should be able to understand the changes even if we do not agree with them.” Given the implications that these changes have for provinces and territories, it was suggested that they be part of a more comprehensive consultation process in the development of policies affecting the transfer payment programs. A small number of provincial and territorial interviewees viewed the lack of access to the data, simulations and methodology as a manifestation of the lack of transparency. This limited their ability to determine whether their payments were accurate. Several interviewees from the Department of Finance Canada indicated that they were aware of some of these issues, but pointed out that some data cannot be released, citing confidentiality. These interviewees added that FPRD had already taken some initiatives such as proactive web seminars beginning in early 2011 to explain the details of the methodologies and approaches used in the calculation of transfer payments. They also noted FPRD‘s ongoing readiness to discuss and answer any relevant questions. These incremental steps were acknowledged by the majority of the provincial and territorial interviewees, but they also noted that these Web seminars did not offer opportunities for adequate interactions with participants. The provincial and territorial interviewees highlighted the usefulness of face-to-face meetings, especially when discussing important issues. Some of the experts interviewed also suggested that FPRD and the Department should improve the transparency of the Equalization and Territorial Formula Financing programs. A number of Departmental interviewees stated that the current one-estimate system for calculating Equalization and Transfer Formula Financing entitlements had placed a significant burden on FPRD to get things right with the first calculation. However, it was also noted that the one-estimate system has worked well since its adoption in 2007. Some of the provincial and territorial interviewees expressed a preference for a two-estimate system, which according to them would provide FPRD with more flexibility for correcting errors and making updates. The majority of the evidence examined led to the conclusion that the transfer payments methodologies are generally appropriate and implemented as per their stated objectives. However, a few questions related to the transparency of the management of the Equalization program and Transfer Formula Financing were raised that require further examination. The Expert Panel on Equalization and Territorial Formula Financing had recommended that the Department should improve transparency and communications. In its 2010–11 Report on Plans and Priorities, the Department of Finance Canada committed to enhance the transparency of the federal transfer system. Within that context, the following recommendation is made to address issues related to the transparency of methodologies to the extent possible. Recommendation 1: The Federal-Provincial Relations Division should continue its efforts to work with provinces and territories to clearly identify and address, to the extent possible, their issues with respect to the transparency of Equalization and Transfer Formula Financing methodologies. ##### 4.2.1.3 Quality and effectiveness of the policy function The policy function is a key activity of FPRD and plays an important role in the ongoing improvement and renewal of transfer payment programs. A considerable amount of effort is devoted by FPRD to this function. This part of the report examines the quality and effectiveness of the function. Findings from the key informant interviews were supplemented with a survey that was used to assess the extent to which FPRD’s outputs met the quality criteria identified in subsection 3.1. In FPRD, policy research, analysis and advice originate in two ways: (1) reactive, where the process starts with requests coming from outside FPRD (for example, a province); and (2) proactive, where analysis and research is internally generated with identification of issues, based on the staff’s understanding of the programs. Proactive research and analysis are usually based on consultations with key players and stakeholders. All Departmental interviewees categorized the work of FPRD’s policy function as high quality. FPRD has an internal review process in place that involves staff at various stages of seniority, usually from analyst to Chief, Senior Chief and Director. The review process is meant to provide the appropriate level of quality control to ensure high-quality policy research, analysis and advice. FPRD was perceived by the staff to have been successful in developing good relationships with other branches of the Department, other federal departments and central agencies, the provinces and territories, and to a lesser degree, with academia. In particular, FPRD regularly contacts the Privy Council Office and Intergovernmental Affairs to keep them up-to-date on the latest concerns and news about inter-governmental relations and activities. In addition, FPRD maintains contacts with a number of departments, such as Health Canada and Human Resources and Skills Development Canada, particularly regarding longer-term health and social policy issues. Most interviewees indicated that regular communication with these organizations was crucial for the fulfillment of FPRD’s responsibilities. It was indicated that management understood the importance of these exchanges for FPRD and encouraged the staff to cultivate these relationships. Although FPRD has only limited direct contact with researchers and analysts in other countries, the staff stated that they follow new developments in various countries attentively in order to draw lessons for Canada. The majority of provincial and territorial interviewees found the policy and research-related work of FPRD to be high quality. The work that was presented in the October 2010 Transfers Sub-Committee meeting was given as a good example of high-quality work done by FPRD. However, some interviewees indicated that the quality of the work was not consistent from one topic to another. Quality-related issues that were mentioned in some interviews included the tendency for some research and analytical work to be narrowly focused and to lean strongly toward a specific policy position; and some reports seemed to be in perpetual draft form and comments received did not seem to be incorporated to finalize these reports. Some of the provincial and territorial interviewees indicated that the quality of the FPRD analytical work could improve if FPRD was more willing to take advantage of provincial and territorial expertise in these areas. These interviewees expressed a strong desire to take an active role in developing and aiding research plans aimed at improving the transfer payment programs. It was also suggested that papers and meeting materials be provided at least one week in advance of meetings so that provincial and territorial officials have adequate time to review and develop their position in order to contribute more meaningfully to the discussion. Other suggestions included that FPRD use time between renewals to undertake some long-term research; and that updated organizational charts be posted on the Departmental website to inform provincial and territorial officials of who is responsible for what files to facilitate accessibility. The majority of the provincial and territorial interviewees categorized their relationship with FPRD as excellent. They indicated that the researchers and other divisional staff were always accessible, responsive, accommodating, and willing to share information. The relationship was, however, described as being one-sided (mostly initiated by the provinces and territories). The majority of experts interviewed categorized the nature of their relationships with FPRD’s researchers and staff as ad hoc, and all had a good overall impression of the researchers and analysts. The group was perceived to be technically competent and hardworking. A small number of experts who had closer exposure to the policy research and other analytical work of FPRD rated it as high quality. The majority of the experts indicated that they did not have enough exposure to the policy research and analysis work of FPRD to be able to comment on its quality. Although it was understood that some of the work could be considered confidential, they argued that the absence of sufficient external exposure and scrutiny would have a negative impact on the quality of FPRD’s research and analytical work. They suggested a compromise where the policy research work would benefit from external scrutiny while respecting those sensitivities; they gave examples of other departments, including Statistics Canada and the Canada Revenue Agency, where these sensitivity issues were addressed by establishing external expert advisory committees comprising academics and other experts with an appropriate security clearance. It was suggested that the Department of Finance Canada could organize and participate in regular conferences and workshops, where FPRD could present its approaches both to inform and to receive feedback from the public and experts. The 2006 report of the Expert Panel on Equalization and Territorial Formula Financing recommended, in general, a more inclusive approach as well as various measures for implementing “a constructive process for ongoing discussions among provincial and federal government representatives.” The Expert Panel also emphasized the importance of “ongoing academic research related to Equalization—through a number of existing organizations and research agencies, along with combined support from federal and provincial governments.”[12] The survey results indicated that the majority of the participants are very satisfied with the quality of the policy-related work of FPRD. On a scale ranging between 2 for “very satisfied” and -2 for “very dissatisfied”, the average score was 1.7. The average score was 1.25 for the quality of research and policy papers presented at the federal-provincial/territorial meetings. In conclusion, based on the evidence collected, the policy research, analysis and advice of FPRD are generally perceived as high quality. It was suggested that the quality of FPRD’s work could be enhanced further through greater involvement of expert and provincial and territorial representations. The following recommendation is made in the spirit of continuous improvement. Recommendation 2: The Federal-Provincial Relations Division (FPRD) should explore how its researchers and analysts can receive additional feedback on their research from various external groups, including experts in fiscal federalism. FPRD should also seek additional inputs to its research plans from the provinces and territories. #### 4.2.2 Efficiency and economy This section assesses the extent to which FPRD conducts its activities efficiently and economically. This theme was assessed using financial and human resources data, document review and internal interviews. The evaluation examined FPRD’s resource acquisition and utilization in relation to its program administration and policy-function activities and outputs supporting its expected outcomes. Key Findings The evidence suggests that FPRD performs its key functions with efficiency and economy. FPRD has taken some innovative measures to enhance the efficiency and economy of its operation. FPRD has a staff of 31 full-time equivalents (FTEs), including an administrative assistant and seven executives. FPRD, with a total operating budget of about$3 million (2010–11), manages the design and further development of the federal government’s transfer payment programs that transfer over \$55 billion annually to provinces and territories implementing, in some cases, very complex formulas. The calculation of entitlements and disbursement of the amounts for the transfer payments were found to be very cost-efficient as it involves only a small group (four members) of economists who are viewed as very experienced and competent in their jobs. Over two-thirds of FPRD’s staff is engaged in policy research and policy development, supporting ongoing development and renewal of the transfer payment programs. The remaining FTEs belong to the Strategic Planning Section that supports liaison between the Department of Finance Canada and its provincial and territorial counterparts, including ministerial meetings.

Figure 1 presents FPRD’s actual personnel and operating costs for the period 2004–05 to 2010–11, which shows some variation in costs over time. FPRD provided administrative and secretariat support to the Expert Panel on Equalization and Territorial Formula Financing, commissioned by the government in March 2005. This increased FPRD’s expenses considerably. After the Expert Panel submitted its report in May 2006, FPRD had to internalize a significant part of its work. In addition, in 2008 the Government of Canada and the Government of Nova Scotia appointed a three-member panel to address the outstanding issues relating to the Crown Share Adjustment Payments to Nova Scotia (2008–09). FPRD also helped coordinate Canada-wide consultations related to the management of the Retirement Income Adequacy Process (2008–09 and 2009–10). All these events added to FPRD’s personnel and other operating costs during fiscal years 2005–06, 2007–08 and 2008–09. The changes in FPRD’s workload were reflected in the pattern of operating costs over the period.

Figure 1. FPRD Personnel and Operating Costs for 2004–05 to 2010–11

Virtually all interviewees from the Department of Finance Canada indicated that FPRD activities were conducted with significant economy and efficiency. Interviewees pointed out that although there was a clear division of roles and responsibilities between its various sections, FPRD was often able to reallocate resources to meet changing requirements. For example, in preparation for the upcoming 2013–14 renewal of the transfer payment programs, FPRD has reallocated resources to meet the additional needs within its existing budget, without seeking additional funds.

In recent years, FPRD has also taken a number of measures to cut its operating costs while enhancing productivity and efficiency. Chief among these measures was the development of the FPRD “Wiki” and the increased use of Internet-based conferencing.

The FPRD Wiki is an innovative Internet-based tool designed to preserve corporate knowledge and to enhance productivity through improved communications, information storage and sharing. It is the repository for information pertaining to FPRD and the transfer payment programs. The Wiki allows staff to retrieve corporate knowledge and documents pertaining to transfer payment programs and other aspects of FPRD’s work using a single user-friendly interface. Interviewees described the Wiki as a very useful and productivity-enhancing tool. It was also described as a useful training tool, resulting in lower costs and less personnel time spent on training new employees.

FPRD’s efforts to take advantage of technology for more frequent interactions with the provinces and territories, for example through Internet-based conferencing, have also allowed it to reduce its travel budget. A cost-cutting suggestion offered by the provincial and territorial interviewees was that FPRD could organize meetings of the Fiscal Arrangements Committee and the Transfers Sub-Committee around the same time and in the same vicinity—they stated this would reduce their own costs as well.

Finally, FPRD decreased the use of outside consultants in recent years as an additional cost-cutting measure.

Overall, based on our observations and analysis, FPRD operations are considered efficient and economical. FPRD has taken a number of steps to improve the efficiency and economy of its operations.

## Appendix A:Management Response and Action Plan

Appendix A: Management Response and Action Plan
Recommendation Management Response Planned Action Lead Target Date
1: The Federal-Provincial Relations Division (FPRD) should continue its efforts to work with provinces and territories to clearly identify and address, to the extent possible, their issues with respect to the transparency of Equalization and Territorial Formula Financing methodologies. Agree. FPRD is currently working to better understand provincial and territorial concerns with respect to transparency of its methodologies. FPRD commits to include discussions of transparency issues at Transfers Sub-Committee meetings over the coming twelve months to allow provincial and territorial officials to explain their concerns regarding transparency of the methodologies currently used to determine transfer entitlements. FPRD will then determine actions that are possible to address these concerns. Federal-Provincial Relations Division Q2 2012–13
2: The Federal-Provincial Relations Division (FPRD) should explore how its researchers and analysts can receive additional feedback on their research from various external groups, including experts in fiscal federalism. FPRD should also seek additional inputs to its research plans from provinces and territories.

Agree.

Employees of FPRD closely monitor and report internally on the output of experts on fiscal federalism. The work of these external experts is taken into account when policy options are being considered. However, FPRD is often working in an environment of Budget secrecy ― where elements of the Budget Plan cannot be discussed outside the Department.

FPRD regularly seeks the views of provincial and territorial officials on its research as part of the Transfers Sub-Committee process. As well, provincial and territorial officials are able to provide input into the Transfers Sub-Committee work plan. Ultimately, the program responsibility rests with the federal government.

FPRD commits, over the coming twelve months, to review its engagement with experts in fiscal federalism —to ensure that it is receiving adequate feedback and input on research activities.

In the next twelve months, FPRD will make it a priority to ensure that provincial and territorial officials are provided with material well in advance of meetings and to circulate more work-in-progress reports. FPRD will continue to discuss current research plans with provincial and territorial colleagues and seek their inputs.

Federal-Provincial Relations Division Q2 2012–13

## Appendix B: References

Blöchliger, Hansjörg, and Claire Charbit. 2008. “Fiscal Equalisation.” OECD Journal: Economic Studies,no. 44, 2008/1.

Boadway, Robin, and Ronald Watts. 2000. “Fiscal Federalism in Canada.” Institute of Intergovernmental Relations Working Paper, Queen’s University, Kingston, Ontario.

Canada. Department of Finance Canada. 2010. Audit of the Management of Transfer Payments to Provinces and Territories – Final Report. Ottawa.

Canada. Department of Finance Canada. 2010. 2010–11 Report on Plans and Priorities. Ottawa.

Canada. Office of the Auditor General of Canada. 1997. 1997 April Report of the Auditor General of Canada. Chapter 8 – “Department of Finance – Equalization Program.” Ottawa.

Canada. Office of the Auditor General of Canada. 2008. 2008 December Report of the Auditor General of Canada. Chapter 1 – “A Study of Federal Transfers to Provinces and Territories.”  Ottawa.

The Council of the Federation. Advisory Panel on Fiscal Imbalance. 2006. Reconciling the Irreconcilable: Addressing Canada’s Fiscal Imbalance. Ottawa.

Dafflon, Bernard. 2007. “Fiscal Capacity Equalization in Horizontal Fiscal Equalization Programs.” In Intergovernmental Fiscal Transfers, Principles and Practice, Eds. Robin Boadway and Anwar Shah, 397–424. The World Bank, Washington, D.C.

Expert Panel on Equalization and Territorial Formula Financing. 2006. Achieving a National Purpose:“Putting Equalization Back on Track.” Commissioned by the Department of Finance Canada.

Expert Panel on Equalization and Territorial Formula Financing. 2006. Achieving a National Purpose:Improving Territorial Formula Financing and Strengthening Canada’s Territories.” Commissioned by the Department of Finance Canada.

Reschovsky, Andrew. 2007. “Compensating Local Governments for Differences in Expenditure Needs in a Horizontal Fiscal Equalization Program.” In Intergovernmental Fiscal Transfers, Principles and Practice, Eds. Robin Boadway and Anwar Shah, 397–424. The World Bank, Washington, D.C.

Richards, John. 2005. “A Fine Canadian Compromise: Perspectives on the Report of the Expert Panel on Equalization and Territorial Funding Financing.” In A Fine Canadian Compromise, Eds.Paul Boothe and Francois Vaillancourt, 81–94.

Schacter, Mark. 2006. “The Worth of a Garden”: Performance Measurement and Policy Advice in the Public Service — A Discussion Paper. Commissioned by the Treasury Board of Canada Secretariat. Ottawa.

Shah, Anwar. 2007. “Institutional Arrangements for Intergovernmental Fiscal Transfers and a Framework for Evaluation.” In Intergovernmental Fiscal Transfers, Principles and Practice, Eds. Robin Boadway and Anwar Shah, 293–317. The World Bank, Washington, D.C.

Shah, Anwar. 2007. “Comparative Conclusions on Fiscal Federalism.” In The Practice of Fiscal Federalism: Comparative Perspectives, Ed. Anwar Shah, 370–393. McGill-Queen’s University Press. Montreal & Kingston, London, Ithaca.

Simeon, Richard, and Martin Papillon. 2006. “Canada.” In Distribution of Powers and Responsibilities in Federal Countries, Eds. Akhtar Majeed, Ronald L. Watts, and Douglas M. Brown, 92–122. McGill-Queen’s University Press. Montreal & Kingston, London, Ithaca.

[1] Canada, Department of Finance Canada, 201011 Report on Plans and Priorities

[2] Compiled from the 2009–10 Department of Finance Integrated Business Plan (p. 30), the 2010–11 Report on Plans and Prioritiesand other sources (e.g., the Branch presentation for the orientation of new employees). The compilation was reviewed and accepted by FPRD as part of the approval of the Terms of Reference for this evaluation.

[3] FPRD, October 2011

[4] Internal Audit and Evaluation also participated in the interdepartmental Policy Program Evaluation Working Group, chaired by the Centre of Excellence in Evaluation of the Canadian federal government, in order to develop best practices and obtain feedback on proposed evaluation methodologies.

[5] Mark Schacter proposes the following criteria for assessing the quality of policy advice: Is it timely? Is it based on adequate consultations? Does it articulate a clear purpose? Is it logical, differentiating facts from assumptions and linking conclusions to recommendations? Is it based on sound evidence? Is it balanced, and does it include multiple viewpoints? Does it present viable options for action? Is it relevant to the actual situation, and does it anticipate related developments? Is it well-organized and well-presented to the reader? Mark Schacter, 2006, The Worth of a Garden: Performance Measurement and Policy Advice in the Public Service – A Discussion Paper.

[6] Hansjörg Blöchliger and Claire Charbit, 2008, “Fiscal Equalisation,” OECD Journal: Economic Studies, no. 44, 2008/1,265–286.

[7] Anwar Shah, 2007 “Institutional Arrangements for Intergovernmental Fiscal Transfers and a Framework for Evaluation,” in Intergovernmental Fiscal Transfers, Principles and Practice, eds. Robin Boadway and Anwar Shah,293–317.

[8] Expert Panel on Equalization and Territorial Formula Financing, 2006, Achieving a National Purpose: “Putting Equalization Back on Track.”

[9] The following Acts and Regulations govern the calculations of transfer payments: Federal-Provincial Fiscal Arrangements Act, Nova Scotia and Newfoundland and Labrador Additional Fiscal Equalization Offset Payments Act, Canada-Newfoundland Atlantic Accord Implementation Act, Canada-Nova Scotia Offshore Petroleum Resources Accord Implementation Act, Provincial Subsidies Act, Newfoundland Additional Financial Assistance Act, Federal-Provincial Fiscal Arrangements Regulations, Newfoundland and Labrador Offshore Revenue Fiscal Equalization Offset Payments Regulations, Nova Scotia Offshore Revenue Fiscal Equalization Offset Payments Regulations, Nova Scotia and Newfoundland and Labrador Additional Fiscal Equalization Offset Payments Regulations, Canada Health Transfer, Canada Social Transfer, and Wait Times Reduction Transfer Regulations.

[10] Transfer Payments Statutory Vote, Version 4.0 – Draft Update January 18, 2010.

[11] Canada, Department of Finance Canada, Audit of the Management of Transfer Payments to Provinces and TerritoriesFinal Report.

[12] Expert Panel on Equalization and Territorial Formula Financing, 2006, Achieving a National Purpose: “Putting Equalization Back on Track.”