Archived - Emergency Preparedness & Critical Infrastructure Protection Summative Evaluation Report

Archived information

Archived information is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

May 5, 2008


Table of Contents

Executive Summary

1 Background

1.1 Introduction
1.2 Description of the Initiative
1.3 Evaluation Objectives and Issues
1.4 Methodology

2 Evaluation Findings and Conclusions

2.1 Relevance

2.2 Achievement of Objectives

2.3 Cost-Effectiveness and Alternatives

3 Summary Conclusions

3.1 Relevance
3.2 Achievement of Objectives
3.3 Cost-Effectiveness and Alternatives

4 Appendices

Appendix 1: Document Review Matrix
Appendix 2: Interview Guide


Executive Summary

Background

This report presents the findings, conclusions and recommendations of a summative evaluation of the activities of Finance Canada (FC) within the Emergency Preparedness and Critical Infrastructure Protection (EP & CIP) component of the overall Public Safety and Anti-Terrorism Initiative, for the period 2002/03–2006/07. The evaluation assessed the relevance of FC’s activities; their cost-effectiveness; and, the progress achieved against objectives stated below.

  • To establish a communication framework and processes to address adverse global and domestic events and situations of emergency and/or crisis in the financial sector.
  • To encourage and coordinate financial sector participants in establishing and developing critical infrastructure protection programs and plans.

Relevance

Finance Canada’s role is to serve as the conduit for information from other government departments and agencies and from international and foreign bodies for the benefit of institutions, organizations and associations within the financial sector; and, to represent the financial sector at federal and international bodies meetings and activities.

The department’s long-term objectives and activities are in accord with the overall aims of the Initiative. The objective and activity statements developed during the early stages of the department’s participation tended to describe the approach which was to be taken, rather than the goals to be achieved. Greater clarity and specificity in the objectives and activities would have been helpful, both in carrying out the work and in assessing progress.

There is a continuing need for FC’s participation in the Initiative, although stakeholder interviewees identified a need for an expanded role—FC taking the lead in exercising EP & CIP measures, including tabletop simulations and larger scale exercises to test specific aspects of emergency plans or business continuity plans.

Achievement of Objectives

The objectives as stated are primarily statements of activities that were to take place. In that sense, the objectives were achieved in that:

  • A communication framework has been established for use during crises.
  • The activities undertaken can be assumed to constitute encouragement and coordination.
  • Shorter-term objectives can be inferred from the planned activities, namely:
  • to raise awareness of critical infrastructure, interdependencies and business continuity planning and emergency preparedness in the financial sector
  • to assure effective communications during planning stages and in the event of an emergency or crisis affecting the financial sector

There is evidence that there is more awareness of EP & CIP in the financial sector since the beginning of the Initiative. This is partly due to the activities of FC, partly due to the financial institutions experience of dealing with actual emergencies [during the life of the Initiative], and partly due to more demanding requirements from the Office of the Superintendent of Financial Institutions (influenced by FC).

FC depends primarily on existing mechanisms, and ad hoc events, to ensure communications during the planning stages of EP & CIP preparedness. The evaluation team found no evidence that a communications framework had been established to ensure that the communications needs of all relevant organizations and governments would be addressed and processes or structures established to avoid duplication and ensure consistent messages.

Cost-Effectiveness and Alternatives

The use of existing mechanisms for communications and decision-making ensured that FC’s activities under the Initiative were cost-effective.

1 Background

1.1 Introduction

This report presents the findings, conclusions and recommendations of a summative evaluation of the Emergency Preparedness and Critical Infrastructure Protection initiative, part of the overall Public Safety and Anti-Terrorism Initiative.

1.2 Description of the Initiative

As part of Budget 2001, $190 million in Public Security and Anti-Terrorism (PSAT) funds were provided to twelve federal organizations over five years to enhance Canada’s capacity in emergency preparedness [EP] and critical infrastructure protection [CIP]. This initiative was led by the Office of Critical Infrastructure Protection and Emergency Preparedness (OCIPEP), now Public Safety Canada, in conjunction with eleven other organizations, including the Department of Finance.

The Department of Finance is the lead sector liaison department responsible for the federally regulated financial sector. The long-term established objectives are:

  • Establish a communication framework and processes to address adverse global and domestic events and situations of emergency and/or crisis in the financial sector.
  • Encourage and coordinate financial sector participants in establishing and developing critical infrastructure protection programs and plans.

Under this initiative, $1.4M was allocated to the Department of Finance over 5 years (2002/03–2006/07) to carry out the following activities:

  • Coordinate with Public Safety, the Office of the Superintendent of Financial Institutions (OSFI), the Canada Deposit Insurance Corporation (CDIC), the Bank of Canada (BOC) and Financial Consumer Agency of Canada (FCAC) to raise awareness of critical infrastructure, interdependencies and business continuity planning and emergency preparedness in the financial sector;
  • Work with financial institutions and industry associations to help raise awareness of critical infrastructure, interdependencies and business continuity planning and emergency preparedness in the financial sector, and to encourage financial institutions to undertake the necessary contingency measures;
  • Liaise with Public Safety, the provinces, the U.S. and other foreign governments in regard to developments and advances in these fields; and
  • Coordinate with financial sector agencies, financial institutions, and industry associations on contingency planning procedures and reference documents on these procedures.

The Department assigned these additional activities to the existing staff of the Financial Sector Division.

1.3 Evaluation Objectives and Issues

This summative evaluation was undertaken in response to the TB requirement for evaluation of the PSAT initiative. It addressed the following evaluation issues and questions, for the period from the start of funding to January 2007.

Relevance

  • Was there a clear and relevant vision and set of objectives for activities carried out under the initiative?
  • Was the Department’s contribution to the initiative consistent with government wide priorities relating to emergency preparedness?
  • Is there a continued need for the Department’s contribution?

Achievement of Objectives

  • Were the planned activities implemented and achieved? Did they produce the expected outputs?
  • How well were the overall objectives met?
  • Were there any unexpected outcomes?

Cost-Effectiveness and Alternatives

  • Were there appropriate management and decision-making structures in place to meet the program objectives?
  • Was the achievement of objectives accomplished in a cost effective manner?
  • Is there a need to consider options and/or alternatives to increase cost-effectiveness or efficiency in delivery?

1.4 Methodology

The evaluation utilized two lines of enquiry: a document review and key informant interviews.

The document review built on the review conducted as part of the formative evaluation, and included the documents referenced in Appendix 1.

A total of 19 individuals were interviewed, representing:

  • the Department of Finance
  • the Office of the Superintendent of Financial Institutions
  • the Bank of Canada
  • the Canadian Payments Association
  • the Canadian Bankers Association
  • Public Safety Canada
  • the Ontario Ministry of Finance
  • Emergency Management Ontario
  • the Canada Deposit Insurance Corporation, and
  • the Financial Consumer Agency of Canada

The interview guide, attached as Appendix 2 was used as a checklist to ensure all relevant issues were covered as appropriate with each interviewee.

2 Evaluation Findings and Conclusions

The evaluation questions guided the data collection efforts, but are deemed to be inappropriate as an organizing principle for this report. This section is therefore structured according to the major themes that emerged within each evaluation issue.

2.1 Relevance

2.1.1 Objectives

The Treasury Board Submission to fund the overall PSAT initiative has as one of its aims "a safer, more secure Canada", by bringing about the situation whereby "the health, safety, security and economic well-being of Canada and Canadians are assured". The Financial Sector Policy Branch of Finance Canada has a vision statement "to serve the public interest by developing policies and providing advice to promote the integrity, soundness and competitiveness of the Canadian financial system". This statement covers all activities in the Branch and, as such, applies to Finance Canada’s role in the PSAT initiative. However, there is no evidence of a specific vision statement for that role.

Finance Canada’s long-term objectives and the fourfold set of program activities laid out for its use of PSAT funding accord with the overall aims of that initiative over the five-year period. The objective and activity statements were developed during the early stages of the Department’s participation, and tended to describe the approach which was to be taken, rather than the goals to be achieved. The use of words such as "coordinate", "raise awareness", "work with", "encourage", and "liaise" proved to be less helpful as the initiative matured and became more focussed on results and mileposts. Consequently, greater clarity and specificity in the objectives and activities would have been helpful both in carrying out the work and in assessing progress.

A review of the objectives for the overall PSAT initiative, as well as those of other related federal government activities such as the National Critical Infrastructure Assurance Program, showed that the government-wide priorities and desired outcomes had not changed significantly during the period under examination. Finance Canada’s objectives and activities statements were consistent with these priorities and desired outcomes throughout the period under examination.

Although not specifically addressed in the objectives, FC interprets its role as related only to federally-regulated financial institutions, not the entire financial sector (e.g. provincially-regulated institutions are not considered). Since PSAT is not limited to federally-regulated sectors, there would appear to be a gap that might need to be addressed in a renewed initiative. Otherwise, any revised objectives should specify that they apply only to the federally-regulated financial sector.

Recommendation:

That the objectives for the second phase of PSAT be clearer, more specific and reflect FC’s role and jurisdiction.

2.1.2 Continued Need

2.1.2.1 Existing role

The Department fulfills its role by acting as the conduit for information from other government departments and bodies, and from international and foreign bodies, which can be of use to organizations and associations within the financial sector. The Financial Sector Division plays that role on an on-going basis relative to the financial sector as a whole, not just as it relates to EP & CIP. Given that the Department chose not to add staff to undertake the tasks required under the initiative, and that some activities were already undertaken along the same lines prior to the additional PSAT funding, the issue to be addressed is whether or not FC recognizes a distinct role for itself related to EP and CIP in its operations.

Although there are a variety of players concerned with EP & CIP in the financial sector, there was little evidence of duplication or overlap with respect to the activities of the various players in the financial sector. For example, the Office of the Superintendent of Financial Institutions (OSFI) is responsible for regulating and supervising federally incorporated financial institutions and has the authority to demand actions and reports that FC cannot. Recognizing this division of responsibility, FC worked with OSFI through the Financial Institutions Supervisory Committee (FISC) to ensure that financial institutions became required to report to OSFI on the status of their BCP plans as part of OSFI’s regular supervisory functions. The FISC is one of several coordination mechanisms that bring together all the federal bodies with a role to play in the financial sector, and may identify and address emerging issues in the sector, including EP and CIP issues. At the provincial level, in Ontario, Emergency Management Ontario (EMO) has responsibilities in the EP & CIP area and runs the Ontario Critical Infrastructure Assurance Program. Through cooperation with the Ontario Ministry of Finance, FC has worked with financial sector stakeholders in the province since 2004. Furthermore, the Canadian Bankers Association has its own coordinating committees which deal directly with its members in the EC & CIP arena, and FC communicates with the CBA on EC & CIP issues.

FC relies on these many bodies to carry out operational EP and CIP planning at the appropriate level, but is responsible for ensuring that other parties are informed, that concerns are communicated, and that all relevant parties are "kept in the loop". FC’s partners in the sector see it as the logical conduit to other federal government departments and agencies.

2.1.2.2 Evolving Role

There have been important changes in the nature and complexity of the emergency preparedness and critical infrastructure protection work in the sector since the start of the PSAT initiative in 2002, and these have affected the nature of the contribution which Finance Canada needs to make. Starting with a good basis of BCP and emergent CIP within the individual organizations in the sector (with some inter-connections), the work increasingly progressed to bilateral and multilateral planning and some associated exercises.

Several interviewees felt they are at the stage in their planning where interdependencies with other sectors need to be addressed (e.g. with the telecommunications and the energy sectors which are not regulated by Finance Canada), but find no plan in place or coordination mechanism to address this aspect. Ad hoc meetings and informal communications have taken place, but they feel FC is best placed to represent the sector’s requirements to Public Safety Canada. In addition, several organizations in the finance sector are of the view that the critical nature of the sector itself is not sufficiently recognized by those responsible for higher-level planning, although FC interviewees confirm that the financial sector is one of PSC’s 10 CI sectors. FC’s role is to encourage, coordinate and communicate, but the definition of that role could be broadened to include taking the lead on some issues. If FC is to be the main conduit between the financial sector and other government bodies, then it is responsible to the sector for ensuring that its concerns and views are taken into consideration when other planning which affects the financial sector is being carried out. FC interviewees stated that they determine which of those concerns and views should be communicated to others within government. Thus, FC considers itself as more than a conduit, in that it applies its own judgement in determining what should be communicated.

2.1.2.3 International Connections

There was a sizable consensus that organizations in the Canadian financial sector, particularly those in the private sector, have the capacity to assess the risks to their continued operation. But given the strong international connections of Canada’s financial sector, there is an increasing need for information from other sources to enable a comprehensive picture of the type and severity of threats to their continued provision of essential services and measures to combat them.

Currently, international connections are primarily[1] with the Financial Stability Forum, an international body with members from the G7 and other international financial organizations, as well as other countries such as Hong Kong and Switzerland, which addresses multinational issues that could affect the functioning of the world’s financial sector. FC presents status reports to the FSF, and brings forward issues of concern. The evaluation team saw no evidence that the results of deliberations at the FSF are communicated consistently to the members of the financial sector in Canada. While such communications do occur on an ad hoc basis (e.g. best practices are posted to the OSFI website), there is a need for a more structured communication plan to ensure that information is shared consistently and promptly within Canada.

2.1.2.4 Testing Emergency Plans

Contingency planning, especially business resumption planning, is well advanced in the financial sector in Canada. There have been a few occasions when these plans have had to be implemented and, for the most part, they have been effective. There has been some testing by means of table-top exercises of the simultaneous implementation of the plans of some of the organizations. However, a major exercise involving all of the important organizations in the financial sector has not yet taken place. Since the commencement of the PSAT initiative, the nature and severity of the threats to the continued provision of essential services in the financial sector have grown. For example, a pandemic would seriously affect the operation of the financial system and cause several of the players to implement their emergency plans at about the same time. Some interviewees are of the view that the time has come for a sector-wide exercise based on a scenario of a severe emergency of longer duration, and see Finance Canada as the logical organization to take the lead. Such a comprehensive exercise should be preceded by a series of table top simulations to test specific aspects of emergency plans or business continuity plans. An example of specific aspects would be interdependencies.

Recommendation:

That Finance Canada take the lead in organizing sector-wide exercising to test the adequacy of existing emergency plans to cope with large scale emergencies of longer duration.

2.1.2.5 Staffing Implications

FC resources available to fulfill its responsibilities under PSAT have been static and part-time in nature. And responsibility has been assigned to different staff members over the course of the initiative, resulting in repeated learning curves which have adversely affected effectiveness. Several interviewees commented that the lack of continuity within the department has resulted in receiving information inconsistently.

The location of the program within the department may also have implications. If FC is to play a stronger leadership role, and have the ability to set part of the interdepartmental agenda, it would be desirable for the program to be assigned higher in the hierarchy than it is, and to benefit from dedicated resources with EP & CIP experience.

Recommendation:

That there be full-time dedicated resources experienced in EP & CIP within the Financial Sector Division.

2.2 Achievement of Objectives

2.2.1 Defining Objectives

The two long-term objectives of the FC activities were to:

  • Establish a communication framework and processes to address adverse global and domestic events and situations of emergency and/or crisis on the financial sector.
  • Encourage and coordinate financial sector participants in establishing and developing critical infrastructure protection programs and plans.

Shorter-term objectives can be inferred from the planned activities, namely:

  • to raise awareness of critical infrastructure, interdependencies and business continuity planning and emergency preparedness in the financial sector
  • to assure effective communications during planning stages and in the event of an emergency or crisis affecting the financial sector

The formative evaluation pointed out the lack of specificity of the objectives and the consequent difficulties is assessing the extent to which expected results were achieved. This summative evaluation experienced the same difficulties.

Because reporting during the period focused on activities rather than results, there is little information available about the extent to which FC influenced the financial sector to prepare for an emergency or crisis. The Annual Report for 2006/07 reports as outcomes (or results), for example:

  • a meeting of FC and the Canadian Bankers Association on pandemic planning
  • promote action by private sector financial institutions to address BCP/pandemic planning issues
  • meetings of a working group
  • developed a departmental communications plan and draft media lines for the Minister of Finance during a national security incident involving the financial sector

These are typical of the outcomes stated in the annual report, yet they are, in fact, outputs[2]. In defining its expected results, FC is in a difficult position—although the long-term result of effective plans is the protection of the financial sector’s critical infrastructure in the event of an emergency or crisis, the achievement of that result cannot be gauged in the absence of an emergency or testing through exercises.

The activities of the department are aimed at ensuring that the financial sector has EP & CIP plans, which are normally considered outputs, not outcomes. In the case of PSAT however, the very existence of plans is the targeted result. A better statement of the expected outcomes of FC’s activities might therefore be: plans that are clear, workable, appropriate and integrated.

Recommendation

That Finance Canada formulates specific short-term objectives or expected results, in measurable terms, for its continued activities under a renewed PSAT.

2.2.2 Implementation

The four activities laid out by the Department at the start of the initiative have been implemented. There is ample evidence that there has been an increase in the extent of collaboration among the federal government organizations concerned with EP & CIP in the financial sector during the time period under examination. The document review and interviews revealed that information-sharing activities are more numerous as a result of the PSAT initiative. There were no significant unexpected outcomes although it could be argued that the work stimulated by the PSAT initiative has fostered the pandemic emergency planning currently underway in the sector.

The evaluation team found considerable evidence of increased activity in the areas of critical infrastructure protection and emergency preparedness in the sector since 2002. While it is not clear the extent to which this increase can be attributed to the Department’s activities, the document review and interviews showed that Finance Canada supported and encouraged these developments and facilitated the exchange of relevant information.

2.2.3 Building Awareness

There is more awareness of CIP, CI interdependencies, BCP and EP in the financial sector now than in 2002: in part because of efforts under PSAT, in part because of the experience of dealing with actual emergencies during that time, and in part because more demanding requirements have been put in place. For example, OSFI now reviews business continuity plans of federally regulated financial institutions and the Bank of Canada has established mechanisms to coordinate business continuity plans with payment systems organizations. It is important to note, however, that most stakeholders indicated that their activities during the time period under examination had unfolded more or less according to their original plans.

2.2.4 Assuring Communications

The evaluation team looked for evidence that a communications framework had been discussed and/or established to ensure that the communications needs of all relevant organizations and governments would be addressed and processes or structures established to avoid duplication and ensure consistent messages. Nothing was found beyond the pre-existing mechanisms, and an MOU between FC and PSEPC delineating responsibilities in a financial crisis.

There is evidence, however, that a communication framework has been established for use in the event of an emergency or crisis, in that FC maintains a contact list and disseminates it to all relevant parties.

The Department’s activities with respect to information sharing and collaboration with PSC, the provinces, the US and other foreign governments would appear to have been intermittent over the time period under examination. By and large, the federal agencies and financial institutions use their existing channels to international organizations to share information. As they exist primarily for other purposes, these organizations in many cases do not attach a high priority to EP & CIP.

Finance Canada has established good liaison with PSC but has been less active with respect to foreign governments. Work with the Ontario Ministry of Finance, as part of Emergency Management Ontario’s Ontario Critical Infrastructure Protection Program, has been on going since 2004 Recently, there has been considerable interaction with Ontario since the vast majority of the critical infrastructure is located in Toronto.

Within the sector, the Department has organized meetings, information sessions and carried out some monitoring and reporting on activities at other levels and in other countries. Some stakeholders reported that information sharing by FC had resulted in their giving more emphasis to some activities and becoming more aware of existing arrangements, and that some new avenues for coordinating effort have been created.

A certain amount of coordination of EP & CIP activities within the financial sector occurs by means of agenda items on existing coordinating committees, such as FISC and Sub–FISC. Finance Canada is represented on virtually all of these bodies and plays a leading role on some of them, thus providing opportunities for the furthering of its PSAT objectives. Some interviewees were of the view that the time had come for Finance Canada to establish within the sector a coordinating committee at the working level to facilitate information exchange and trans-sectoral planning.

2.3 Cost-Effectiveness and Alternatives

2.3.1 Management and Decision-Making

The management and decision-making structures in place benefited from the clear understanding of the roles to be played in the PSAT initiative by the various organizations involved. To a large extent, these roles are logical extensions of the raisons d’Ítre of these organizations. This factor was particularly important, as the way in which the objectives and activities were expressed did not lend itself to the establishment of a good management and decision-making structure that encompassed the entire sector, both public and private. At the federal government level, FISC is tasked with "providing a forum for information sharing and discussion among [federal] agencies on supervisory and financial stability matters, including strategies and action plans for addressing problems in the sector and at individual institutions"[3].

There was no specific, continuing inter-organizational decision-making body within the sector for the initiative. Instead, some items for decision were brought to existing committees and working groups while others were handled through ad hoc consultations. Towards the end of the period under examination Finance Canada’s sector representation and advocacy role became more important. This role would have been facilitated had there been a body charged with reconciling different views and concerns, and identifying priority issues for consideration by FISC and other relevant organizations.

Recommendation

That FC establish a Financial Sector Working Group, with public and private sector representation, tasked with improving the coordination of EP & CIP in the financial sector and ensuring that gaps in planning are identified and acted upon.

2.3.2 Monitoring and Reporting

As previously stated, one of the difficulties in assessing the impacts of FC’s activities was the absence of hard data. FC reports to PSC as part of its Annual Reports, but those reports deal primarily with activities undertaken during the previous year. It should be noted that PSC has not requested results information such as "how many institutions have plans" or "how many plans comply with OSFI requirements", that are directly relevant to FC’s stated objectives. Nevertheless, such information would allow for a more accurate assessment of the progress made in the sector. FC is cognizant of the fact that reporting such information might also reveal business confidential information. Consequently, FC is in the position of not being able to publicly report on its progress against its stated objectives. Creating more specific short-term objectives will obviate this problem and allow for the development of performance measurement and reporting process.

2.3.3 Cost-Effectiveness

The evaluation team found that considerations of cost-effectiveness, efficiency in delivery and/or alternatives to the existing arrangements did not loom large in the minds of interviewees or in the documentation. For the most part, financial sector agencies view PSAT initiatives as an extension of their normal ongoing responsibilities and consequently do not cost them separately. The financial institutions do consider new techniques and technologies as part of their ongoing review to find efficiencies in all of their operations, including EP & CIP. For the most part, the means chosen to deliver the PSAT initiative in the financial sector are inherently cost-effective, as they make extensive use of existing mechanisms and authorities. This has led, inter alia, to Finance Canada’s dedicated resources being kept small and correspondingly little incentive to find efficiencies.

In the course of its inquiries, the evaluation team was not presented with any alternatives with respect to the main lines of the way the work was organized. There was wide consensus that Finance Canada was the logical choice to take the lead in the PSAT initiative in the financial sector, and that the organizations involved were the correct ones. Suggested changes in the way the program was delivered to make it more efficient focus on: a clearer exposition of objectives and activities and the linkages between them; a regularized system of monitoring and reporting; and, implementing a formal process of performance measurement. Having a dedicated resource with expertise in both the sector and EP & CIP and attaching Finance Canada’s lead role to a higher profile position would both serve to increase the cost-effectiveness of the Department’s investment.

3 Summary Conclusions

This report has been structured to present useful information to management on major themes. This section summarizes the conclusions of the evaluation team against the specific evaluation questions.

3.1 Relevance

Was there a clear and relevant vision and set of objectives for activities carried out under the initiative? Was the Department’s contribution to the initiative consistent with government wide priorities relating to emergency preparedness?

The activities carried out by FC under the PSAT Initiative were consistent with departmental and government-wide priorities.

Is there a continued need for the Department’s contribution?

Yes, although it is recommended that the department’s contribution be expanded to include a more pro-active leadership role.

3.2 Achievement of Objectives

Were the planned activities implemented and achieved? Did they produce the expected outputs?

Planned activities were implemented and outputs produced.

How well were the overall objectives met?

A communication framework has been established for use during crises. No formal communication framework for planning stages—FC relies on existing or ad hoc mechanisms.

The second objective, to "encourage and coordinate" does not lend itself to measurement of results. However, the evaluation team concluded that the activities undertaken can be assumed to constitute encouragement and coordination.

Were there any unexpected outcomes?

There were no unexpected outcomes.

3.3 Cost-Effectiveness and Alternatives

Were there appropriate management and decision-making structures in place to meet the program objectives?

The activities undertaken are similar to those already carried out by FC as part of its on-going responsibilities for promoting the soundness and security of the financial sector. Consequently, it made use of existing management and decision-making structures.

Was the achievement of objectives accomplished in a cost effective manner?

The Department did not add staff and made extensive use of existing mechanisms for communications, thus minimizing costs.

Is there a need to consider options and/or alternatives to increase cost-effectiveness or efficiency in delivery?

No. Recommendations related to staffing and performance measurement are aimed at improving effectiveness overall, rather than reducing costs.


Notes:

1. In addition to the primary interactions with the FSF, FC also maintained periodic contacts with the U.S. and the U.K. [Return]

2. TBS defines outputs as “direct product or services stemming from the activities of an initiative”, for example, reports, meetings and plans; whereas an outcome is defined as “an external consequence or result”. [Return]

3. Description of Financial Sector Authorities and Agencies, Finance Canada (undated). [Return]


Appendices

Appendix 1: Document Review Matrix

Document Review Matrix
Performance Indicators Evidence of/search for: Responses

Clarity & relevance of vision
& objectives statements

Extent to which vision and
objectives reflect PSAT vision
and objectives
1. Vision statements
(PSAT & Finance)
 
2. Objectives statements
(PSAT & Finance)
 

Extent to which departmental
activities respond to EP and
PSAT priorities
3. Statements of activities  
4. Statements of EP &
PSAT priorities
 

Comparison of planned vs.
implemented activities (incl.
reasons for divergence, if any)

Number and types of
information sharing activities
undertaken (e.g. conferences,
consultations, intra-sectoral meetings)

Decisions made/actions taken
as a result of the department’s
activities

Extent of partnerships created /
enhanced within government
and with the financial sector
5. Planned activities  
6. Activities implemented
(including information
sharing)
 
7. Results of conferences,
meetings etc.
 
8. Partnerships and partnership
activities
 

Extent to which PSEPC, OSFI,
CDIC, BOC and FCAC have
collaborated or coordinated
their activities vis-à-vis the
financial sector

Extent of collaboration with
financial institutions and
industry associations

Extent to which awareness of
critical infrastructure,
interdependencies and business
continuity planning and
emergency preparedness in the
financial sector has increased

Extent to which the financial
sector is undertaking necessary
contingency measures

Degree of cooperation between
financial sector agencies,
institutions and industry
associations on contingency
planning procedures and
reference documents on these
procedures; extent to which
identified gaps have been dealt
with

Extent of information sharing
and collaboration with PSEPC,
the provinces, the U.S. and
other foreign governments
9. Joint projects/activities  
10. New relevant
projects/activities
undertaken as a result of
Finance activities
 
11. Instances of cooperation
on developing
procedures/documents
 
12. Meetings, exchanges of
information
 

Changes from original plans

Emergence of topical/critical
issues not considered in
original plans
13. Articulated original plans  
14. Documentation of
changes to plans
 

Extent to which roles and
responsibilities are appropriate
and articulated

Existence and appropriateness
of monitoring and reporting
systems

Feedback on management and
decision-making processes
15. Statements of roles and
responsibilities
 
16. Monitoring and reporting
systems
 
17. Management and
decision-making
processes
 

Actions taken to
re-focus activities

Strategies developed to deal
with problems and/or lessons
learned
18. Actions taken to re-focus
activities?
 
19. Strategies developed to
deal with problems
and/or lessons learned?
 

Appendix 2: Interview Guide

Evaluation of Department of Finance Activities Related to Emergency Planning and Critical Infrastructure Protection

The Zeta Group is undertaking this evaluation on behalf of the Department of Finance, which is the lead liaison department responsible for the federally regulated financial sector. The Department’s role is to facilitate information sharing, to raise awareness within the financial sector of the need for emergency planning to protect critical infrastructure, and to encourage action. This evaluation covers the 5-year period between 2002 and January, 2007.

The questions in this guide are meant to elicit information about the various elements of the Department’s role and its impact within the financial sector. Please feel free to disregard questions you feel you cannot respond to.

All your responses will be kept anonymous.


1. What is your role related to emergency planning and critical infrastructure protection? What do you think the role of Finance is/should be? Are these roles appropriate?

About activities

2. Is there duplication or overlap in the activities undertaken by the Department of Finance and those of other bodies? Provide examples, please.

3. What emergency planning activities have you undertaken related to the financial sector? Do these differ from original plans? How and why?

4. Have you collaborated or coordinated your activities with those of other organizations within government and the financial sector? Examples?

5. How timely and effective are the information sharing and collaboration on contingency planning procedures, and reference documents on these procedures: (please provide examples)

  • among PSC, the provinces, the U.S. and other foreign governments?
  • between the Department of Finance and financial institutions and industry associations?
  • among financial sector agencies, institutions and industry associations

6. What information sharing activities have you been involved with? How successful were they? What were the results?

About management/processes

7. Have gaps in planning procedures and reference documents been identified? How have the identified gaps been dealt with?

8. Are there any changes from original plans? Why?

9. Are there any topical/critical issues not considered in original plans that have emerged? How have they been handled?

10. How are activities/progress monitored and reported on? Is this system of monitoring and reporting sufficient and appropriate?

11. Is the process for decision-making appropriate?

12. Has there been a need to refocus your activities? How was that handled?

13. Have problems emerged? Which ones? How were they handled?

14. What is working particularly well? Has that information been shared with others? Have any changes been made as a result of those successes?

About results

15. Do Canadian financial institutions have the capacity to assess their risks and prepare contingency plans?

16. Is the financial sector currently undertaking the necessary contingency measures? Examples?

17. Have new or enhanced partnerships emerged as a result of the department’s activities?

18. Have the department’s activities influenced your decisions/actions?

19. Do financial institutions and industry associations have complete and up-to-date contingency plans and measures? Please provide examples.

20. Do you think the financial sector is now more aware of issues around critical infrastructure interdependencies, business continuity planning and emergency preparedness? How much of that awareness can be attributed to the activities of the Department of Finance?

21. Do you have any other comments or suggestions?