ASSISTANT DEPUTY MINISTER (REVIEW SERVICES) (ADM(RS))

ASSISTANT DEPUTY MINISTER (REVIEW SERVICES)

Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

AUDIT OF DEFENCE PROJECT MANAGEMENT

1259-3-0068 ADM(RS)
978-0-660-47050-4
D2-460/2023E-PDF
1
November 2022
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

TABLE OF CONTENTS

INTRO Project Management Information and Data CONCLUSION &

Framework Management ANNEXES
Pages 3–10 Pages 11-13 Pages 14-16 Pages 17-22
Acronyms Finding 1 Finding 2 Overall Conclusion
Report Guide Recommendation 1 Annex A: Management
Executive Summary Action Plan
Context Annex B: About the
Audit
Key Themes
Annex C & D: Defence

02 03
Project Management
Reviews and
Governance

01 04
November 2022 ADM(RS) 2
section 01
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

ACRONYMS
ADM(DIA) Assistant Deputy Minister (Data, Innovation and Analytics) KPI Key Performance Indicator

ADM(Fin) Assistant Deputy Minister (Finance) L1 Level 1

ADM(IE) Assistant Deputy Minister (Infrastructure and Environment) MGI Materiel Group Instruction

ADM(IM) Assistant Deputy Minister (Information Management) NORAD North American Aerospace Defence Command

OCI Office of Collateral Interest

ADM(Mat) Assistant Deputy Minister (Materiel)
OPI Office of Primary Interest
ADM(RS) Assistant Deputy Minister (Review Services)
PAD Project Approval Directive
ARA Accountability, Responsibility and Authority
PD Project Director
C Prog Chief of Programme
PMB Programme Management Board
CFD Chief of Force Development
SDO Senior Designated Official
DCB Defence Capabilities Board
SRB Senior Review Board
DND/CAF Department of National Defence/Canadian Armed Forces
SSE Strong, Secured, Engaged
DRMIS Defence Resource Management Information System TB Treasury Board
DSP Defence Services Programme VCDS Vice Chief of the Defence Staff

November 2022 ADM(RS) 3

section 01
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

REPORT GUIDE

Here are some guidelines for navigating the document.

This document is best viewed on a device such as a laptop,

desktop or tablet, as opposed to printing.

This document, if printed, should be done so in colour to

maintain the integrity and intent of the graphical
components.

R This icon indicates a recommendation made by Assistant

Deputy Minister (Review Services) (ADM(RS)), for which
the Management Action Plans can be found in Annex A.

C This icon indicates a consideration made by ADM(RS).

November 2022 ADM(RS) 4

section 01
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

EXECUTIVE SUMMARY
Overview Key Findings
Project management is a critical component of maintaining operational capability and
A project management framework and governance is in place to provide guidance,
managing critical technology, infrastructure and equipment within the Department of
processes and oversight for projects.
National Defence/Canadian Armed Forces (DND/CAF). DND/CAF is working towards
fulfilling the defence mandate, as articulated in the 2017 Defence Policy: Strong, Interdependencies are primarily managed at the project level with limited holistic visibility
Secure, Engaged (SSE), while concurrently implementing CAF reconstitution, North at the departmental level. Challenges with interdependency management, such as
American Aerospace Defense Command (NORAD) Modernization and defence policy ineffective tracking and communication across stakeholders throughout the project
updates. Investment in the operational capability of the CAF brings both challenges lifecycle can impact the delivery of related projects and project elements, and increase
and opportunities for the department in executing the Defence Services Programme the risk to delivery of key requirements. It can also lead to downstream impacts to project
(DSP), the essential component of project management. costs and schedule.
DND/CAF is making progress addressing various recommendations identified through A lack of data quality control impacts the availability of reliable information to support
internal and external reviews relating to costing, contingency usage and scheduling, effective project management and senior management decision making. Projects are
and more. The departmental project management framework is evolving to expand challenged with data management due to training, system and process limitations. While
upon key guidance to support successful project delivery. some review or quality assurance of data is occurring within the chain of command, there
is an opportunity to improve the review and validation of data.
Project Objective and Importance
A robust project management framework is key to supporting the management of Overall Conclusion
tradeoffs between cost, schedule and requirements, to managing risk, ensuring
stakeholder engagement and collaboration, and managing key interdependencies. DND/CAF has progressed significantly in recent years by developing and improving the
The objective of the audit was to determine if Defence project management practices departmental project management framework to support successful project delivery. Two
and governance are functioning as intended, to support the successful delivery of key areas for the further improvement of project management are interdependency
major capital, equipment, infrastructure and technology projects. For the audit management and project data management. Addressing these areas, as well as those
methodology, including the scope and scope exclusions, see Annex B. This audit identified in other reviews, will improve the ability of DND/CAF to deliver projects on
examined project management in the context of a significant number of other recent time, within scope, on budget and to meet the identified high-level mandatory
and ongoing reviews to show where further improvements can be made. For further requirements and other expected outcomes.
information on ongoing and recent reviews, see Annex C.

November 2022 ADM(RS) 5

section 01
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

EXECUTIVE SUMMARY – FINDINGS AND RECOMMENDATIONS

FINDINGS RECOMMENDATIONS
Project Management Framework Recommendation 1: Vice Chief of the Defence Staff (VCDS), in collaboration with project sponsors and implementers,
A robust project management framework and should improve existing monitoring and reporting of interdependencies to ensure broader and holistic oversight and
governance are in place to provide guidance, processes communication.
and oversight for projects. Efforts are underway to
implement improvements in the areas of costing, OPI: VCDS
contingency usage and scheduling, as highlighted through OCI: To be determined by OPI
other recent internal and external reviews. Greater
holistic visibility and communication of project Consideration 1: Regarding the management of project interdependencies, potential options for consideration include
interdependencies throughout the project lifecycle mandating projects to report on interdependency risk to the Programme Management Board (PMB) and Senior
require improvement. Review Boards (SRB), requiring attestations (as part of the SRB checklist procedures) by the key implementers to
confirm that adequate consultation is taking place throughout the lifecycle of the project, and maintaining a tracking
tool or database to monitor the impacts of project changes across all interdependencies.
Data Management
A lack of data quality control impacts the availability of Consideration 2: VCDS, in collaboration with ADM(Mat), ADM(IM), ADM(IE), ADM(DIA), and ADM(Fin), should
complete and reliable data to support effective project leverage Defence X to identify and implement controls to improve project management data quality and consistency.
management and senior management decision making.
Projects are challenged with data management due to Consideration 3: VCDS, as the Senior Designated Official (SDO) for the management of projects and programmes in
training, system and process limitations. Additional Defence, will continue to develop and expand upon departmental guidance in support of project management
guidance and processes are needed to ensure activities for all of DND/CAF. This guidance should be further developed in collaboration with other implementing
consistency to support data analytics and reporting. Level 1s (L1), with considerations for improvement of data management in the areas of data standardization, guidance
and Defence Resource Management Information System (DRMIS) usage, and referred to in the Project Approval
Directive (PAD), if applicable.

Table 1. Executive Summary – Findings and Recommendations

November 2022 ADM(RS) 6

section 01
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

CONTEXT
Project Management Within DND/CAF Capital Projects by
As described in the PAD, project management is the systematic planning, organizing and monitoring of allocated resources to Implementer in the DSP
accomplish identified project objectives and outcomes. Project management is a critical component of maintaining operational
capability and managing critical technology, infrastructure and equipment within DND/CAF. Portal (342)
200
The DSP is defined as the unified architecture of all DND-approved services, activities, projects, programmes and portfolios deemed to 180
be essential to the delivery
Evaluation of affordable and effective Defence services to the Government and Canadians. DND/CAF is progressing in 160
its efforts to fulfil the defence mandate, as articulated in the 2017 Defence Policy, SSE, while concurrently in the midst of the largest
recapitalization effort in over forty years. This significant investment in the operational capability of the CAF brings both challenges 140
and opportunities for departmental project management. Since SSE was launched, 130 projects have reached closeout and/or have 120
been completed, including seven projects valued over $1B with a total project approval amount of $14.5B. The DSP is currently
100
tracking 342 capital projects.
80
Major projects are defined as projects for the one-time acquisition of new equipment, materiel and/or services where the total 60
project value inclusive of tax equals or exceeds $10M. Major projects, as shown in Figure 1, have five stages within the project 40
lifecycle: Identification, Options Analysis, Definition, Implementation, and Closeout. Projects are led by the project sponsor, who have
20
an identified operational need, for the first two phases, and by the project implementers: Assistant Deputy Minister (Infrastructure
and Environment) (ADM(IE)); Assistant Deputy Minister (Information Management) (ADM(IM)); and Assistant Deputy Minister 0
IE IM Mat Other
(Materiel) (ADM(Mat)), who deliver the project to completion, for the last three phases. For the distribution of projects across
implementers, see Figure 2. This audit focused on the latter three phases of the project lifecycle. Projects 127 35 175 5
Figure 2. Capital Projects by Implementer

Figure 1. Project Approval Process and Lifecycle Phases

Identification Options Analysis Definition Implementation Closeout

November 2022 ADM(RS) 7

section 01
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

CONTEXT
Project Governance
Governance is the mechanism used by senior management to oversee the achievement of program outcomes in accordance with Treasury Board (TB) Policy and as prescribed in the
Financial Administration Act. Key principles for effective governance include defining accountabilities, responsibilities and authorities (ARA), providing oversight and monitoring of risks,
deliverables, challenge regarding capability ladder decisions, and insights/advice to mitigate risks. There are three levels of governance: government-wide; departmental; and project-
level.

SRBs are critical to project governance. Their role is to enable corporate challenge and oversight in support of the Defence Capabilities Board (DCB) and PMB. SRBs also oversee project
Evaluation
development and provide management advice to the Project Leader as well as project risk and project performance management oversight throughout the project life cycle. SRBs are
one of many governance boards and committees which support projects and provide departmental oversight. For more details on defence governance boards and committees, see
Annex D.

The PAD Resourcing Integration

The PAD contains departmental direction and guidance to Resourcing is a challenge for each L1, Performance
Scoping
Management
ensure that the programme is delivered in a manner with competing demands for both
consistent with higher level policy and guidance. Part one financial and human resources across
lays out the expectations of leaders and managers in the projects and DND/CAF. L1s and project
delivery of capabilities articulated in the Defence Policy. teams are working to deliver key tasks
Requirements Scheduling
Part two provides a comprehensive, step-by-step guide to with existing resources, and mitigate
project and programme management within DND, tailored any resource shortages through Project Management
to both the type and complexity of projects. The PAD prioritization of personnel, prioritizing Body of Knowledge -
conforms to recently published TB Policies: the Policy on tasks and business planning. Key Project
the Planning and Management of Investments and Resourcing challenges are amplified Management
Procurement Costing
the Directive on the Management of Projects and given labour shortages, military Knowledge Areas
Programmes. The PAD establishes new streamlined project postings, staff turnovers, and due to
approval paths that align governance, documentation, unique knowledge and security
process and delegations in accordance with the level of requirements for complex projects. Risk Human
project complexity and risk. Management Resources

Communication

November 2022 ADM(RS) Figure 3. Project Management Body of Knowledge - Key Project 8
section 01 Management Knowledge Areas
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

CONTEXT
Key Stakeholders’ Roles and Responsibilities1
Phase Accountable Supporting VCDS Support/Challenge

Identification Project Sponsor ADM(Mat)

(Project Leader) ADM(Fin) CFD
Options Analysis Project Director Project Manager

Definition
Project Implementer ADM(Mat)
Implementation (Project Leader) ADM(Fin) C Prog
Project Manager Project Director
Closeout

Figure 4. Key Project Stakeholders. This figure illustrates the key stakeholders and their accountabilities throughout the life-cycle of the project.

Project Team Roles and Responsibilities

Project Director • The Project Director’s (PD) role is to ensure project objectives, linked to a validated operational requirement, are established early in a project and maintained through to project completion.

Project Manager • The Project Manager manages the project, while working with the PD to ensure the approved project activities are achieved.

Project Sponsor: Business • Responsible for defining the operational requirements for the capability to be implemented, and for confirming that the delivered capability satisfies the specified requirements.
Ownership

Project Implementer: Functional • Responsible for defining and delivering the required capability once the DCB has selected the option for the capability with which to proceed following the completion of the Options Analysis
Delivery phase.

Assistant Deputy Minister • Provides support to the Deputy Minister as Accounting Officer and is responsible for ensuring the integrity of financial management and comptrollership at DND and in the CAF, accountable for
(Finance) (ADM(Fin)) the Defence Investment Plan, provides financial attestations.

Chief of Programme (C Prog) • Supports the VCDS in providing leadership for corporate strategy management processes through objective analysis, exercising the challenge function for projects in the Definition,
Implementation and Closeout phases, and sound advice on strategic planning options and resource allocations.
Chief of Force Development • Harmonizes, synchronizes and integrates the force development activities of the CAF to develop the capabilities required to produce strategically relevant, operationally responsive and tactfully
(CFD) decisive military forces. The usage of the challenge function for projects in the Identification and Options Analysis phases.
Table 2. Key Stakeholders’ Roles and Responsibilities
November 2022 ADM(RS) 1 Stakeholder roles and responsibilities as described in the PAD
9
section 01
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

KEY THEMES

The Key Findings were aligned into two themes as follows:

1 Project Management Framework

2 Data Management

Image: Pte Ken Lewis, an AVN student on course 0604 at the

Canadian Forces School of Aerospace Technology and Engineering
(CFSATE), looks up part numbers in Canadian Forces Technical
Orders (CFTOs).

November 2022 ADM(RS) 10

section 01
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

PROJECT MANAGEMENT FRAMEWORK

FINDING 1: A robust project management framework and governance are in place to provide guidance, processes and oversight for projects. Efforts are underway to implement
improvements in the areas of costing, contingency usage and scheduling, as highlighted through other recent internal and external reviews. Greater holistic visibility and
communication of project interdependencies throughout the project lifecycle require improvement.

Why It Matters
Without a centralized approach to interdependency management, there is a risk that project changes to schedule or scope will not be tracked and communicated to all relevant
stakeholders. This can impact the delivery of related projects and project elements, and can increase the risk to delivery of key capabilities. Interdependencies can result in a
number of downstream impacts, with the most immediate being the project-level impact to cost and schedule. DND/CAF relies on internal governance at the L3, L2 and L1 levels to
remedy and help aid projects in mitigating project risks. High-priority projects may benefit from a mechanism to allow current issues and risks to be briefed to higher level
governance for advice, direction and information-sharing purposes.

What We Found
Project Interdependencies are factors required Project Interdependencies
for the successful delivery of an individual Project interdependencies can have a substantial impact on the successful delivery of a project. For example, one sampled
project. Interdependencies can be either project has seen increasing costs and schedule impacts as a result of infrastructure interdependencies that were not
dependencies or constraints. Dependencies are accounted for early in the project’s life. As a result, this project, as well as a related project with shared infrastructure, are
tasks and events that occur outside the Project experiencing cost and schedule increases. Project interdependencies are primarily identified in the early phases of a project
Team’s control that must happen before the through the Business Case Analysis, reported in Defence Services Program Portal project dashboards and managed within the
project can progress. Dependencies can include project. There does not appear to be holistic tracking of project interdependencies at a departmental level. The defence
deliverables from other projects or other project management framework, through the PAD and L1 guidance, guides the management of numerous elements and
stakeholders. For example, shared infrastructure projects that are integrated and interdependent, thus impacting both other projects and DND/CAF as a whole. While
between two projects that may delay a future interdependencies are identified at the outset of a project, challenges exist in managing the interdependencies throughout
project would be a dependency. Constraints are the project lifecycle as dependent projects begin and additional constraints arise. Increased consultation, oversight and
factors that must be accounted for when reporting could reduce the risk that project updates and their impacts are not accounted for or sufficiently communicated to
planning the project. For example, a project all relevant stakeholders. Changes to a project’s cost, schedule, or a reduction or change to a capability represent an
constraint may be a shared pool of funding for a interdependency risk and should be managed accordingly. The realization of interdependency risks can have substantial
project's equipment and infrastructure that will downstream impacts across various domains and L1 organizations. Other reviews have highlighted that projects are not
not be sufficient for all project requirements. always adequately identifying indirect resource requirements such as key oversight or internal support functions.

November 2022 ADM(RS) 11

section 02
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

PROJECT MANAGEMENT FRAMEWORK

FINDING 1: Continued
What We Found
Project Interdependencies (continued)
Recent work has been done to address interdependency management in DND/CAF. For instance, infrastructure and
information management interdependencies should benefit from the recent development whereby ADM(IM) and ADM(IE)
are being invited to SRBs to ensure early collaboration. Secondly, C Prog and ADM(IE) are working on revisions to the PAD
to incorporate infrastructure interdependency considerations. ARAs within the PAD and within project documentation are
generally clear, well defined and relevant. Additionally, the SDO and their responsibilities within investment planning and
management, including project management, procurement and materiel, information management, and real property,
have been appointed by the Deputy Minister. Finally, further work is being done to address integration and interoperability
of the military across the CAF and with Allies, through the newly formed Chief of Combat Systems Integration organization.
Recognizing the progress made, other options and processes should be explored to further improve the holistic
management of interdependencies.

Project Guidance
Image: Two CF-18 Hornets from 401 Tactical Fighter
Project guidance, in the form of templates, guides, tools and direction, is abundant and varies across L1s, with a standard Squadron in Cold Lake, Alberta over the Canadian
base of higher level guidance found in the PAD. The PAD provides guidance and templates to inform decision makers of Arctic in March 2021.
defence project management while giving clear direction to project teams. It provides an overview of project processes,
considerations, stakeholders, documentation and more. There is a lack of standardized scheduling and costing tools across Notable Practices:
project implementing L1s. ADM(Fin) costing guidance is detailed and being more proactively communicated with the
introduction of a costing bootcamp. This should aid in the PAD’s recommended early engagement and communication • ADM(IM) has detailed interdependency guidance and
between ADM(Fin) and projects during project costing exercises. Scheduling challenges, along with other key project tools.
management areas such as prioritization and resourcing, are being addressed as the result of numerous recent project • Many projects are recording and sharing lessons learned
management reviews. For more details, see Annex C. The recommendations from these reviews have helped improve throughout the project lifecycle.
departmental practices (e.g., the implementation of 3 point scheduling estimates to provide a range of pessimistic, likely
and optimistic schedule estimates).

November 2022 ADM(RS) 12

section 02
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

PROJECT MANAGEMENT FRAMEWORK

FINDING 1: Continued
What We Found
Risk Management
Risk Management occurs both formally and informally within projects and across L1s, where risks are reported and escalated through SRBs, management briefings and dashboard
reporting. Projects engage with numerous governance bodies, which can be a standard part of the project approval process or unique to each L1, to progress projects and raise and
report key risks. These governance bodies provide a forum for discussion and challenge of capability and project trade-offs as the project moves towards interim and full operational
capability. The challenge function provides senior management with the opportunity to review project progress, risk areas and leverage their departmental knowledge in project
decision making.

Although processes and oversight are in place to report and manage risk, projects continue to face unmitigated and unforeseen risks and issues. While interdependencies are
identified and reported in some risk reporting frameworks, this is not uniform across DND/CAF or L1s. There is a defined process in the PAD for how projects report risk and are
selected for higher level governance bodies. Not all projects are escalating risks in a timely manner. For example, one project experienced significant delays for which key risks were
briefed to an L1 oversight committee, and subsequently to SRB, and corrective actions were taken. These risks/issues were not formally briefed to senior levels until after the project
was back on track (approximately 20 months after their identification).

R1 Recommendation 1: VCDS, in collaboration with project sponsors and implementers, should improve existing monitoring and reporting of interdependencies to
ensure broader and holistic oversight and communication.

OPI: VCDS
OCI: To be determined by OPI

C1 Consideration 1: Regarding the management of project interdependencies, potential options for consideration include mandating projects to report on
interdependency risk to PMB and SRBs, requiring attestations (as part of the SRB checklist procedures) by the key implementers to confirm that adequate
consultation is taking place throughout the lifecycle of the project, and maintaining a tracking tool or database to monitor the impacts of project changes across all
interdependencies.

November 2022 ADM(RS) 13

section 02
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

DATA MANAGEMENT
FINDING 2: A lack of data quality control impacts the availability of complete and reliable data to support effective project management and senior
management decision making. Projects are challenged with data management due to training, system and process limitations. Additional guidance and
processes are needed to ensure consistency to support data analytics and reporting.

Why It Matters
Data management and performance measurement are important as they enable the identification of issues and of unmitigated risks that may have been avoidable. It is difficult to
track and measure the outcomes of the project and make strategic or evidence-based decisions if the data being collected is unreliable. A lack of consistent and effective data
management limits the extent to which project data can be leveraged for continuous improvement purposes and reporting on Program outcomes.

What We Found
Data Quality Notable practices:
Projects use a combination of information systems and tools to manage their data and project information. DRMIS is the system of record for • ADM(Mat) has improved
project, financial and other key data. Data quality in DRMIS requires improvement as the current data used by projects for tracking, monitoring guidance and developed a
and project reporting, is non-uniform, poorly labelled and incomplete. Data is infrequently updated or often taken from other reporting tools strategy for project
that, while more regularly updated, require manual retrieval and cleaning for analysis. management data
analytics. This work is
being supported by the
Project data is not always available or stored in DRMIS and is often held on desktops by staff or within the owner’s organization, limiting its
further development of the
availability for sharing of data amongst organizations. For example, L1s do not always share historical operations and sustainment costing data
Material Group Instruction
with ADM(Fin) to support costing of future equipment needs. As DRMIS is not user friendly, project teams do not always use DRMIS to its full
(MGI-6) guidance.
capabilities. This is due to a number of factors, including a lack of training on DRMIS for project staff, lack of accessibility and the limited non-
• Defence Construction
financial capabilities of DRMIS for project usage. These data quality and DRMIS issues limit the Defence Team from moving forward with
Canada, as a 3rd party and
developing its analytics capabilities, in line with industry standards. For example, project data updates are completed at the discretion of the
contracting authority, is
project team, and dashboards are not frequently updated with supporting comments. These issues are exacerbated by limited controls to verify
reviewing key ADM(IE)
project data. Data quality issues further limit the ability for project stakeholders to conduct analysis and support projects.
project documents and
data.

November 2022 ADM(RS) 14

section 03
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

DATA MANAGEMENT
FINDING 2: Continued
What We Found
Project Reporting
DND/CAF Data Strategy and Program2
Project performance and risk reporting leverages DRMIS data through dashboards,
briefing decks and performance reporting to support senior management decision
making. As DRMIS does not have the ability to track original baselines (prior to changes Canada’s Defence Policy: Strong, Secure, Engaged
from a new expenditure approval), DRMIS is not providing a historical view of schedule
Report to Clerk of Privy Council: Data Strategy Roadmap for
or cost changes and baselines. It should be noted that original baseline data is often the Federal Public Service
reported by project teams through SRB presentations using a manual process. Project
TBS Policy on Service and Digital
reporting contains key risks and Key Performance Indicators (KPI), such as scope,
schedule and cost. The PAD and supplemental guidance from each L1 identifies metrics
for projects to track for performance reporting.

Supporting Initiatives: Defence Data Strategy and Data Program The Department of National Defence and Canadian
The DND/CAF Data Strategy envisions leveraging data in all aspects of Defence Armed Forces Data Strategy

programs, enhancing our defence capabilities and decision making, and providing an
information advantage to the CAF. There are linkages between elements that support
the Defence Data Strategy, such as the data quality framework, data strategy Supporting Data Strategy Data
Data Quality Analytics Vision Data Literacy Change
Implementation Governance and Guiding
implementation plan, Assistant Deputy Minister (Data, Innovation and Analytics) Documents Plan
Framework
Principles
Framework Management Plan
Framework
(ADM(DIA)) Functional Planning Guidance, and more. These tools, in combination with
Accountabilities, Defence Program
ADM(DIA)’s guidance, will help improve data management within project Authorities Responsibilities and Joint Directive on Analytics Initiating DAOD 1000-6: Data,
DAOD 6500-0: Data
Management and
Authorities Framework Data Management Directive Information and IT
management. The Data Strategy Pillars include: Data Management Tools, Data Tools for ADM(DIA) *Revision required*
Analytics

and Environment, Data Literacy and Skills, and Data Culture. Further, the Defence
Resource Business Modernization, Defence X and Data Strategy, new reporting tools, Guidance &
ADM(DIA) Functional Planning Guidance
Direction
approaches, frameworks and system improvements, being developed by DND/CAF may
help address DRMIS issues and increase efficiency of project data entry and reporting,
once implemented. Figure 5: ADM(DIA) Data Program Key Documents

November 2022 ADM(RS) section 03

2 As described on the ADM(DIA) intranet site. 15
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

DATA MANAGEMENT
FINDING 2: Continued
What We Found
Review and Validation of Data
Data review and/or quality control on project data occurs to various degrees across L1s. There is no consistent approach to reviewing and performing quality control on project data
at a departmental level. Instead, project teams and their chain of command review the data and reporting of their own projects while other stakeholders such as ADM(Fin) or C Prog
may review information that is presented on slide decks prior to SRBs. In addition, as an internal control for project information for SSE projects, Functional Authority Delivery Group
representatives attest to the information on their L1 projects, once per year.

While project information is often challenged by the chain of command and in review boards, it is not validated to ensure quality and conformance with guidance. DND/CAF would
benefit from projects having its data validated for reasonableness and accuracy. This would allow quick course correction for projects that may have unreliable or incomplete data
entered in the system of record. L1 guidance on data input and quality requirements varied in depth and quality and lacked standardization. This is expected to improve with the
revisions being made to the departmental and MGI-6-1, implementation of ADM(DIA)’s Data Strategy Implementation Plan, Data Governance and Quality Frameworks.

C2 Consideration 2: VCDS, in collaboration with ADM(Mat), ADM(IM), ADM(IE), ADM(DIA), and ADM(Fin) should leverage Defence X to identify and implement
controls to improve project management data quality and consistency.

C3 Consideration 3: VCDS, as the SDO for the management of projects and programmes in Defence, will continue to develop and expand upon departmental guidance in
support of project management activities for all of DND/CAF. This guidance should be further developed in collaboration with other implementing L1s, with
considerations for improvement of data management in the areas of data standardization, guidance and DRMIS usage, and referred to in the PAD, if applicable.

November 2022 ADM(RS) section 03

16
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

OVERALL CONCLUSION
Project management requires strong leadership, detailed planning, identification and management of risks and interdependencies to deliver numerous key tasks during the project
lifecycle. Effective project management facilitates the timely delivery of a broad suite of capabilities on time, on budget and in support of DND/CAF’s operational and
organizational needs. DND/CAF has progressed significantly in recent years by developing and improving its departmental project management framework to support successful
project delivery and expanding upon key guidance that is constantly evolving. Room for improvement exists in the areas of interdependency management and project data
management.

A robust project management framework is in place to provide guidance, processes and oversight for projects. Without active and holistic interdependency management
throughout the project lifecycle, there is a risk that project changes to schedule or scope will not be tracked and communicated to all relevant stakeholders. This can impact the
delivery of related projects and project elements, and can increase the risk of failing to deliver key capabilities. Interdependencies at the project or departmental level can result in
a number of downstream impacts, with the most immediate being the project level impact to cost and schedule. The need for improved interdependency management may be
amplified with key departmental priorities such as NORAD Modernization.

A lack of data quality control impacts the availability of complete and reliable information to support effective project management and senior management decision making.
While there is guidance at the departmental and L1 levels to support data and information management, additional guidance and processes are needed to ensure data quality and
consistency to support analytics and reporting. While some review or quality assurance of data is occurring within the chain of command, the review and validation of data entry
needs improvement.

DND/CAF has progressed significantly in recent years by developing and improving the departmental project management framework to support successful project delivery. By
addressing shortfalls to interdependency management and project data management, as well as areas and recommendations identified in other reviews, defence project
management will continue to improve. This will enhance the ability of DND/CAF to deliver projects on time, within scope, on budget and to meet the identified high-level
mandatory requirements and other expected outcomes.

November 2022 ADM(RS) 17

section 04
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

ANNEX A – MANAGEMENT ACTION PLAN

ADM(RS) Recommendation R1 Management Action

The VCDS, through the Policy Suite DG Steering Committee, will ensure the monitoring and tracking of
interdependencies is improved by:
Recommendation 1: VCDS, in collaboration with
project sponsors and implementers, should improve • Ensuring SSE Functional Authority Delivery Group Annual Attestation Process, which includes updating of
existing monitoring and reporting of initiative documentation and interdependencies assessments, is communicated across L1s.
interdependencies to ensure broader and holistic • Target – Already completed annually as an evergreen process.
oversight and communication.
• Ensure recent governance and oversight changes to the PAD are achieving desired effect, in particular with
OPI: VCDS infrastructure interdependencies.
OCI: To be determined by OPI • Target – September 2023 – This requires C Prog and ADM(IE) coordination to develop KPIs to measure
and track the effectiveness of Defence Capability Infrastructure changes to the PAD implemented on June 27,
2022. An assessment using these KPIs will leverage data gathered over months in order to be valid.

• Improve current “static” reporting mechanisms for project interdependencies by leveraging emerging digital
tracking methodologies, including the use of Power BI and Dashboards.
• Target – December 2023 – This is an evergreen task with emerging digital tracking methodologies
figuring prominently in our current and future tool sets.

This Management Action Plan requires collaboration across multiple stakeholders and is reliant upon Human
Resources capacity and competency levels necessary to leverage new digital enabling tools.

Target Date – December 2023

OPI : VCDS
OCIs: ADM(Mat), ADM(IM), ADM(IE), CFINTCOM, RCN, RCAF, C Army, ADM(DRDC), CANSOFCOM, CJOC, CMP, SJS,
ADM(DIA)

November 2022 ADM(RS) 18

section 04
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

ANNEX B – ABOUT THE AUDIT

The findings and recommendations of this report were derived from multiple sources of evidence
collected throughout the planning and conduct phase of the project. These sources of evidence
were verified with the OPIs to ensure their validity. The methodology used in this audit were as
follows:
Document Review Audit Objective
The audit team completed a review of relevant internal/governmental policies, The objective of the audit was to determine if Defence project
legislations, directives, communications, procedures, guidelines and templates. management practices and governance are functioning as
Documents were maintained for evidence, as required, and were substantiated intended, to support the successful delivery of major capital,
with other methods of evidence collection. equipment, infrastructure and technology projects.

Interviews
The audit team conducted interviews with key stakeholders. These responses
Statement of Conformance
were used to improve the team’s understanding of areas of concern, existing
The audit findings and conclusions contained in this report
processes and controls, and risks.
are based on sufficient and appropriate evidence gathered
in accordance with procedures that meet the Institute of
Data Analysis
Internal Auditors’ International Standards for the
The audit team carried out data analysis on project data from DRMIS. This data
Professional Practice of Internal Auditing. The audit thus
captured project costing (budgeting) and scheduling (milestone) within the
conforms to the Internal Auditing Standards for the
project management warehouse of DRMIS.
Government of Canada as supported by the results of the
quality assurance and improvement program. The opinions
Case Studies
expressed in this report are based on conditions as they
The audit team conducted a case study of six major capital defence projects.
existed at the time of the audit and apply only to the entity
These case studies were used for audit tests and provide a non-population
examined.
representative view of project issues, challenges and best practices.

November 2022 ADM(RS) 19

section 04
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

ANNEX B – ABOUT THE AUDIT

Audit Criteria
Criteria A: Major Capital projects implemented by ADM(Mat), ADM(IM) and ADM(IE) are supported by a robust project management framework.
• Governance is in place to support the Definition to Closeout phases, including the management of tradeoffs between cost, schedule and
requirements.
• Policies and procedures are in place to support a robust project management framework.
• Risk management practices, consideration of stakeholder engagement and management of interdependencies are integrated into the project
management process.
• Selected projects (case studies) are aligned with the project management framework in place.

Criteria B: The data collected by Major Capital Project implementers is reliable and usable.
• Major Capital Project implementers ADM(Mat), ADM(IM) and ADM(IE) collect and capture data in the system of record to report on project status.
• Quality (timely, accurate, complete and relevant) data is collected by Major Capital Project implementers (ADM(Mat), ADM(IM) and ADM(IE)).
• Conduct data analytics/trend analysis to support partners.

Audit Scope and Timeframe Scope Exclusion

• Major Capital projects in the Definition, Implementation and Closeout project • Identification and Options Analysis phases, due to recent audit coverage
phases. in these areas
• Projects implemented by ADM (Mat), ADM(IM) or ADM(IE). • Infrastructure prioritization, funding and portfolio management
• Scope periods: • Agile procurement, innovation and Gender Based Analysis Plus (GBA
• For departmental practices and processes: August 2019 – September Plus) (ongoing Evaluation of Project Management will focus on these
2021 (from new PAD introduction). areas)
• For project documentation, costing and scheduling data: September 2011 • Design and Operating Effectiveness of Investment Planning
to September 2021 • Minor capital and Vote 1 projects
• CANSOFCOM and ADM(DIA) (from a project sponsor perspective)
Conduct work started in September 2021 and was substantially completed in April • L0/L0.5 Governance Board Structure and Composition
2022.

November 2022 ADM(RS) 20

section 04
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

ANNEX C – Select Defence Project Management Reviews

Ongoing: Recent:
1. ADM(RS) Audit of Defence Project Management 1. ADM(RS) Audit of Preliminary Requirements (March 2019)
2. ADM(RS) Land Equipment Acquisition Evaluation 2. ADM(RS) Targeted Audit of Monitoring and Implementation of SSE OAG
3. ADM(RS) Acquisition Project Management Evaluation - Agile Audit of the National Shipbuilding Strategy (2021)
Procurement, Innovation and GBA Plus 3. PBO Report on the Cost of CSC (2021)
4. ADM(RS) Thematic Evaluation of HMLRs 4. ADM(RS) Land, Air and Space, Naval, and Integrated Force Development
5. ADM(IE) Infrastructure Support to Defence/Canadian Armed Forces Evaluations (November 2021)
Capability Acquisition and Renewal Review 5. ADM(RS) Audit of DRBM (Ongoing)
6. PSPC/ADM(RS) Joint Evaluation on the Risk-based Approach to Defence
Procurement Project

November 2022 ADM(RS) 21

section 04
 Reviewed by ADM(RS) in accordance with the Access to Information Act. Information UNCLASSIFIED.

ANNEX D – Defence Project Governance3

Defence Capabilities Board (DCB)
DCB’s mandate is to provide VCDS, on behalf of the Deputy Minister/Chief of the Defence Staff, with the situational awareness and
decision support with respect to the development and validation of future Defence capabilities. This board serves as the approval
authority for all Strategic Context Documents (SCD) and Business Case Analysis (BCA) prior to acceptance of an initiative into the
departmentally approved balanced portfolio
DCB Infrastructure and Environment Board (IEB)
The IEB, chaired by ADM (IE), ensures that IE performance is meeting expectations and focused on enabling Canadian Armed Forces
Governance Structure

capabilities. It provides strategic advice and corporate guidance to ADM (IE) on infrastructure and environment matters. The IEB ensures
the performance of the IE portfolio and the enabling real property management is regularly and systematically assessed for CAF
IEB operational suitability and relevance, utilization, efficiency, condition and financial performance. The primary goal is to ensure that IE
performance is meeting expectations and focused on enabling CAF capabilities.

Information Management Board (IMB)

IMB The IMB is the senior Information Management (IM) governance body in DND/CAF. It is chaired by ADM(IM) and C Prog. The IMB
provides strategic leadership and recommends priorities on all matters related to the delivery and support of IM/IT in DND/CAF.

Investment and Resource Management Committee (IRMC)

Investment and Resource Management Committee (IRMC)’s primary mandate is to promote the effective allocation and management of
IRMC the Department of National Defence’s available financial resources. The Investment and Resource Management Committee (IRMC)
provides advice to the DM on Budget priorities and requirements consistent with the strategic objectives. The Committee oversees the
allocation, oversight and control of the Department’s financial resources, control of risks, reviews financial policies and practices and
oversees the management and progress of major investments.
PMB Programme Management Board (PMB)
The PMB provides Associate DM, VCDS and the Chief Financial Officer with decision support and advice with respect to the composition
of the Investment Plan (IP) and the management of elements of the DSP.

SRB Senior Review Board (SRB)

The SRB is a departmental committee that supports the Project Leader in the successful delivery of the capability for which an
investment project has been established. There are two major components to the role of the SRB:
Figure 6. DND/CAF Project Governance Structure • To provide "corporate challenge" and stakeholder oversight of the project
• To advise the Project Leader on the development and management of the project, with a focus on providing cross-functional
input to discussions on project risk and performance
SRB members: ADM(IM), ADM(IE), ADM(Mat), and ADM(Fin), C Prog and CFD analysts, as well as other key stakeholders.

3 Board descriptions as described in the PAD

November 2022 ADM(RS) 22

section 04