Step 4 — Met, Partially Met or Not Met? Assessing Control Effectiveness

Step 3 gathered artifacts. This step converts them into a result that a system owner may dispute and an authorizer will rely on.

Here is the uncomfortable fact underneath it: the result vocabulary is a departmental invention. ITSP.10.033 does not define met or partially met. NIST SP 800-53A uses a different pair entirely — satisfied and other than satisfied.³ The four-value scale most Canadian departments use is convention, not standard.

If "partially met" is not defined in evidence terms before the first assessment runs, it will mean something different to every assessor who uses it.

The Four Results, Defined in Evidence#

Write these down, in your methodology, before the first control is assessed. The published Beyond Checklists post makes this point as a principle; the table above is the definition.

Assess by Statement, Not by Control#

ITSP.10.033 splits every control statement into lettered parts:

Each of those statements carries "a separate alphabetic designator (A., B., etc.)".¹ A control with parts A through D is four requirements. Assessing it as one produces results nobody can act on: the system owner cannot tell what to fix and the authorizer cannot tell how much is missing.

The single most practical improvement most programs can make is to record results per part. It follows directly from the catalogue's own structure, and it makes "partially met" mean something — A, B and D met; C not met — instead of nothing.

Design vs Operating Effectiveness#

A control can fail two ways, and they are not the same failure.

A finding should say which. "AC-7 not met" tells the system owner nothing about whether to change the design or fix the deployment. Map it to the designed → implemented → operating ladder from Part 4, and name the rung.

The "Partially Met" Trap#

This is the section that earns the post.

Partially met becomes the place assessors put anything they are not comfortable failing. The symptoms are recognizable: most results partially met; no letter named; remediation described as "strengthen" or "enhance"; and a system owner who cannot tell what would change the result.

Applied consistently, this rule tends to split the pile three ways: some results move to met because the gap was cosmetic, some to not met because the gap was the control, and what remains are genuine partials — now actionable, because each one names its part.

Not Applicable, Revisited#

Tailoring said not applicable. Evidence sometimes disagrees — the system that "has no removable media" turns out to have a USB-backed backup job.

That is not an assessment problem; it is a tailoring problem surfacing late. Reopen the tailoring decision, record the correction, and assess the control. The tailoring record from Step 2 is what makes this a two-line change rather than an argument about who said what.

Writing the Result So It Survives#

A finding needs five things and no adjectives:

1. The control and the partAC-2 part D
2. What was expectedAccounts disabled within the defined period after departure
3. What was foundThree of twenty sampled leaver accounts active after 90 days
4. The evidence relied onLeaver log extract, 2026-09-01; sample list attached
5. The gap, in one sentenceDeparted-staff accounts are not consistently disabled

Compare: "Account management is weak and should be improved." Nobody can dispute it, fix it, or rate it. Precision is what makes the result both defensible and useful — which are the same property seen from two sides.

Consistency Across Assessors#

Where CtrlFort Fits#

Calibration between people is hard. Calibration inside a system is a property.

CtrlFort Assess computes determinations from evidence against each control's written assessment criteria in the deterministic engine, so the same evidence produces the same result whichever assessor is looking at it — the consistency the assessment intelligence architecture is built around. Because the criteria are written down per control rather than carried in an assessor's head, calibration becomes a property of the methodology instead of a meeting before the report.

CtrlFort AI drafts the finding text from the structured result — what was assessed, what was reviewed, what was missing, how the determination was reached — so the narrative cannot quietly say more or less than the determination. The assessor validates it. The judgment is still theirs; the platform just refuses to let a vague one through.⁴

Final Thoughts#

Define the results in evidence terms. Assess by part. Name the rung of the ladder. Do those three things and the closing meeting is about the system, not about what the assessor meant.

Previous: Step 3 — Control Responses and Evidence Collection · Next: Step 5 — From Finding to Risk Statement

Frequently Asked Questions#

Is "partially met" a real assessment result or a departmental convention?#

A convention. ITSP.10.033 does not define result vocabulary, and NIST SP 800-53A uses satisfied / other than satisfied. The four-value scale is widely used in Canada and works well — provided each value is defined in evidence terms and applied per lettered part.

What is the difference between design and operating effectiveness?#

Design asks whether the control, as specified, would achieve its objective. Operating asks whether it is actually running that way. A control can pass one and fail the other, and the fix is different in each case.

How do you assess a control whose parameters were never defined?#

You cannot, honestly. Record the parameter as undefined, treat it as a tailoring gap, get it defined, then assess. Inventing a value at assessment time produces a result nobody agreed to be measured against.⁵

How do two assessors reach the same result on the same control?#

Written result definitions, results recorded per part, a decision tree they both walk, and a calibration pass where they compare before the report. Disagreement then becomes a specific question — which of the three did you answer differently? — rather than a matter of opinion.

References#

1. ITSP.10.033 — Concepts and structure ↩Canadian Centre for Cyber Security · https://www.cyber.gc.ca/en/guidance/cyber-security-privacy-risk-management/itsp10033/concepts-structure
2. ITSP.10.033 — Security and privacy controls and assurance activities catalogue ↩Canadian Centre for Cyber Security · https://www.cyber.gc.ca/en/guidance/cyber-security-privacy-risk-management/itsp10033
3. NIST SP 800-53A Rev. 5 — Assessing Security and Privacy Controls in Information Systems and Organizations ↩National Institute of Standards and Technology · https://csrc.nist.gov/pubs/sp/800/53/a/r5/final
4. ITSG-33 Annex 2 — Information System Security Risk Management Activities ↩Canadian Centre for Cyber Security · https://www.cyber.gc.ca/en/guidance/annex-2-information-system-security-risk-management-activities-itsg-33
5. Tailoring — glossary entry, citing NIST SP 800-53 Rev. 5 ↩National Institute of Standards and Technology · https://csrc.nist.gov/glossary/term/tailoring