A quarterly report that leads with tickets closed and uptime percentage is answering a question you did not ask. I know, because I used to write those reports. I ran engineering services for a mid-market managed services provider, and before that I spent eight years inside Microsoft Consulting Services. I have sat on the delivery side of the table for most of my career. From that chair, the report you receive every ninety days is not a lie. It is an honest account of the wrong things.
This is the gap I want to describe. Not the gap between a good MSP and a bad one, but the gap between what the quarterly report measures and what your firm actually depends on. A competent provider can be doing real work, and the report can still leave you unable to answer the questions your insurer, your auditor, or your managing committee will eventually ask. The work and the reporting are two different disciplines, and most engagement models pay for the first one only.
The report measures effort. Your exposure lives in the gaps.
Tickets closed and uptime are effort metrics. They tell you the provider was busy and the lights stayed on. Neither tells you whether your firm is in a defensible posture. A provider can close every ticket inside the service-level window and still leave a server unpatched for nine months, because nobody opened a ticket about the patch. Effort metrics measure what got asked. Your exposure lives in what nobody thought to ask.
So when I look at a quarterly business review, I am looking for three things, and most reviews contain none of them. First: what changed in the environment in the last ninety days, on purpose and by accident. New users, removed users, firewall rules added, policies edited, systems joined or retired. Second: what got patched, what did not, and why. The "did not" list is the one that matters. A patch deferred for a stated business reason is a decision. A patch deferred because nobody noticed is a finding. Third: what is coming in the next ninety days that requires a decision from you. Renewals, capacity moves, license thresholds, vendor changes. The decisions you will be asked to make, surfaced before you are forced to make them under pressure.
A provider doing those three things is reporting. A provider listing tickets closed is invoicing.
Configured is not the same as working.
The single most useful distinction I carry from the delivery side is the difference between a control that is configured and a control that works. A configuration screen reports intent. It cannot report reality.
A backup job that is scheduled is configured. A backup that has been restored and verified is working. The distance between those two states is the distance between a recovery plan and an assumption. I have seen recovery time objectives buried in the appendix of a services agreement, measured in business days rather than hours, quietly excluding the exact practice-management or case-management system the firm could not operate without for an afternoon. The number was in the contract. Nobody had tested it. A backup you have never restored is not protection. It is a hope with a line item.
Multifactor authentication is the same. A policy that is enabled is configured. A policy that nobody has bypassed through a legacy protocol or a "temporary" exception that has outlived two staff rotations is working. The dashboard shows the first. Only inquiry shows the second. This is why a serious read interviews people and tests samples rather than reading screens. The screen reports what was set. The conversation reports what happens.
Access drifts, and drift is nobody's fault until it is your exposure.
Standing access is the quietest problem in any IT environment, and it is the one I would check first. Engineers rotate off accounts. Projects end but the administrative grant that supported them does not. Support contracts renew and the access list renews with them, unexamined. None of this is a character failing. It is the natural entropy of a working relationship, and it accrues to the client, not the provider.
The control that addresses it is almost free. Not a review, which is a word that lets everyone nod and move on. A re-attestation: a named list of every person and system with administrative access to your environment, dated, and signed by someone on the provider's side and someone on yours. Quarterly. It takes fifteen minutes and it prevents the kind of finding that surfaces, badly, the first time a partner with a confidentiality wall learns that an outside engineer held read access to a matter file. The same discipline applies to offboarding. When someone leaves your firm, the provider closes a ticket. A disabled mailbox is not a revoked session token. A removed desk phone is not a removed remote-access certificate. The right question is not whether the offboarding ticket closed. It is the date each account, credential, and access grant was actually revoked.
The layer that proves the work is the layer most often missing.
Every IT operation runs on three layers of documentation. The first is what the vendors require: configuration guides, runbooks, incident templates. They exist because they had to. The second is what the team actually uses: the script the senior engineer wrote for a reason the senior engineer remembers, the notes that exist because somebody got tired of retyping them. The third layer is the one that proves the work was done. Signed access reviews. Dated patch logs. Change records that match what actually changed rather than only the changes that behaved. Evidence retained as evidence.
The gap between the second layer and the third is the gap between "we do the work" and "we can show the work." A firm cannot forward tribal knowledge to its cyber insurance underwriter. It cannot hand a regulator a senior engineer's memory. An underwriter, an auditor, or a regulator will accept the third layer and nothing below it. Plenty of competent providers live entirely in the first two layers, doing genuine work, generating none of the proof. That is not negligence. It is the absence of a discipline nobody was paid to maintain.
Why this is structural, and what it means for you.
I want to be precise about the cause, because it is easy to misread as blame. Your provider is not withholding these things. The incentives simply do not reward unprompted disclosure. The engagement model is built so the client asks and the provider answers, and the parts the client did not know to ask about sit where they sit. Renewal conversations are about price and seats, not about whether the controls in the agreement still match the firm's risk. Nobody at that table is paid to raise the second question. That is not a flaw in any one provider. It is the shape of the arrangement.
So the fix is not to change providers, and it is certainly not to manage them more aggressively. The fix is to introduce someone whose only job is to ask the questions the model does not surface, and who does not work for the provider. A good provider welcomes that. It hands them defensible evidence they were not otherwise positioned to generate, and the rare provider who resists is itself a finding. That is the entire idea behind an independent assessment: a set of questions that would not otherwise get asked, by someone with no commercial stake in the answers. I built Tidebreak to sit in that chair, because every firm I served on the delivery side eventually reached the same question and the honest answer was that they could not get it from me. They were paying me. The question was legitimate. It just had no legitimate answer from inside the relationship.
If your quarterly report tells you tickets closed and uptime held, it is doing what it was built to do. Ask it the three questions instead. Ask for the restore test, the access re-attestation, and the layer-three proof. The answers, or the silence where the answers should be, will tell you more than any uptime figure ever has.