Options compared: RAC vs static frameworks
The explicit winner for measuring knowledge transfer is retrieval coaching, not static frameworks. Static frameworks such as CITB’s competence model define the target state well: a Route to Competence, a SKEB statement covering skills, knowledge, experience, and behaviours, and an Implementation Plan with a timetable for compliance (CITB). What they cannot do is show whether a worker actually retrieves the right knowledge under pressure. Completion of a framework tells you a box was ticked; it does not tell you how fast the person found the answer or whether the source they landed on was correct.
Retrieval coaching closes that gap because it instruments the retrieval event itself. A platform such as mentaport.xyz logs every query and every source hit, which means each retrieval can be scored on two dimensions static frameworks ignore: latency (how long the person took to surface an answer) and source accuracy (whether the material they retrieved was the right material). Those two signals are what let you map a competency outcome to real-time performance rather than to a training record.
Run the arithmetic before you commit. Subtract your current onboarding days from 91; if the difference is small, the 46-day figure may not transfer to your context, and a static framework remains the cheaper baseline. If the gap is wide and your teams already log queries, the infrastructure spend is the smaller line item.
One rule governs adoption: instrument first, coach second. If you cannot capture retrieval events — query text, latency, source hits — and map them to competency outcomes, retrieval coaching gives you no measurable advantage over a checklist. Keep the static framework, keep the baseline, and revisit when the logging exists.
What the evidence does NOT establish
The strongest objection to retrieval coaching is not that it fails, but that nobody has shown how long it lasts. No source in the available sources establishes whether retrieval coaching improves long-term retention beyond onboarding. The mentaport.xyz pilot measured speed and proficiency during the program window; it did not follow those 342 hires months later to see whether the retrieval advantage persisted, decayed, or reversed. If you are evaluating a vendor, ask for a delayed post-test at a fixed interval after onboarding ends, with the same retrieval-accuracy and latency measures used at baseline. A program that cannot produce that data has demonstrated onboarding acceleration, not durable competence.
Generalization is the second gap. The CITB competence frameworks define competence requirements for individuals across the built environment, built around a Route to Competence, a SKEB document covering skills, knowledge, experience and behaviour, and an Implementation Plan. That structure is specific to construction and the Building Safety Act context. Nothing in the available sources shows that retrieval coaching validated against CITB-style frameworks transfers to software, healthcare, finance, or teaching. Before adopting it elsewhere, run a local validation: map your own competency outcomes to retrieval events, then confirm the correlation holds in your domain rather than assuming it carries over.
Scale is the third gap. The pilot evidence comes from a single organization and a single cohort of 342 hires. No multi-site replication is cited in the available sources. One site can produce a 28% speed advantage through local factors — a strong internal knowledge base, a motivated cohort, a champion manager — that would not reproduce elsewhere. Treat single-site results as a hypothesis, not a benchmark.
| What the evidence supports | What it does not |
|---|---|
| Faster onboarding proficiency in one 342-hire pilot | Retention after onboarding ends |
| Retrieval accuracy and latency as measurable signals | Transfer to non-built-environment industries |
| Comparison against traditional mentorship at one site | Replication across multiple sites or cohorts |
These gaps do not invalidate retrieval coaching; they define its evidentiary boundary. The reader rule stands: adopt it only if you can instrument retrieval events — queries, latency, source hits — and map them to competency outcomes. If you cannot, a static framework remains the defensible baseline, because it at least documents what competence requires even when it cannot measure transfer.
Mapping a retrieval event
Start with the raw event, not the framework. A learner types “safety protocol for confined space” at 10:32:04. The system returns SOP-123 from the vector index at 10:32:08. That is a four-second round trip, and it is the atomic unit you log. Everything downstream — accuracy, latency, source hit — attaches to this one record. If you cannot capture the query string, the timestamp, the returned document ID, and the response timestamp, you do not have a retrieval event; you have an anecdote. This worked example is illustrative of the logging fields, not a measured result from your organization.
Checkpoint 1 is source accuracy. Confirm that SOP-123 is the current version by verifying it against the CITB SKEB document for that occupation. The CITB competence framework pairs a Route to Competence with a SKEB statement listing the skills, knowledge, experience, and behaviours required for the role. If the retrieved SOP predates the current SKEB revision, the retrieval is inaccurate regardless of how fast it arrived. Log the verification source and the verification timestamp alongside the event.
Checkpoint 2 is latency. Compare the four-second round trip against the threshold your organization set during the mentaport.xyz pilot. If the event meets that threshold, mark it as a pass; if it does not, mark it as a fail and record the delta. Do not average latency across a session — a single slow retrieval on a safety query is the signal you need, and averaging hides it.
Use the following checklist to log each retrieval event. It is copy-usable as written.
| Field | Example value | Rule |
|---|---|---|
| Query string | safety protocol for confined space | Capture verbatim |
| Query timestamp | 10:32:04 | Record to the second |
| Returned source ID | SOP-123 | Log the document identifier |
| Response timestamp | 10:32:08 | Record to the second |
| Latency | 4 seconds | Compute response minus query |
| Accuracy check | Current version confirmed | Verify against CITB SKEB |
| Latency threshold | Met | Compare to pilot threshold |
| Competency outcome | Mapped | Link event to competency |
Two rules keep this honest. First, every field must be populated at the moment of retrieval, not reconstructed later from memory or from a completion report. Second, the competency outcome field must point to a specific outcome in your framework — if it points to a checkbox, you have rebuilt a static framework with extra steps. The reader rule stands: adopt retrieval coaching only if you can instrument these events and map them to competency outcomes. If you cannot, keep your static framework for baseline assessment and revisit the instrumentation question before you change anything else.
Decision rules for adoption
Adoption of retrieval coaching should be governed by two instrumented conditions, not by enthusiasm for the model. The first condition is scale: retrieval coaching depends on a vector index that only pays for itself when the organization generates enough retrieval events to make indexing, ranking, and re-ranking worthwhile. The second is observability: you must be able to log the retrieval event itself — the query, the latency between query and answer, and whether the returned source was the correct one. If either condition fails, a static framework remains the better instrument.
Apply the scale rule first. If your organization generates too few retrieval events per year to make indexing, ranking, and re-ranking worthwhile, keep a static framework such as CITB's competence model, which pairs a Route to Competence with a SKEB document (Skills, Knowledge, Experience and Behaviour) and an Implementation Plan. That structure is cheap to maintain at low volume and gives you a defensible record without requiring a retrieval index. Below that threshold, the cost of instrumenting queries, latency, and source hits exceeds the value the index returns, because the corpus of retrieval events is too small to distinguish signal from noise. Set the exact volume threshold from your own event logs rather than importing a number from another organization.
Apply the observability rule second. If you can log retrieval latency and source accuracy — that is, you can capture each query, time the interval to a usable answer, and verify whether the cited source actually resolved the question — then adopt retrieval coaching as your knowledge-transfer measurement. This is the operational test: not whether you like the model, but whether your systems emit the events it needs. Where those logs exist, retrieval accuracy and application speed become the metrics, and checklist completion stops being the proxy for competence.
Apply the regulatory rule third, and treat it as an override rather than a substitute. If your competency framework is regulatory — for example, one built to satisfy the Building Safety Act, under which individuals must prove they are competent to do the job they are on site to do — keep the static SKEB for compliance. The SKEB statement, the Route to Competence, and the Implementation Plan are the artifacts regulators and sector bodies expect. Add retrieval coaching alongside them for real-time evidence of transfer, not in place of the compliance record.
| Condition | Rule |
|---|---|
| Fewer than 100 hires per year | Use static frameworks (CITB SKEB); retrieval coaching lacks the event volume for index ROI |
| Retrieval latency and source accuracy are logged | Adopt retrieval coaching for knowledge-transfer measurement |
| Framework is regulatory (e.g., Building Safety Act) | Keep static SKEB for compliance; add retrieval coaching for real-time evidence |
Read the three rules in order. Scale decides whether retrieval coaching is economically viable; observability decides whether it is technically possible; regulation decides whether a static framework must remain regardless. Only when scale and observability both hold, and regulation does not forbid substitution, should retrieval coaching carry the measurement load on its own.
What to do next
| Step | Action | Why it matters |
|---|---|---|
| 1 | Audit whether your LMS or learning platform can log retrieval events — queries, latency, and source hits — as discrete, timestamped records tied to individual learners. | This is the gating condition of the canonical decision rule: without instrumented retrieval events, retrieval coaching cannot be mapped to competency outcomes. |
| 2 | If instrumentation exists, define the mapping from each retrieval event type to a specific competency outcome before any coaching rollout begins. | Adoption is only justified when retrieval events can be mapped to competency outcomes; an unmapped event stream produces no measurable knowledge transfer. |
| 3 | If instrumentation does not exist, keep static frameworks for baseline assessment and follow the MoodleDocs sequence: import competency frameworks, add competencies to a framework, then set a competency rule. | Static frameworks require importing frameworks, adding competencies, and setting competency rules before they can be used — skipping any of these three leaves the baseline unusable. |
| 4 | For any teacher-facing AI competency framework, cross-check your design against the conclusions and recommendations in the UNESCO IITE report's analytical report. | The report's conclusions and recommendations on framework design are the documented reference point for teacher AI competency frameworks. |
| 5 | Re-run the decision rule at the next review cycle: instrumented retrieval events plus outcome mapping means adopt retrieval coaching; otherwise hold static frameworks as the baseline. | The rule is conditional, not permanent — the choice flips only when the instrumentation and mapping capability changes. |
Frequently Asked Questions
What does a static framework like CITB's competence model actually define?
It defines a Route to Competence, a SKEB statement covering skills, knowledge, experience, and behaviours, and an Implementation Plan with a timetable for compliance.
What two scoring dimensions do static frameworks ignore that retrieval coaching captures?
Latency (how long the person took to surface an answer) and source accuracy (whether the material they retrieved was the right material).
What does completing a static framework actually tell you?
It tells you a box was ticked, but not how fast the person found the answer or whether the source they landed on was correct.
How does a platform like mentaport.xyz enable scoring of retrieval events?
It logs every query and every source hit, which means each retrieval can be scored on latency and source accuracy.
What arithmetic should you run before committing to retrieval coaching?
Subtract your current onboarding days from 91; if the difference is small, the 46-day figure may not transfer to your context and a static framework remains the cheaper baseline.
When does a static framework remain the cheaper baseline?
When the gap between your current onboarding days and 91 is small, since the 46-day figure may not transfer to your context.
Quick answers
| Which option is the explicit winner for measuring knowledge transfer? | Retrieval coaching, not static frameworks. |
| What do static frameworks such as CITB's competence model define well? | The target state: a Route to Competence, a SKEB statement covering skills, knowledge, experience, and behaviours, and an Implementation Plan with a timetable for compliance. |
| What can static frameworks not show? | Whether a worker actually retrieves the right knowledge under pressure, because completion only tells you a box was ticked — not how fast the person found the answer or whether the source they landed on was correct. |
| How does retrieval coaching instrument the retrieval event? | A platform such as mentaport.xyz logs every query and every source hit, so each retrieval can be scored on latency and source accuracy — the two dimensions static frameworks ignore. |
| What arithmetic should you run before committing? | Subtract your current onboarding days from the 91 figure; if the difference is small, the 46-day figure may not transfer to your context and a static framework remains the cheaper baseline. |
Also worth reading: Workplace AI literacy 2026: Article 4 splits retrieval coaching 60/40 — train or verify: Workplace AI literacy 2026: Article · Team coaching tools: Docebo vs Zoom video at 12-seat threshold: Team coaching tools: Docebo vs · New hire coaching: Retrieval-Augmented Coaching (RAC) cuts 31% vs mentor 2026: New hire coaching: Retrieval-Augmented Coaching