
Start with a workflow that proposes changes but cannot execute them. Use restricted test data, preserve existing access rules, and require an authorised person to approve each specific action. Then test refusals, expired approvals and changed records—not just successful outputs. Expand access only after the business owner has reviewed evidence that these boundaries work as intended.
For ai workflow automation, the narrow question is not whether a model can produce a plausible answer. It is whether the surrounding process preserves authority when that answer becomes an action.
The NIST AI Risk Management Framework provides voluntary guidance for managing AI risks throughout design, development, use and evaluation. It is not product certification. The test design below is a practical recommendation consistent with that lifecycle approach, not a claim that NIST prescribes these exact controls.
Consider a hypothetical workflow that reads an internal request and proposes a change to a business record. Separate its stages: reading permitted information, drafting a proposed change, obtaining approval, and submitting the approved change.
For the first test, stop at drafting. A reviewer can compare the proposal with the source material without giving the workflow permission to modify live records.
Write down three boundaries before testing:
Choose a task with a clear owner and a reviewable result. A narrow task produces more interpretable evidence than a pilot combining document search, account changes and external messaging.
Do not make a prompt such as “only access authorised records” the permission mechanism. Treat it as an instruction to the model, while testing access restrictions in the connected application and integration separately.
Use a dedicated test identity with explicitly limited access. Avoid borrowing an administrator's account merely to get the demonstration working. If different users have different permissions, include those differences in the test rather than assuming one successful session represents everyone.
Test both directions: can the workflow retrieve what it needs, and is it prevented from retrieving what it should not see? Review whether generated summaries or execution logs expose restricted information as well as whether the original record is accessible.
Restricted access may reduce context and make drafts less complete. That is a trade-off to investigate, not an automatic reason to widen permissions. The appropriate response might be a human handoff or a narrower task.
An approval should identify what the reviewer is authorising: the target record, proposed changes, destination and relevant supporting information. Avoid a general approval that silently covers later changes to the proposal.
Design tests around the interval between review and execution. What happens if the record changes, the reviewer loses authority, or the workflow regenerates its draft? A conservative pilot should stop and request fresh review when a material part of the approved action changes.
Include rejection and non-response. Neither should be interpreted as permission. Also test repeated submission of the same approved action and decide how duplicate attempts should be handled before enabling live writes.
Human approval adds delay and review effort. Measure those costs alongside draft quality. An approval step that overwhelms reviewers needs redesign; removing it merely to improve a speed metric changes the scope of the pilot.
The NIST generative AI profile complements the AI RMF with cross-sector risk-management guidance. It does not establish any vendor's performance. Your own evaluation must therefore distinguish demonstrated behaviour from assumptions about the model or supplier.
Build a small, documented test set containing normal requests, ambiguous requests, missing information and requests outside the allowed scope. Include source material that asks the workflow to ignore its rules; evaluate it as untrusted content, not new authority.
For each case, record the expected outcome, actual result and reviewer decision. Keep output quality separate from permission enforcement: an accurate draft does not compensate for unauthorised access.
Set acceptance conditions before running the evaluation. A failed boundary test should block scope expansion until it is understood and retested. Passing a limited test set is evidence about those conditions—not proof that every future case will behave correctly.
For help defining a bounded pilot, use the SMERT AI Consulting service page. Bring a process map, permission roles and sample approval decisions rather than only a preferred model.
Related: AI consulting services
Ready to experience the benefits of hospitality automation? Our team is ready to help you get started.