Skip to main content

Clinical AI evaluation guide

Evaluating clinical AI data flows

Start with the complete path of the information, then ask what each service receives, retains and allows someone to access.

A statement about where a record is stored answers one part of a clinical AI evaluation. It does not automatically explain how a recording is transcribed, where text is sent for AI processing or what remains in a log afterward.

A useful review follows the information through the proposed configuration. The practice can then ask more precise questions about each boundary and the evidence needed to understand it.

Evaluation questions for a proposed deployment.

This is an operational due-diligence framework, not legal advice, a security assessment or a statement that a particular deployment meets regulatory requirements.

List the information entering and leaving the workflow

Begin with the input types and outputs that are actually proposed. These may include a recording, transcript, structured summary, proposed action, reviewed communication and execution history. Do not assume that a roadmap input is part of the initial scope.

For each item, identify the source system, the person or process creating it, the destination and the reason it is needed. Include temporary copies and derived outputs. A summary can still contain sensitive information even when the original recording remains elsewhere.

Can the team identify every place where the proposed workflow creates, receives, stores or transmits the information?

Ask different questions about storage and processing

For storage, ask who controls the location, who can retrieve the information and how changes or deletions are handled. For transcription and AI processing, ask which service receives the input, where processing takes place and which terms govern that processing.

These can be different boundaries. A practice may control the primary record while another service processes a copy of some content. The review needs to establish the actual path rather than infer it from the name of the storage platform.

Which processing step, if any, takes information outside the primary storage environment, and what evidence describes that step?

Define what retention means at each stage

Retention can differ across source records, temporary processing, logs, backups and support systems. Ask what is kept, for how long, under whose control and for what purpose. Ask how the stated settings relate to contractual terms and the actual configuration.

Terms such as "private" and "zero retention" need a precise scope. Establish whether a statement covers the model request, operational metadata, abuse-monitoring records, backups or every other copy involved. An attractive label cannot answer all of those questions by itself.

Also ask whether information may be used for model training or another secondary purpose, and which settings and agreements support the answer. Do not infer a provider's policy from a different plan, region or product.

Can each retention and use statement be tied to the proposed service, configuration and agreement?

Review who can access information and who can act

Read access and action authority are related but distinct. A reviewer may need to see the source context without permission to execute an outbound operation. Administrative and support access also deserve explicit treatment.

Ask how access is limited to the appropriate practice and patient, how roles are assigned and how those permissions are tested. Then ask who can approve a clinical action or communication and how the system knows which version was approved.

For outbound work, examine the recipient source, the approval record and what happens if the outcome is uncertain. A responsible evaluation should be able to explain how the team investigates that uncertainty before repeating an operation.

Can the proposed workflow demonstrate both the information boundary and the authorization decision?

Match the explanation to evidence and agreements

Ask for the proposed data-flow explanation, relevant service settings, processor information, retention terms and the responsibilities assigned to each party. The available evidence should describe the configuration being considered, including any unresolved items.

Regulatory and contractual duties depend on the actual relationship and use. The practice's qualified advisers should establish which agreements, reviews and safeguards are required. A cloud service name, an encryption statement or a broad marketing claim is not a substitute for that assessment.

Keep a list of the questions that remain unanswered. A gap does not disappear because the system diagram is attractive. It needs an owner, an evidence request and a decision about whether the evaluation can proceed.

Start with a bounded workflow and clear review responsibilities

A fictional or fully de-identified example can make an initial discussion more useful without requiring a patient record. Define the workflow, the data it would involve, the actions it could create and the people responsible for review.

Before introducing patient data or outbound actions, agree the acceptance evidence and decision responsibilities for the proposed scope. Record what was validated and what remains outside that scope. This keeps a limited evaluation from being mistaken for approval of the whole product.

Discussing the Pexify approach

Pexify is being developed around clinic-controlled information, practitioner authority and controlled care execution. Its public design principles are a starting point for discussion. The actual configuration, processing path and available scope still need to be confirmed.

Back to insights