criticalaccess.aiAI for rural hospitals

Playbook 02 · Six weeks

A six-week pilot, with the staff you have.

A pilot is a small, honest test with an end date. One tool, one department, one owner, and a decision at the end. You don’t need a data scientist. You need a baseline, a tally sheet, and the willingness to stop if it isn’t working.

For a project lead and one departmentSix weeksOne tool, one decision

01 · Before you start

What it takes, and what you get.

This route assumes your governance group has approved the use case. If you don’t have a group yet, Your first 90 days sets one up.

Who leads
A project lead from the department that will use the tool. Not the vendor, and not IT alone. IT supports, and the vendor answers questions.
Time
Six weeks of use, after a short stretch of preparation. The decision meeting goes on the calendar before week 1.
What you need
One approved use case. A signed business associate agreement if patient information is involved. A few staff who volunteered, including one skeptic. A way to count: a tally sheet, a report from your EHR, or a shared spreadsheet.
What you’ll have at the end
A one-page result with the baseline next to it, and a decision to keep, change, or stop.

02 · What to measure

Three questions. Count each by hand if you have to.

Pick measures your own staff can collect. A number you can’t get without the vendor’s dashboard is the vendor’s number, not yours.

  • A

    Did it help?

    One or two numbers the problem is about: minutes per task, items reworked, claims sent back, calls answered. Collect them the same way before and during the pilot.

  • B

    Are people using it?

    Who used the tool this week, how often, and who stopped. A purchase is not use. If use falls, the first measure stops meaning anything.

  • C

    Is it right, and is it safe?

    Each week, a person checks a sample of the tool’s output against the source. Every error and near miss goes in a log, with how it was caught.

03 · The route

Six mile markers, one decision.

The first step happens before the clock starts. The last one is a meeting that is already on the calendar.

  1. Mile1Before week 1

    Write the one-page pilot plan

    One page: the problem, the tool, the owner, who is taking part, what you’ll measure, what would make you stop, and the date you’ll decide. If it runs to a second page, the pilot is too big.

    • A named owner who works in the department
    • A small group of volunteers, including at least one skeptic
    • The three measures: did it help, are people using it, is it right
    • A stop rule, decided now: the result or the problem that ends the pilot early
    • The decision meeting is on the calendar for the end of week 6
    • The governance group has approved the plan
    • The business associate agreement is signed before any patient information goes in
    Where small hospitals get stuck

    Letting the vendor run the pilot. The vendor’s job is to make the tool work. Your job is to find out whether it helps. Those are different jobs, and only one of them belongs to someone on your payroll.

  2. Mile2Week 1

    Measure before you change anything

    Spend the first week counting how things work today. Use what you have: a tally sheet at the desk, a report from the EHR, a few timed samples. Then train the group and let them practice where nothing is at stake.

    • A baseline for each measure, collected the way you’ll collect it for the rest of the pilot
    • A note on who counted, how, and over which days
    • Every participant has had hands-on practice
    • Practice used test data or the vendor’s training setup, not live patient records
    • Everyone knows how to report a problem, and who to tell

    If the baseline has to come out of your EHR or billing system, the data readiness guide covers how to ask for it.

  3. Mile3Weeks 2–3

    Start small, and watch closely

    Go live with the small group only. For the first week, hold a five-minute check-in every day. Most early problems are about workflow, not the tool: where the device sits, who does which step, what happens on a busy day. Fix those fast.

    • Only the volunteer group is using the tool
    • A person reviews the tool’s output before it is used or filed
    • A daily check-in for the first week, then weekly
    • Every problem is written in one log, with the date and what was done
    • The measures are being counted each week, by your staff
  4. Mile4Week 4

    Check the stop rule at the midpoint

    Halfway through, put the numbers next to the baseline and read the stop rule out loud. Look first at use. If people have quietly gone back to the old way, find out why before you look at anything else.

    • Who is still using the tool, who stopped, and the reason each person gave
    • Each measure compared with its baseline
    • The error log reviewed, line by line
    • The stop rule checked honestly. If it has been met, stop
    • If you adjust something, change one thing at a time and note the date
    Where small hospitals get stuck

    Small numbers. In six weeks at a 25-bed hospital, you won’t have enough cases to prove much with statistics. Don’t pretend otherwise. Look at every case instead of an average, write down what you saw, and be plain about what the pilot can and can’t tell you.

  5. Mile5Week 5

    Ask the people doing the work

    Numbers tell you what happened. The staff tell you why. Talk to every participant, and to a few people who chose not to take part. Then get the full cost of keeping the tool, in writing.

    • A short conversation with each participant: what got easier, what got harder, what they’d tell a colleague
    • A conversation with at least one person who opted out
    • If patients were affected, what they said, and any complaints
    • The cost at full scale, in writing from the vendor, including setup and support
    • Your own costs: staff time for training, review, and upkeep
  6. Mile6Week 6

    Decide: keep, change, or stop

    Bring one page to the governance group and make the decision on the date you set. All three answers are good outcomes. A pilot that ends in “stop” did its job.

    • A one-page result: baseline, what changed, what went wrong, what staff said
    • A decision, recorded with the reason
    • If you keep it: an owner, a training plan for everyone else, and a date for the next review
    • If you change it: what changes, and a new decision date
    • If you stop: the tool is switched off, and your data is returned or deleted as the contract says
    • The staff who took part hear the outcome first
    • A short note goes to the board
    Where small hospitals get stuck

    The pilot that never ends. Nobody makes the decision, the tool stays on, the contract renews, and no one is watching it. That’s why the decision meeting goes on the calendar before week 1.

04 · After week 6

You know more than the brochure did.

Whatever you decided, you now have your own evidence from your own building. That is worth more at contract time than any reference call.

Before you expand a tool that touches patients or billing, check the regulatory map for rules that apply to that use.

What done looks likeA decision, with the evidence behind it

  • A decision, made on the date you set, by the people who own it.
  • A one-page result: baseline, what changed, what went wrong, what staff said.
  • A use record: who used the tool each week, and who stopped.
  • A log of every error, and how each one was caught.
  • If you kept it: an owner, a training plan, and a date for the next review.
  • If you stopped: the tool is off, your data is handled as the contract says, and the lesson is written down.

Stuck at a mile marker?

Tell us which step and what’s in the way. If other hospitals are stuck in the same place, the answer becomes part of this playbook.