Screening

This page assumes Fettler is installed and turned on in a project. That is installing it and in a project, in that order. The screen is the last thing you switch on, and most projects never switch it on at all.

# 1. decide whether you need it at all       <- most trees do not
# 2. add "screen" to the tree in .fettler.json     <- works at once
#    restart the assistant, and test a refusal
fettle roots                                 # confirm what is screened
fettle read one-of-the-records.pdf           # confirm it refuses

That is the whole feature for most people who want it. Names, addresses, conditions and free text need a model, which is a second program and a download you fetch yourself — installing the models is that half, and nothing on this page depends on it.

Do you need this?

Switch it on when a tree holds records about people, and a person outside your team could read the transcript. Fettler reads PDF, Word and Excel — the formats regulated data actually lives in. What it returns goes into an assistant's context, then into a transcript, a log, and whatever ships those onward. The screen sits between the file and that chain.

Note Do not put "screen": true on a source tree to see what happens. One of the patterns matches any email address, and source trees are full of them — in comments, in test fixtures, in package metadata. Every read that touches one will refuse. That is the detector working correctly, in a tree nobody meant to screen.

What gets screened

The two commands that hand content back: read and search. Nothing else returns file content, so nothing else is screened. find returns paths, roots returns your own configuration, and extract writes files inside the tree rather than sending them out.

It judges what is being sent, not what is in the file. A document may hold a patient's details on page 40. Read page 2, and page 2 is sent, because page 2 discloses nothing. Judging the whole file would lock every useful tree, and the way out of that is to turn the screen off.

A search is judged one file at a time, as the matched lines plus whatever context brings with them. The screen sees exactly what you would have seen.

Switch on the first tier

This needs nothing installed and works the moment you restart the assistant. Add "screen" to the tree that holds the records, in .fettler.json:

{
  "trees": {
    "work": { "path": "." },
    "records": {
      "path": "../records",
      "can": ["list", "read"],
      "screen": ["identifiers", "clinical"]
    }
  }
}

Three ways to write it, and one that is refused:

A misspelt category is refused too. "lgal" would quietly subtract a screen, and you would not find out until data left a tree you believed was covered.

Note Three regulation-shaped words are refused outright, and the refusal says what to write instead. phi, pii and sci name a regulation rather than what actually runs, so each promises coverage that only arrives once a model is installed while reading as though it were there from the start. They are refused rather than quietly translated, because a translation would keep them alive in configurations indefinitely, still over-promising.

The four categories

What the patterns catch

These run inside fettle itself. There is no model, no second program, and no new download.

All six are one category, and that is the point of the name. They are all identifiers with a shape somebody fixed or a label announcing them. None of them reads clinical language, and a refusal from any of them reports as identifiers — never as a judgement about what kind of document it came from.

They are narrow on purpose, and two of them are deliberately label-driven. No two hospitals agree on the shape of a record number, so the only honest thing to match is the label that introduces it. Matching bare identifier-shaped text would refuse every part number in every spreadsheet — and a check that cries wolf is a check somebody switches off.

Nine bare digits are an order number far more often than a social security number, so nine bare digits do not match. Neither do ten bare digits as a phone number, nor sixteen bare digits as a card.

Carving out a folder that does not need it

A scope's screen replaces the tree's, exactly as its can does. That is what lets a scope take screening away:

{
  "trees": {
    "records": {
      "path": "../records",
      "can": ["list", "read"],
      "screen": true,
      "scopes": {
        "anonymised": { "can": ["list", "read"], "screen": false },
        "exports":    { "can": ["list", "read"], "screen": ["identifiers"] }
      }
    }
  }
}

• A scope that says nothing about screen inherits. Every scope anybody wrote before this feature existed says nothing, and reading that as "screen nothing here" would punch a hole in the first tree anyone switched it on for.
• screen is not a permission. The seven permissions are named after what happens to a file. A screen is a property of content leaving the boundary, and it is not a verb.

Check it, then prove it

Restart the assistant first. The server reads .fettler.json when it launches and does not re-read it, so a screen you just added is not on yet. Then ask for the roots:

$ fettle roots
work          /work/shoddy  (default)
              can: list read create update rename delete
records       /work/records
              can: list read
              screen: identifiers clinical
              anonymised  can: list read  screen: nothing

no "models" directory is declared, so clinical, legal, scientific screen nothing here - only the identifier patterns run

declared by /work/shoddy/.fettler.json

The screen line appears only where a screen is on, and a scope shows one only where the scope itself states one. A scope with no screen line is inheriting.

Now read a file you know holds an identifier:

$ fettle read ward-round.docx
refused: this response would disclose regulated data: 2 in identifiers. What was found is
deliberately not quoted back, because a refusal that named it would put it in the
log and the transcript, which is the disclosure being refused. Read a narrower
range, or take the screen off this scope if the content is not what it looks like.
$ echo $LASTEXITCODE
13

A category and a count, and never the text itself. A refusal that named what it found would write it into the log and the transcript — the exact harm the screen exists to prevent, delivered by the screen. Every screening refusal answers 13.

The two findings here are the labelled MRN: line and the labelled DOB: line, which is why the count is two and the category is identifiers. The clinical prose around them was not read by anything — no model is installed.

Prove it refuses

The configuration being right and the screen holding are two different claims, and only the second matters. Ask the assistant to do each of these, and watch what comes back.

Driving burler by hand

When the model tier is the part in doubt, run burler yourself. It reads one request per line and answers one line:

$ burler --models C:/models/screening
{"op":"screen","categories":["clinical"],"payload":"the weather was fine"}
{"ok":true,"findings":[]}

Type the request line, press return, and read the answer.

An empty findings array means it checked and found nothing. A missing one is a fault. The two are never spelled the same way, on either side of the pipe.

When it refuses and you did not expect it

Every refusal names its own cause. Read the first few words: they say whether something was found, or whether the screen could not run. Those are different problems.

The reference page lists every message the screen can produce, with what each one means.

Why every one of these refuses

Serving content because the check broke is the one outcome this must never produce. A sidecar that will not start, a model that will not load, an answer nobody can parse and an inference that outran its clock all refuse and say which. There is no path through it that fails any other way — including a fault nobody anticipated, which becomes a refusal too.

It reads as an outage, and that is the intended trade. A screen that quietly stopped screening would be worse in exactly the way that matters.

What it costs

• Only the categories you screen cost anything. Models load one at a time, when a category is first wanted. A tree screening only clinical loads one model, not three — and a tree screening only identifiers loads none and never starts burler at all.
• Long documents are split into overlapping windows. These models read a few hundred words at a time, so a long page is cut into pieces that overlap — which is what stops an entity sitting on a boundary from being missed by both halves.
• The cache is bounded on purpose. An unbounded one would be a memory leak wearing an optimisation's hat, and one holding the very text nobody wants held.

What it does not do

What it stops is the ordinary accident: a regulated document read into a transcript by something that was only trying to help. It is not built to withstand anybody trying.

Where the rest of it is written down