When Does an AI Proposal Become Archival Description?

0Editorial note and method

1The disagreement is real

Whether a machine should draft descriptive text is a real disagreement, and it is not a disagreement between people who have understood these systems and people who have not. Three positions are worth separating. Each can be held firmly by a serious person, and the divisions below are the author’s way of organising the argument rather than a report of what the profession currently believes.

The first is refusal. Description is an interpretive act, performed by a qualified person, on behalf of an institution that can be held to it afterwards. A system with no professional formation and no answerability cannot perform that act, and putting its output in the institution’s voice is a category error rather than an efficiency. The profession’s own ethics do not settle that either way. The Code of Ethics treats description as an exercise of professional judgement and asks the archivist to be transparent about that role by documenting descriptive work and collections policy decisions, and it says nothing at all about machines Society of American Archivists. Refusal is compatible with it, and so is the third position below. It is not a stage institutions pass through on the way to agreeing with suppliers, and this note treats it as a legitimate outcome rather than as resistance to be managed.

The second is constrained experiment. Use these systems where the output stays close to the source and an error is cheap, visible and reversible. Transcription, handwriting recognition, candidate dates, format and language identification. Keep interpretation manual. Revisit in two years, when there is more evidence and fewer sales calls.

The third is conditional provisional use. Let a system propose descriptive text, keep the proposal marked as provisional, and require a person with the right authority to accept it before a reader ever sees it.

This note argues from the third position, and it is worth being blunt about what that costs. Conditional use puts a new load on the institution. Somebody has to define the conditions. Somebody has to hold the authority. Somebody has to have the hours. An institution that cannot fund all three is better served by the first position than by a version of the third that exists only in a policy document.

The three positions are further apart on method than on principle. Each of them can be held while accepting that a statement published under the institution’s name is the institution’s statement, whatever produced the first draft. The argument is about what has to happen in between.

One distinction has to be made before any of that can be argued, because collapsing it is what makes the rest of this note necessary. Review can fail while the proposed text is perfectly acceptable. The quality of what a system produces and the quality of the control exercised over it are separate variables, and a good rate on the first says nothing about the second. What the control is worth depends on whether the role holding it has the context, the authority and the hours to inspect an ordinary item, reject it, and send it somewhere else. This note has no evidence about how those roles are staffed in current implementations, so it treats capacity as something an institution has to specify and test rather than as something the author claims to have seen.

2“Human in the loop” is too cheap a claim

The phrase does an enormous amount of work for very little money. It asserts that a person was present. It says nothing about whether that person could have changed the outcome, and it is offered in procurement conversations as though those were the same fact.

Presence is not control. A control has parts, and every part can be missing while a person remains, technically, in the loop. Does the reviewer hold the authority to reject, or only to edit? Do they see the context needed to judge the proposal, or only the proposal itself? Do they have the competence for this particular decision, which may turn out to be a rights decision wearing a descriptive costume? Do they have the time to disagree, given the queue behind them? Are the criteria written down, or improvised per item and forgotten by Thursday? Is there any record afterwards of what they decided and why they decided it?¹

Governance work outside the sector has been explicit about this for years, and archivists should feel free to borrow the vocabulary without adopting the worldview. The European regulation treats oversight as a set of capacities held by a person: to interpret the output, to stay alert to the tendency to over-rely on it, to disregard, override or reverse it, and to interrupt or stop the system. The separate requirement that whoever exercises oversight hold the necessary competence, training, authority and support sits not in that article but in the one that binds the organisation deploying the system European Union. A cross-sector risk framework does the same work for the organisation around the person, treating documented roles and responsibilities, training, human oversight, monitoring, feedback, appeal and override as arrangements an organisation makes rather than as properties the software has NIST.

None of that is archival doctrine, none of it was written with a fonds in mind, and nothing here claims that either instrument reaches archival description. Both are imported for one narrow purpose, which is that they give names to the difference between a person being present and a person being in control.

The profession has since said something of its own. The ARA Code asks that any use of these systems in the creation, processing or presentation of records that materially affects their informational or evidential value be made transparent to users, requires such processing to be undertaken with appropriate human oversight and quality control, and leaves the member responsible for the accuracy of whatever was generated on their instruction Archives and Records Association. That is binding on the association’s individual members rather than advisory, which is worth saying plainly. What it does not do, and does not set out to do, is say what oversight has to consist of before an institution is entitled to call it oversight. Appropriate is left to the institution, which is the right place for it and also the place this note’s argument starts.

There is a second thing the phrase never says, which is who the person is. It names no role, no level of authority and no quantity of time, and none of those follow from the fact that somebody was present. Who receives the queue, and with what authority and capacity, varies by institution. This note treats the role, its authority and its capacity as things an institution has to specify and test, which is the argument for writing them down: a control that has not been assigned to anybody in particular is not a control, it is an expectation.

3Extraction is not description

Treating machine output as one category is the root mistake in this area. A transcript and a biographical note are not the same kind of claim. Reviewing them the same way wastes effort in one direction and invites harm in the other, and it leaves the institution with a risk register in which every line says “AI output”.

The useful axis is distance from the source. At one end the system is reading marks that are physically present on an object. At the other it is asserting things about people, events and significance that no amount of looking at the object would settle. Appraisal sits beyond the far end, and nothing in this note proposes moving it.

The right-hand column is the point. Those are different failure modes and they need different readers.² A misread character is caught by anyone who can read the original. A wrong authority link is caught by someone who knows the vocabulary and the collection. An invented biographical claim may not be caught inside the institution at all, because it is fluent, plausible and about a person no member of staff has any reason to know.

Titles deserve a specific note, because they are where the temptation is strongest. A title in a finding aid is not a headline. It is a required element with rules behind it: it separates a formal title, which appears on the material itself, from a devised one, and it asks for professional judgement in deciding when a title has to be devised rather than transcribed DACS, Title. The standard does not say who or what may draft the words, so a machine-drafted title is not ruled out by the rules. It is owed the same assessment against them, and against the collection the material sits in, that a devised title gets from the archivist who wrote it. The output can be grammatical, evocative and wrong at exactly the level a reader cannot check.

Published guidance for the sector goes a certain distance and then stops. Candidate titles, dates and short summaries, suggested subject terms and proposed groupings are to be treated as non-authoritative suggestions, and all outputs are to be reviewed by archivists Colavizza and Jaillant. What that leaves open is which of those outputs an institution may release provisionally, at what level of description, and against what verification burden. Those are institutional policy choices, and the rest of this note is an argument about how to make them inspectable rather than an attempt to make them on anybody’s behalf.

4The institutional acceptance boundary

Most accounts of machine-assisted description have one gate. The system writes something, a person approves it, it appears in the catalogue. One gate hides four separate decisions, and the confusion between them is where institutions get hurt.

A proposal is not a description. It is a candidate produced by a system that holds no authority. This note proposes that it be retained as generated and kept apart from whatever the accepted text becomes, so that anyone can later see what was actually proposed rather than what survived editing. No standard cited here requires that. The reason for asking anyway is that the moment a proposal is silently overwritten by its accepted form, the institution has destroyed the only evidence that would let it audit its own review.

A disposition (the recorded outcome of a review decision: accept, modify then accept, reject, defer, escalate or stop) is what a person does with that candidate. Only the first two produce an institutional statement. The rest are outcomes too, and a system that cannot record a rejection is not recording review. It is recording throughput and calling it review, which is the same substitution the whole note is about.

Acceptance is not publication. An accepted working description can still fail quality review, and it can still be held back by a rights or access decision made by a different role for different reasons. Collapsing those into one act is how a sensitivity problem becomes a public problem.

Publication is not finality either. Correction is a state in the model, not an embarrassment outside it. This part is not new and does not need arguing. The conceptual model the profession has been building treats agents, the activities they carry out, the rules they apply and the dates on which they act as entities in their own right; it treats an archival description as a record resource in its own right, produced by a describing activity, attributed to an agent, dated, revised over time, and supported by sources cited either for the whole description or for a single statement within it; and it already counts a mechanism as a kind of agent Records in Contexts. The ontology built on that model declares the mechanism a subclass of agent, held apart from a person and from a position, and describes it as something that may generate or modify records Records in Contexts Ontology. All of that supports representing what a system did. None of it hands the system any authority. The encodings go further than most people notice. A maintenance event carries its type, its date, a description of what changed and the agent responsible, with the agent type recorded as human, machine or unknown EAD3. Preservation metadata has recorded events, agents and their roles for years, software included. It comes from preservation rather than from description and says as much itself, and it is borrowed here as a working pattern rather than as a requirement that anything be evidenced PREMIS.

What the boundary names, then, is not a moment of transformation and not one universal click. A machine output becomes part of an institution’s description through one or more recorded acceptance events, governed by institutional roles and rules. Such an event may cover a single statement, a whole record, or a defined batch handled under a stated rule. It may be a disposition recorded item by item, or acceptance into a working description with quality control following separately, or a decision taken by a role rather than by a person whose name a reader ever sees, or one event in a revision history that carries on for decades. Description is cumulative, distributed, inherited and repeatedly revised, and a boundary that only worked for one archivist accepting one sentence would not describe the work.

The purpose of the event is narrow. It marks when the institution assumed responsibility for the use of an output, and it preserves enough evidence for that decision to be inspected and revised afterwards. It does not make the description final and it does not make it correct. To say that an institution has accepted a statement is to say that it will answer for it, and nothing more. It means that when a researcher writes to ask where the date came from, there is an institution on the other end of the letter rather than a system that has since been replaced. Every archivist reading this knows a catalogued statement can be wrong for fifty years. The boundary is about who owns the mistake, not about preventing it.

5Six proposed conditions for meaningful review

If presence is not control, something has to replace it. What follows is the author’s proposed test for whether review is doing institutional work. Its parts draw on archival descriptive control and on governance practice from outside the sector, and several of them have support in the sources cited elsewhere in this note, but no standard, professional body or framework has adopted these six as a set or called them a minimum. They are a proposal, and they are offered to be argued with.

Condition four is the one most easily satisfied on paper, and its failure leaves nothing behind that an interface could display. The European regulation identifies over-reliance on the output of a system as a risk that oversight arrangements have to address, which is why it asks whoever exercises oversight to remain alert to automation bias (the tendency to accept a system’s suggestion because it is the suggestion in front of you) European Union.

That is a governance framework naming a risk, which is not the same as research measuring one. This note does not have the measurement, and it will not pretend that a regulation citing a phenomenon is evidence of its size in an archival reading room.²

Two things these conditions do not settle. The first is who decides what risk is acceptable. Not the supplier. The institution decides, through its practitioners, with the specialists a particular collection requires, with the communities a record describes where they can be reached, and inside whatever legal and ethical obligations already bind it. A supplier that offers a default risk setting is offering to make that decision on the institution’s behalf, and it should be declined in those words.

The second is ethics. These are process conditions. An institution could satisfy all six and still publish a description that is accurate, authorised, fully evidenced and indefensible, because whether a record should be described in public at all, and in whose terms, is not a workflow question. The conditions make it possible to find out who decided. They do not make the decision good.

6Review depth follows use and consequence

A wrong character in a transcript inside an internal discovery aid costs a search result. The same wrong character inside a quotation on a public collection page is a misquotation attributed to the institution, and it will be cited by someone who has no way of knowing it was machine-produced. The output has not changed. The use has.

Work the case through properly, because it is the case in which the distinction does the most work. An institution runs handwriting recognition across a series of nineteenth-century letters. Accuracy is good, not perfect. Used internally, the transcripts make an unsearchable series searchable for the first time, the errors are visible to anyone who opens the image next to the text, and the cost of an error is a search that misses. That is a defensible use of an imperfect output, provided the limitations are stated and nobody quotes from it.

Now take one of those transcripts and put a sentence from it on the public page as the text of the letter. Nothing about the transcript changed. Everything about the institution’s exposure did. The reader has no image to check against, no reason to doubt it, and every reason to quote it in something else. The verification that use requires is a person reading the passage against the original, and that is a different job with a different cost, done by a different person, at a different point in the workflow.

A third case is worse than either, because it does not look like a decision at all. A system detects a personal name in a letter and links it to an authority record. The link is plausible. It is also a claim that this person, in this collection, is that person, with that life and those relationships, and it will propagate: into the authority file, into aggregated discovery, into somebody’s research, into a family’s understanding of their own history. Nobody reviewed a sentence, because there was no sentence. There was a link, offered beside the name with a tick next to it, and accepting it took a quarter of a second.

This is why a confidence measure cannot carry the control on its own. What such a measure reports varies from system to system, and may be a token likelihood, a calibrated estimate, the weight of whatever evidence was retrieved for the answer, or something else again that the number itself does not announce. Whatever it reports, it is a property of the output. It knows nothing about whether the sentence is going into a staff spreadsheet or onto a published record, and the institution’s exposure is set almost entirely by that second thing. A measure that has been validated against the institution’s own material can help decide what to look at first. What sets how hard the looking has to be is the intended use and the cost of getting it wrong.

The tiers describe review depth, not permission. Nothing here says a Tier C output may not be produced. It says that if it is produced and then used publicly, the review that accepted it has to have been proportionate to that use, and that for some decisions the proportionate answer is that the machine does not participate at all.

Rights and access are the clearest case at the far end. A rights decision is a legal and ethical judgement about a specific person’s material, taken under whatever legal and ethical obligations already bind the institution, by a role that carries the authority to take it. An automated signal can put material in front of that role, and routing material for specialist attention is a reasonable thing to ask a system to do. What such a signal cannot do is determine the decision or stand in for the authority to make it. A score that sends a file to the rights officer is doing useful work. A score that clears the file is doing the rights officer’s job without holding the rights officer’s mandate.

Three design questions remain open inside the framework proposed here. They are the author’s own, and none of them is a position held by anybody else. The first is that biographical and administrative history are better treated as two classes of output than as one. The subjects differ and so do the consequences: an administrative history of a body wound up decades ago carries a different risk from a biographical note about a living person’s father, and the table in section 3 folds the two into a single row in a way that understates the difference. The second is that provisional public access needs an explicit scope, a stated review point and a correction rule, all decided when the material is released rather than when somebody complains about it. Internal is a statement of intention rather than a property of a system, and it should not be assumed to hold indefinitely. The third is that a rights or access decision should be recorded separately from the acceptance of descriptive text and taken by the role authorised for it, because a decision offered in the same interface as a descriptive proposal, at the same moment, is being invited to inherit that proposal’s disposition. None of the three is settled here, and any product that ignores them, the author’s included, deserves the question.

7What proof looks like

The controls above are worth nothing if nobody can inspect them afterwards. So what does the evidence for one accepted descriptive statement actually look like, and how much of it is there?

Less than people expect. The proposal, retained as generated. The evidence available at the time of the decision, including what was missing. The rule, convention or local policy applied. The role that decided, the disposition, the timestamp. The exact difference between the proposal and the accepted text, and the reason recorded then rather than reconstructed for an auditor two years later. Then what happened afterwards: quality review, rights clearance, publication, and any correction event with its date and its trigger.

None of that set is required by anything cited here. No standard asks an institution to keep the proposal as generated, or the field-level difference between the proposal and the accepted text, or the reason the reviewer held at the time. Those three are the author’s proposal, and they are asked for because they are what makes a review inspectable afterwards rather than merely recorded. How much of the set is worth keeping for any one statement follows from the consequence of getting that statement wrong and the cost of holding the evidence.

Most of the categories already have somewhere to live. The descriptive standard provides an element for the sources used, the rules applied, the agent responsible and the dates of creation and revision, and it has provided for them since long before anyone needed them for this, although it offers them as added value rather than as a minimum DACS, Description Control. The older general standard set out an archivist’s note, the rules or conventions used, and the date of the description ISAD(G), which is evidence of a long-standing model of descriptive control rather than evidence that every institution filled it in. What is different now is only that the agent is two things: a proposal from a mechanism, and an acceptance by a role the institution can name.

The honest complication is cost. Field-level evidence for every accepted statement can make an assisted workflow slower than doing the work by hand, which would be a real argument against the whole enterprise. The answer is not to skip the evidence. It is to stop pretending every field needs the same amount of it. A batch of transcripts accepted under one stated rule needs one decision record for the batch and a note of the exceptions. A biographical statement about a named person needs its own.

There is a limit to what any of this can capture, and it is worth stating before somebody sells a completeness claim on the back of it. A decision record shows what the reviewer was given. It cannot show what they read. It cannot show that they opened the image, or that they knew the series well enough to notice that the date was implausible for that correspondent. The evidence is a record of the conditions of the decision, not of the attention paid to it. Aggregate monitoring is the only partial answer: if one reviewer accepts everything and another rejects a fifth of it, the difference is visible even though neither reviewer’s attention is.

The example is deliberately unglamorous. A plausible place name, wrong, corrected three months after publication by a reader who knew the building.³

8What users should be told

Internal decision evidence and public disclosure are different things, and conflating them produces both a privacy problem and a comprehension problem. A reader has no use for the ledger. What this note proposes they be told is four things, in language that does not require them to care about any of this.

That a machine contributed to a description, and roughly what it contributed. That the description is provisional, if it is. What the known limitations are, including which parts of the collection this applies to and which it does not. And how to tell the institution it is wrong, in a way that produces a response rather than a receipt.

The last one is the easiest of the four to get wrong. A correction route that nobody is funded to read is worse than no correction route, because it converts a reader who knows something into a reader who has been ignored. If the institution cannot staff the route, the honest disclosure says who to write to and how long it usually takes, which is at least true.

Coverage is the other thing to settle. A single site-wide notice saying the institution uses these tools cannot be resolved to the description in front of the reader, and read strictly it covers every description in the catalogue, including the ones written by people over the last forty years. For some institutions that will be the level they can actually maintain, and it is better than silence. Where machine assistance has materially affected a particular description, this note proposes disclosure at the finest level the institution can keep true, which is usually the record.

Proportion matters in the other direction too. What a reader can use is that a machine drafted the scope note and that a member of staff accepted it under a stated rule. Whether the individual who accepted it is named in public is a separate question, and it has different answers in different descriptive environments: some traditions have signed descriptions for a century, and in other settings naming one member of staff moves correspondence that belongs to the institution into a particular person’s inbox. The distinction underneath is the one to hold to. Internal evidence names a person, because answerability inside an organisation requires it. Public disclosure names the institution, because that is who is answerable outside it.

9A procurement test

The argument reduces to questions that can be asked in a meeting by an archivist, a director, a rights officer or a technical reviewer. They name no product, and they apply as well to a system an institution builds for itself as to one it buys. They should be asked of every supplier, including the one that published this note.

Ask them in writing and keep the answers. The answers that should worry a committee are the confident ones with no detail. That the system is designed with a human in the loop. That everything is logged. That accuracy is high. None of those answers any question above.

The last question is the test. The first seven can be answered from a slide deck. The eighth requires that somebody kept the evidence, that a wrong proposal was recorded rather than tidied away, and that whoever built the system is willing to walk an institution through a case where its own output was wrong. Where there is no permission to show a customer’s material, a controlled demonstration answers it honestly. What cannot be substituted is the wrong proposal, what the reviewer had in front of them at the time, and the disposition that was recorded.

10Conclusion

Human judgement does not make description correct. It never has. Archivists have been wrong about dates, names, provenance and significance for as long as there have been finding aids, and what the standards built in answer to that was a way of documenting who described what, under which rules, and when, so that the next person can find the error and fix it. That the means exist is not the same as saying every institution has used them, and the standards make no such claim either.

That is the whole claim in this note. Judgement does not confer accuracy. It locates responsibility in an institution that can be asked to explain itself and argued with by the people a record affects. A machine proposal is not description. It becomes description when an authorised role accepts it, on the record, with the evidence retained, and it stays contestable afterwards.

The framework here is the author’s proposal, published in a form that invites checking. Where it is wrong about practice, the practice is right, and the note lets that be established in public rather than settled in private.

Open review

This note is a synthesis rather than a professional standard, and it is published to be checked. If something in it is wrong, or imprecise, or missing a piece of context a practitioner would treat as essential, the address is [email protected]. Substantive corrections are recorded in the change log with the date the change was made, and credited by name where the person who sent them is content to be named.