Computational results
Twelve searches of the two halves of the Farnese letter recovered the same eighteen letter assignments and the null. The whole-letter search recovered them in two of three restarts. These are results from EHum's public transcription, before the later manuscript corrections.
This supplement gives the methods, recorded results, and qualifications behind the article. Search results contains one row for each of 32 recorded restarts, including unsuccessful and overfitted keys. A score ranks candidate keys under a particular model and search design; it is not a probability that a reading is correct.
Exploratory searches
The initial tests asked how to divide the digits into codes. Separating at
particular digits, taking pairs at fixed even or odd positions, and requiring
particular digits to begin every pair did not give a consistent solution.
For example, division at 0 produced 869 segments with a mean of 6.6 digits,
but 12% had only one digit and the longest had 41. Frequent two-digit sequences
could be letter codes or adjacent letters; their frequency alone did not
decide between the two.
The search also tried designs represented in Aloys Meister's Die Geheimschrift (1906), especially the papal keys on pp. 176–179:
| Design tested | Recorded outcome |
|---|---|
| Single-digit polyphony: one digit can represent several letters | Restarts generally disagreed and did not produce readable Italian. One configuration repeated a key, but its reading was still not Italian. |
| A frequency-derived inventory of about fifty units, assigned to letters, syllables, or words | Readings overused frequent units such as e, di, ce, and per. Restricting the units to letters did not solve the text. |
| Variable-length polyphonic and syllabic keys | Optional nulls and syllables could improve the numerical score without producing a coherent reading. |
| Mignanello-type design: paired vowels and single-digit polyphonic consonants | Trials with and without 3 as a null gave unreadable text and disagreeing restarts. |
| One code of one or two digits per letter | Recovered the eighteen letter assignments shown below. |
These were exploratory searches with changing implementations, inventories, and scoring rules. Their negative results describe the searches made; they do not exhaustively exclude the unsuccessful designs.
The first successful configuration still allowed optional word and syllable
units. Four of its six restarts, numbered 0, 3, 4, and 5, agreed on the
eighteen letters and 9 as the null. The first opening was
dopo_lapartitadelmontepinmcian_il_manon, with _ representing the null.
The same keys assigned 81 to la and used other optional units that damaged
the reading. Their common letter assignments were more informative than those
extra units.
Two further configurations show why a better score need not mean a better
key. The three-null configuration used 9, 0, and 7 as nulls and allowed
second codes for letters. It scored −0.777 per retained digit while producing
worse text. The dotted, unspaced homophone configuration scored −0.687 and
−0.685, also with extra assignments that spoiled the reading. Its denominator
was 6,533 retained digits because it kept marked digits as distinct symbols.
These values cannot be treated as a controlled comparison with the tightened
search's −0.783.
Training corpus
The normalized corpus contained 4,954,638 letters in 19,054 rows. A row was an extracted text line, not necessarily a complete paragraph or document.
| Source group | Contents | Normalized letters before the split | Training letters |
|---|---|---|---|
| Italian Wikisource | Machiavelli, Il Principe, Lettere, Discorsi sopra la prima Deca di Tito Livio, and Istorie fiorentine; Castiglione, Il libro del Cortegiano; Vasari, Vite (1550) | 4,224,884 | 4,177,843 |
| Cardauns, Nuntiaturberichte aus Deutschland, I.7 (1912) | Italian-looking lines selected from OCR of correspondence and related material from 1541–44 | 729,754 | 721,950 |
| Total | 4,954,638 | 4,899,793 |
The Cardauns volume is Berichte vom Regensburger und Speierer Reichstag 1541, 1542. Nuntiaturen Verallos und Poggios. Sendungen Farneses und Sfondratos 1541–1544. It includes correspondence of the Farnese secretariat but does not print the April letter studied here. Extraction selected Italian-looking lines by a language heuristic; it did not identify the author of each passage. The Wikisource metadata does not establish the underlying print edition for every work.
Normalization lower-cased the text, removed accents, folded v and w to
u, j and y to i, k to c, and x to s, deleted h, and collapsed
repeated identical letters. Punctuation and other non-letter characters became
spaces. Only normalized rows longer than ten characters were retained. A
random row shuffle with seed 1 reserved 200 rows, containing 54,845 letters,
and left 4,899,793 letters for training. Of the reserved letters, 7,804 came
from Cardauns.
The corpus was not deduplicated. An audit found that 16 of the 200 reserved
rows also occurred in training after normalization and removal of spaces.
Duplicate literary passages, OCR damage, German editorial text, and Wikisource
metadata remained. In particular, 694 rows beginning dc title contained
281,860 letters, 5.7% of the corpus. The audit checked exact row matches, not
near-duplicates or overlap between whole documents.
The resulting split is not an independently held-out corpus. This affects the language-model evaluation and the synthetic controls. The duplicate rows do not establish that the target letter was in training. No cleaned-corpus retraining and recovery test was made, so its effect on the search outcome remains unmeasured.
Language model
The model predicts a symbol from the preceding four symbols: a character
five-gram. Its alphabet consists of nineteen letters, abcdefgilmnopqrstuz,
and one boundary symbol. For each context, it blends the observed next-symbol
distribution with the shorter-context estimate. If a context has total count
N and T distinct next symbols, the observed distribution receives weight
N / (N + T). The remaining weight goes to the shorter-context estimate;
the base estimate is uniform. This is interpolated Witten–Bell smoothing.
Two versions used the same row split. The boundary-trained version retained word spaces as boundary symbols. The unspaced version removed word spaces before training but retained beginning- and end-of-row sentinels. It therefore learned cross-word letter sequences such as those in delquale. The tightened search used the unspaced model. All 3,200,000 combinations of four context symbols and the next symbol were represented by float32 base-10 log probabilities, allowing one table lookup per emitted letter.
A later audit evaluated the unspaced model on the 184 reserved rows that did not have an exact training match: 41,153 letters. Each row began with an empty context. The word and letter shuffles used seed 2, with one shuffle of each row; scores were summed over letters before averaging.
| Evaluation input | Mean log10 probability per letter | Perplexity |
|---|---|---|
| Original text | −0.749528 | 5.617 |
| Words shuffled within each row | −0.883132 | 7.641 |
| Letters shuffled within each row | −2.105830 | 127.594 |
| Original text scored with a unigram baseline | −1.170894 | 14.822 |
Perplexity is 10 raised to the negative mean log10 probability. The model preferred the original to the word shuffle in 160 of 184 rows, and to the letter shuffle in all 184, with no ties. Metadata and possible near-duplicates remained in these evaluation rows. These are predictive scores per plaintext letter, distinct from the key-search scores per retained ciphertext digit.
Key search
An outer simulated-annealing search changed the key. For each candidate key, an inner beam decoder searched for a division of the digit stream into one- and two-digit codes. A decoder state held its digit position, the last four emitted symbols, and its accumulated score. At the same position and context, only the highest-scoring path survived; at most eight different contexts survived at each position. This approximation can discard a path that later context would favor.
The search began with nineteen mandatory letter slots, including q, and
an optional null. Mandatory describes the inventory, not a known assignment:
the searches began from random keys without fixed letter assignments. The
clerk's recovered alphabet uses eighteen letters; the spare q slot is not
evidence for a nineteenth plaintext letter.
| Tightened-search setting | Value |
|---|---|
| Candidate codes | Single digits and pairs occurring at least three times in the scored fragments |
| Candidate inventory, whole letter | 108 codes |
| Input cuts | At above-dot/comma tokens and unreadable tokens; page boundaries also end fragments |
| Mark treatment | 178 above-dots and 25 above-commas trigger cuts; four above-dashes do not |
| Unreadable positions | 44; one also has an above-dot/comma mark |
| Retained input | 6,331 digits; 219 scored fragments containing 6,305 digits |
| Short fragments | 26 retained digits lie in fragments shorter than three and are not scored |
| Beam width | Eight contexts per digit position |
| Proposal attempts | 500,000 per restart |
| Move choice | Swap two active assignments, 45%; move a slot to an unused code, 45%; deactivate an optional unit, 10% |
| Worse-key acceptance | exp(Δ/T), where Δ is the change in total score |
| Cooling | Geometric, from 30 toward 0.5; invalid proposals still advance the schedule |
| Null penalty | 1 per occurrence; no emitted letter and no context reset |
| Undecodable-digit penalty | 4, only when neither the next single digit nor the available pair has an assignment |
| Word and syllable units | None |
| Full-text search seed | 11; three restarts, numbered 0–2 |
| Recorded CPU time | Roughly 38–39 seconds per restart on the original machine |
Initialization favored frequent candidate codes. Optional units began inactive; a move to an unused code could activate one. Each scored fragment began with a fresh language-model context. The key score was the sum of its best surviving fragment scores, including the penalties. Reported scores divide by the retained-digit count, including the short unscored fragments. The CPU times are observations from one machine, not a portable benchmark.
Variable-length decoding with Italian five-grams, logarithmic scores, and spelling normalization already appears for known keys in Lasry, Megyesi, and Kopal, “Deciphering papal ciphers from the 16th to the 18th Century”, §6.10, pp. 522–524. Here the outer search also changed the unknown key.
Recovery and controls
| Input | Restarts | Recovery of eighteen letters and null | Total score of best key | Score per retained digit | Retained digits |
|---|---|---|---|---|---|
| Whole letter | 3 | 2 of 3 | −4957.0 | −0.783 | 6,331 |
| First half, ff. 70r–71r | 6 | 6 of 6 | −2014.2 | −0.777 | 2,591 |
| Second half, ff. 71v–73r | 6 | 6 of 6 | −2936.5 | −0.785 | 3,740 |
| Clean synthetic cipher | 3 | 3 of 3 | −3881.2 | −0.630 | 6,165 |
| Noisy synthetic cipher | 3 | 3 of 3 | −4235.8 | −0.692 | 6,119 |
| Shuffled real ciphertext | 2 | Neither gives the recovered key | −8553.5 | −1.351 | 6,331 |
The third whole-letter restart stopped at −7278.8, or −1.150 per retained
digit. Each half was searched separately with seed settings 11 and 23 and
three restarts per setting. All twelve agreed on the eighteen letters and
9 as null. The spare q slot took 77 in the first half and 81 in the
second; it took 81 in the successful whole-letter restarts. The contextual
reading of 81 as et was established after the letter search.
Stable letter key and known synthetic keys
Every letter row below was recovered in both successful full-text restarts, all twelve half-text restarts, and all three restarts of the corresponding synthetic case. The et row is different: it gives the contextual assignment for the letter and the known encoding assignment for each synthetic cipher, not a recovered word unit. Leading zeroes are significant.
| Unit | Farnese letter | Clean synthetic truth | Noisy synthetic truth | Search status |
|---|---|---|---|---|
| a | 5 |
6 |
5 |
Recovered |
| b | 72 |
61 |
72 |
Recovered |
| c | 27 |
57 |
16 |
Recovered |
| d | 73 |
64 |
18 |
Recovered |
| e | 8 |
8 |
9 |
Recovered |
| f | 37 |
60 |
21 |
Recovered |
| g | 75 |
70 |
31 |
Recovered |
| i | 6 |
9 |
3 |
Recovered |
| l | 03 |
78 |
53 |
Recovered |
| m | 3 |
1 |
1 |
Recovered |
| n | 7 |
3 |
7 |
Recovered |
| o | 4 |
7 |
4 |
Recovered |
| p | 57 |
83 |
78 |
Recovered |
| r | 2 |
0 |
2 |
Recovered |
| s | 0 |
4 |
8 |
Recovered |
| t | 80 |
46 |
17 |
Recovered |
| u / v | 1 |
5 |
6 |
Recovered |
| z | 07 |
74 |
34 |
Recovered |
| null | 9 |
2 |
0 |
Recovered optional unit |
| et | 81 |
41 |
45 |
Not represented as a word unit in the tightened search |
The synthetic comparisons therefore recovered 19 of 20 known undotted
assignments: eighteen letters and the null, with et missed. The unused
q slot took 28 in the clean case and 09 in the noisy case, neither of
which is the known et code.
The synthetic encoders also had the following dotted assignments. A caret follows the digit carrying the dot. These assignments were excluded from the recovery comparison because marked digits cut the search input.
| Unit | Clean synthetic truth | Noisy synthetic truth |
|---|---|---|
che, represented as ce after normalization |
7^3 |
3^7 |
chi, represented as ci after normalization |
73^ |
37^ |
| qua | 4^5 |
7^1 |
| que | 45^ |
71^ |
| qui | 6^0 |
1^3 |
| non | 60^ |
13^ |
Construction and limits of the controls
The clean and noisy synthetic ciphers used different invented keys, generated
with seeds 1 and 2 respectively. Nine letters received distinct single digits;
the remaining digit was the null. Nine more letters and et received distinct
pairs of unequal non-null digits. The dotted inventory is shown above.
The encoder inserted a null after each word with probability 0.33 and between
identical consecutive undotted codes. It used the dotted non and undotted
et for whole words, qua/que/qui for the corresponding sequences, and
the normalized ce code for ce. Words it could not encode were skipped.
The source words came from the 200 reserved corpus rows. As the later audit found duplicate text in training, these controls did not use wholly unseen Italian. They demonstrate recovery of the recorded invented assignments under these conditions, not performance on an independent document-level test set.
The clean cipher has 6,287 digit positions. The noisy cipher has 6,285
positions, including 44 made unreadable. Noise independently selected each
digit for substitution with probability 0.012, choosing among a hand-set
list of confusions, such as 1/2, 3/5/7, and 0/8/9; unreadable
positions were then selected. The clean and noisy ciphers differ in key as
well as noise, so their score difference does not isolate the effect of
transcription damage.
The shuffled input was a permutation of the real transcription's 6,577 digit-position tokens, including marks and unreadable positions. Its shuffle seed was not recorded. Its two resulting keys shared nine of twenty assignments, including the null, but neither gave readable Italian. This is one shuffled-input control, not a calibrated null distribution or a significance test.
All these recovery tests share the Italian model and its biases. The split test is the strongest internal check because different portions of the letter yield the same letter assignments and null without sharing their ciphertext. It does not validate every code division or editorial conjecture. Recorded deterministic reruns of the tightened whole-letter search and seven control settings reproduced their results.
Search-results table
search-results.tsv preserves the recorded scores at their original precision: one decimal for totals and three for scores per retained digit. Ratios recomputed from the rounded totals may differ in the last place.
| Column | Meaning |
|---|---|
experiment |
Descriptive identifier: discovery, three-null, homophones, whole-letter, half-1, half-2, synthetic-clean, synthetic-noisy, or shuffled |
seed |
Search seed setting when explicitly documented; NA otherwise. Synthetic key-generation seeds are separate. |
restart |
Zero-based restart number within the setting |
total_score |
Best recorded penalized score in that restart |
score_per_retained_digit |
Recorded total divided by the retained-digit count |
retained_digits |
Denominator used by the search, including any short unscored fragments |
scored_fragments |
Number of fragments actually decoded |
letter_slots |
The eighteen mandatory letter-slot assignments, in code=letter notation; these slots were searched, not fixed in advance |
q_slot |
The nineteenth mandatory slot, kept separate because the recovered alphabet does not use a separate q |
optional_units |
All active optional assignments, including nulls (_), words, syllables, and second letter codes; NA means none |
For the homophone and three-null configurations, a correct assignment can appear in an optional second letter slot rather than the mandatory slot. The table preserves that distinction and the order of optional assignments. It does not relabel exploratory word assignments as established cipher codes.
Vision pilot
On 15 September 2026, Grok 4.6, GPT-6 Astra, and Gemini 3.1 Pro, each at a
high reasoning setting, read four short unlabeled image strips. Three strips
came from lines whose glyph counts matched the transcription one for one;
the fourth included the known 2-for-1 transcription error in Montepulciano
on 70r. The strips contained 55 reference digits in total.
The reported digit-error range was roughly 12–18%. The models confused the
same forms that troubled the transcription: hooked 1s, joined digits,
and 3/5/7 shapes. Some responses inserted or omitted glyphs or treated
bleed-through as dots.
The scoring allowed fractional credit for alternative readings and free extras at crop edges. The reference largely followed the public transcription. The prompts, individual responses, scoring implementation, and image crops are not available in this supplement. The record supports a reported pilot, not a reproducible comparative benchmark. In particular, the article's count of automatic edit proposals is not a measured human transcription-error rate against which these model results can be compared.
Glyph classification
The glyph classifier compared each doubtful digit with the clerk's other examples. It first leveled each cipher line, removed its slant, and proposed cuts at blank gaps and thin joins. A dynamic-programming alignment matched transcription tokens to image intervals using cut costs, per-digit width priors, and, after an initial width-only alignment, shape scores. Penalized events allowed a token without its own ink or ink without a token. Three bootstrap rounds refined the alignment; disputed reference labels were filtered during refitting.
The main ink components of each crop were scaled to 24 pixels within a 32 × 32 field and centered by their center of mass. Features combined 64 principal components of the pixel image with five geometric measurements: width, height, top, bottom, and ink area in x-height units. A seven-neighbor classifier used distance-weighted votes. The current line was excluded from the reference-neighbor lookup, and an outlier cost discouraged stains and fragments from receiving confident labels.
| Evaluation against provisional transcription labels | Glyphs | Agreement |
|---|---|---|
| Current line excluded from neighbor lookup | 6,497 | 90.549% |
| Blank cuts on both sides, current line excluded | 2,859 | About 95.6% |
| Joined to a neighbor, current line excluded | 3,638 | About 86.6% |
| Vote share at least 0.9, current line excluded | 4,485 | 98.885% |
| Current folio excluded from neighbor lookup | 6,497 | About 89.7% |
The high-vote subset is 69.03% of the labeled glyphs. These are agreement rates against provisional labels, not accuracy against independently verified digits. Principal-component fitting and segmentation used the full dataset; line or folio exclusion applies to neighbor selection. Vote shares are not calibrated probabilities that the manuscript digit is correct.
All 196 suspects with a vote-share difference of at least 0.5 were examined on review sheets, as were the 37 flagged tokens without ink and the 26 glyph-sized stretches without tokens. The sheets included the surrounding line, seven reference glyphs, and the edition's proposed reading; review was therefore not blind to linguistic context. Some flagged events occupied the same stretch rather than representing separate errors.
Across the broader set of 212 suspect-type decisions, the reviewer preferred the transcription in 120 cases, the classifier in 47, a third digit in 14, and left 31 open. Agreement with the classifier alone was not sufficient to change the transcription. The review found five duplicated groups totaling 21 digits, fourteen omitted digits, and two swapped pairs, among other corrections. Overall, 29 deletions and fourteen insertions changed the transcription's count from 6,577 to 6,562. The article and edition document the adopted readings.
Not every glyph accepted by the classifier received an individual visual check. Blots, overwriting, the archive stamp, and page-edge smears leave readings open. The classifier's agreement rates cannot certify the remaining digits or settle whether a difficult reading is a transcription error, a clerk's error, or an editorial mistake.
Blind rereads
On 15 September 2026, two fresh GPT-6 Astra readers received separate packets: one for 71v and one for 72r, the two pages with the most editorial departures. Each received the key and a literal digit-to-unit decoding in thirteen segments, with instructions to propose a continuous reading and account for every substantive departure. Neither received the edition, the error-tolerant reading, other folios, historical identifications, or manuscript images. Each first response was preserved before comparison with the edition; neither reader was then shown the edition or asked to revise it.
| Folio | Material supplied | First response |
|---|---|---|
| 71v | Reading packet | Reading and apparatus |
| 72r | Reading packet | Reading and apparatus |
The published packets retain the task, key, digit rows, and notation, with
execution-specific wording removed. The response bodies preserve the first
readings, including mistakes. They record proposed readings of the public
transcription before the subsequent manuscript check, rather than the final
edition. The 72r packet retains a copy typo (o for digit 4) together with
the correction given to the reader before its response.
The independence was limited to the subsequent reading task. Both readers used GPT-6 Astra, the same key and transcription, and the language model's selected strict segmentation. The packets omitted unreadable tokens and short fragments, used the first listed transcription alternatives, represented above-dots and above-commas alike, and omitted above-dashes and uncertainty flags. A segment break therefore did not preserve the number of missing digits. The readers could reconsider code boundaries but did not always do so. Neither had seen the manuscript images.
On 72r, the reader recovered verà la magior piena exactly from twenty digits that the edition had partly absorbed into a gap. Restoring verrà made the wholly supplied [che potrà] unnecessary. On 71v, o[l]tre che replaced the edition's S.M.tà s[a] che: the new reading needs an ignored dot and one missing digit, and fits the following statement that the Pope's forces are also divided. Both revisions were adopted.
The responses also contain mistaken segmentations, unaccepted conjectures, and readings later superseded by the photographs. Their broader conjecture brackets are not directly comparable with the edition's letter-level brackets, and no agreement percentage is claimed. The rereads provided alternatives to check against the digits; they did not independently recover the key or verify the manuscript.