Legal technical debt piles up unnoticed and only shows up as distorted data. Here is what that looks like up close: one boolean field in the schema, 142 records, and a system that confidently reported untruths for half a year.
We build a copyright-protection platform. The far end of the process is collecting money from a won court case through bailiff offices. This is a pipeline: request the writ of execution, receive it, send it, serve it, open enforcement proceedings, then either money or an appeal against inaction. Every step has its own document, its own deadline, and its own chance of stalling.
In the summer we rewrote the data model behind this pipeline. Below is what was broken, which forks in the road we hit, and what the mistakes cost.
A flag instead of history
Three problems in the original schema — all the same class.
The proceeding was unique. The block for appealing a bailiff's inaction sat as flat fields at the root of the recovery document: court, judge, dates, result. A second proceeding for the same recovery overwrote the first.
This is not an edge case. Appeals happen repeatedly: file a complaint, get a form-letter response, file again. The schema allowed exactly one attempt, and the entire previous history vanished when a new one was entered.
Rulings lived wherever they landed. Opening, refusal, closure of proceedings — these are separate documents with scans. The scan sat in cloud storage, the date in a legacy field, and there was no version history. Re-upload a corrected scan and the previous one is gone.
Refusal to open proceedings was a boolean flag. Here is where it gets interesting. The field refusal, 142 records set to true. We looked at those 142 records and saw that 104 of them already had an enforcement case number.
So in 104 of the 142 cases the refusal had long been overcome: the specialist fixed the defect, resubmitted, the enforcement proceeding was opened, work is underway. The flag stays and keeps signaling. Every morning the digest carefully reported problems that do not exist.
The status machine nobody turned off
Statuses were a separate story. The pipeline was designed as an honest pipeline with transitions. In practice, legacy handlers wrote boolean flags past the status field.
What the inventory found:
- Over four thousand recoveries with status "closed" but an uncleared "in progress" flag. Two sources of truth, both in use, both lying in different ways.
- Three
enumvalues that no document matched. Dead branches that the code handled diligently and templates covered with conditions. - The same outcome recorded as two strings: "Refusal" and "Refused". 75 and 422 records respectively. Any query by outcome silently dropped part of the data.
The last item is the clearest. Nobody typed this by hand — at different times, different handlers simply wrote their own string. The divergence grew on its own: between the inventory and the migration rollout, two more "Refusal" records appeared.
How we fixed it
Three layers, bottom up.
Layer one: proceedings as an array
Flat fields were replaced with an array of subdocuments. Proceeding type, court, judge, dates of sending, filing, and registration, rulings, result. The API: create a case plus targeted endpoints for each field, all through atomic assignment using the positional operator.
The backfill is idempotent with a mandatory dry run. The old flat fields became "case number one"; the fields themselves stayed in place untouched.
This is an important fork. Twenty-six legal document generators read the flat fields. Rewriting them in the same release means mixing the data migration with the consumer migration and losing the ability to roll back. We kept dual writes and connected the generators later, through a separate dispatcher: proceeding data is overlaid onto the document in memory, and the generators themselves did not change by a single line.
An array of subdocuments rather than a separate collection — because a recovery has a handful of proceedings, they are always read together with the parent, and positional access removes the transaction question.
Layer two: rulings as an append-only log
An array of rulings on the recovery. Each one is a logical document with a type (opening, refusal, closure, other) and a nested array of versions. Files in object storage, the current version is the last one, writes only through atomic append. No deletion.
No deletion is a deliberate choice. A bailiff's ruling is an external legal fact. It can be quashed, contested, replaced — but it cannot disappear from history, because our own documents and deadlines reference it.
A contested ruling links to the proceeding by reference. The date is mirrored into the legacy field — again for the sake of the same generators.
We deliberately kept the foundation thin: deadlines are not baked into it. The bet paid off — the next release added calculation of the re-presentation deadline for the writ of execution without touching the foundation by a single line. The deadline depends on the grounds for closing the enforcement proceeding: two months or six, pinned to the last day of the month. Until then, people worked it out in their heads.
Layer three: refusal becomes an event
This was a real fork. Three options for what to do with the refusal flag.
A. Make refusal a terminal status. It fits the pipeline neatly, but requires a separate return branch.
B. Keep the boolean flag and add logic around it. Cheap.
C. Refusal is a ruling in the log, and the "refusal is active" attribute is computed. More expensive, the schema gets more complex.
The data settled the argument. 104 of 142 refusals are already overcome. So refusal is not a final state by nature — it is an event in the life of a recovery, like any other ruling. A terminal status would describe the rare case as the normal one.
We picked option C. The refusal-active attribute stopped being stored and became computed: for new records, from the rulings log; for the 142 legacy records, by the rule "flag set, no enforcement case number, case not closed".
Separately: we did not migrate those 142 records. A computed rule is cheaper and more honest than a migration that can get edge cases wrong and cannot be rolled back.
A status-change log and a single write helper at every call site came along for the ride. It contains an idiom worth stealing: keepIf — update the status only if the new value does not roll progress backwards. Legacy handlers now write the flag and the status at the same time, so the desync stopped growing.
What went wrong
Regression twenty hours later
The first release with the proceeding array passed review and went to production. Twenty hours later, the modal for adding an enforcement proceeding started returning 500.
The cause: the backfill wrote null into the case-type field — and null was not in the enum. Over two thousand records became invalid on update.
The lesson in one line: if an enum field has default: null, then null must be explicitly in the list of allowed values. The schema said nothing, because the write went through an operation that bypasses validators. Hotfix the same day.
Three Mongoose mines
All three were found in review, before production. Writing them down because each one costs hours.
The global option for disabling timestamps in bulkWrite is silently ignored. It has to be set on each operation individually. We did not know that, and updatedAt on over two thousand records was overwritten with the migration date. We checked — nobody reads the field, no harm done. But this is exactly the case where "harmless" is something you learn after the fact.
A pipeline update does not coerce types and treats strings with a dollar sign as field paths. If you want to write a literal string, wrap it in $literal. Otherwise you get the value of a field instead of the string, quietly and without an error.
updateOne with a filter {_id: undefined} coerces to an empty filter and updates the first document in the collection. Not zero documents. Not an error. The first document it finds. This is the worst of the three, because on an empty test database it does not reproduce.
A smoke test found a hole in the spec
After the status layer shipped we ran a browser smoke test. It turned out the canonical final status had appeared in the model, but the list filter had no option for it. A user cannot filter completed cases by the new status.
The spec did not cover it, review did not catch it, tests checked the model rather than the interface. Found in five minutes of manual clicking. Follow-up the same evening.
A delayed echo eleven days later
The most instructive one. Eleven days after the release, it turned out the main list table was still reading the legacy flat fields. Cases with new proceeding blocks had five court columns standing empty.
This is the direct price of dual writes. They give a safe migration — but every reader migrates separately, and until it does, it shows the old picture. A popular model has more readers than an inventory reveals.
Had I done this over, I would attach an explicit list of every reader of the field to the migration plan, found with grep, with a checkbox next to each. Not "migrate the model" but "migrate the model and these eleven places".
Results
Proceeding backfill: 2,144 cases created, of which 286 for appealing decisions, zero failures, counters reconciled against the database.
The "last action date" formula learned to read the new array. Until then it understated the date for 621 of the 1,003 cases under appeal, by more than a month for 466 of them. After: zero understated.
The applied effect of this one fix: operational false positives from the stalled-case detector fell from 18 a day to zero, and the "Inaction" section of the morning digest shrank from about 950 lines to 601.
The second number matters more than the first. Nobody reads a 950-line digest. A 601-line digest is honestly still too large — but that is a question of thresholds, not of data lying.
The "Refusal"/"Refused" divergence was eliminated by a migration, both strings collapsed into one. The growth of desync between the status and the boolean flags stopped via co-writing: zero new contradictions on the control checklist.
142 zombie refusal flags replaced by a computed attribute; 104 of them correctly stopped signaling.
What's left
An honest list, because an article about technical debt without a section on our own technical debt would be hypocrisy.
The refusal scan from the old storage is not linked to the ruling in the log — only the date gets written to the canonical record. Clicking the refusal marker again creates duplicates.
The date mirror does not update if you re-issue an already linked ruling: you have to reopen the select. A file over fifty megabytes returns a 500 instead of a clear error.
The debt calculation formula lives in three places and is edited in sync. A comment about this stands in all three — which is not a solution but an acknowledgment.
The mapping from the grounds for closing an enforcement proceeding to the re-presentation deadline awaits legal verification. Public sources confirm it weakly. The change is a one-line constant, but it cannot be touched without a lawyer's sign-off.
Takeaways
A boolean flag is lost history. If an entity can be overturned, overcome, or repeated, it is an event, not a state. An event is written to a log; a state is computed from the log.
Data settles modeling arguments better than intuition. We could have argued for a week over whether refusal is terminal. One query against production closed the question in a minute: 104 of 142.
Dual writes are the right migration tool and at the same time a source of delayed bugs. Use them, but produce the full list of readers in advance.
Do not migrate data where a computed rule will do. A migration is irreversible; a rule is a one-line edit.