San Francisco Chooses Between Speed, Accuracy in Records Overhaul

San Francisco's Department of Technology is sitting on a backlog. Somewhere inside the city's centralized digital asset management system-used by agencies from the San Francisco Public Library to the Planning Department on Van Ness Avenue-thousands of duplicate and corrupted image files have accumulated over more than a decade of piecemeal digitization. Now, with a records modernization initiative moving into its next phase this summer, city staff face a cluster of decisions that will determine how the archive looks, who can access it, and what it will cost taxpayers to fix.

The timing matters because the city is mid-cycle on a broader infrastructure push. The SF Digital Services office, which sits under the Department of Technology at 1 South Van Ness, has been consolidating legacy systems since 2022, and image asset quality has emerged as a persistent sticking point. Duplicate records inflate storage costs, create legal ambiguity around official documents, and slow down public records requests-a pressure point the City Attorney's office has flagged repeatedly in recent budget cycles.

The Problem Behind the Problem

Duplicate images aren't just a housekeeping annoyance. For the Planning Department, which processes hundreds of conditional use permit applications per year and relies on photographic evidence in its case files, a degraded or duplicated image attached to the wrong parcel record can stall a project for weeks. The Mission District, where permit disputes over new construction regularly attract community scrutiny, has seen cases where document image quality became a contested issue in neighborhood review hearings.

The SF Public Library's San Francisco History Center, located at the main branch on Larkin Street in the Civic Center, maintains a separate but related challenge: its digitized photograph collection contains an estimated 200,000 items, and librarians have flagged duplication rates as high as 15 percent in some legacy batches-meaning potentially 30,000 files require review, re-tagging, or deletion. That figure comes from internal library planning documents circulated ahead of the department's fiscal year 2025-26 budget presentation to the Board of Supervisors.

The costs are not trivial. Cloud storage for municipal digital assets in a system of this scale typically runs in the range of tens of thousands of dollars per year just for the redundant files-budget that city officials say could be redirected toward expanding public access terminals at branch libraries in the Excelsior and Bayview neighborhoods, both of which serve high proportions of residents without reliable home internet.

The Decisions Coming This Fall

Three choices are now on the table, according to the Digital Services office's publicly posted project roadmap updated in June 2026. First, the city must decide whether to run automated deduplication software-faster and cheaper, but prone to false positives that could delete legitimate archival variants-or conduct a hybrid process with human review, which costs more and takes longer. Second, officials must set a cutoff date: records created before January 1, 2010 may be treated under a different retention standard than more recent files, which has implications for what the public can later request under the California Public Records Act. Third, the department must decide whether to open a public comment period before finalizing the new image standards, a step that community archivists and organizations like the San Francisco Heritage, based in the Haas-Lilienthal House on Franklin Street, have pushed for in writing.

The Board of Supervisors' Government Audit and Oversight Committee is scheduled to take up the broader digital records modernization report at a hearing expected in September. That hearing will likely set the political temperature for how aggressively the city moves. Supervisors representing districts with active land-use battles-including District 9, covering the Mission, and District 10, covering Bayview-Hunters Point-have a direct constituent interest in clean, accessible permit records.

For residents and researchers, the practical upshot is this: if you have a pending public records request touching property records, historical photographs, or permit documents filed before 2015, expect possible delays this fall as the system migration proceeds. The SF Public Library's History Center has advised patrons to complete high-priority research requests before October, when the deduplication work is expected to begin in earnest. The city's choices in the next 90 days will shape how reliably San Francisco's institutional memory holds together-and how quickly a clerk on Seventh Street can pull up the right file on the first try.