SF's Digital Archive Push Hits a Wall of Duplicate Images — and Officials Are Starting to Talk

San Francisco's Department of Technology is under fresh pressure to address a problem that has quietly ballooned inside the city's public-facing digital infrastructure: thousands of duplicate images clogging the records management systems used by agencies from the Planning Department on Van Ness Avenue to the San Francisco Public Library's main branch on Larkin Street. The issue has moved from a technical footnote to a policy headache, with city officials, open-government advocates, and information professionals now publicly staking out positions on how — and how fast — the city needs to act.

The timing matters. San Francisco is midway through a digitization push that city budget documents from fiscal year 2025–26 tied to a broader open-data mandate, requiring agencies to migrate legacy paper and photo records into searchable, publicly accessible repositories. That migration has exposed what archivists and records managers have long known: when you scan fast and audit slowly, duplicates accumulate. The result is bloated storage, degraded search results, and — in cases involving planning permits or code-enforcement photos — the real risk that staff pull the wrong image when making a consequential decision.

What the Experts Are Saying

Professionals in the city's library and records communities have been vocal, if not always on the record. The San Francisco History Center, housed inside the main Public Library at Civic Center, manages more than a million photographic items, and staff there have described the duplicate problem as endemic to any large-scale digitization effort rather than unique to San Francisco. The center has used deduplication protocols tied to metadata standards set by the Dublin Core Metadata Initiative since at least 2019, according to publicly available library documentation.

The San Francisco Planning Department, which relies on image archives to document site conditions for roughly 30,000 permit applications processed annually, confirmed in a March 2026 staff report to the Planning Commission that it was reviewing its document management workflow. The report did not assign a dollar figure to the problem or set a remediation deadline, but it acknowledged that redundant files were generating retrieval inconsistencies. Open-government group SF Digital Rights Coalition — which has been active on Mission Street since 2021 — has called for a citywide image-deduplication audit with results published on DataSF, the city's open-data portal.

Technology sector voices have also entered the conversation. Several AI and data-management firms based in SoMa have pitched the city on automated deduplication tools that use perceptual hashing — a technique that identifies visually identical or near-identical images even when file names differ. One vendor presentation circulated at a City Hall briefing in May 2026 cited internal pilot results showing a 34 percent reduction in redundant files within a single agency's image library over 90 days, though those figures have not been independently verified by the city.

The Practical Stakes for City Residents

For ordinary San Franciscans, the consequences are not purely bureaucratic. Neighborhood groups in the Tenderloin and in Bayview-Hunters Point have raised concerns that duplicate or mislabeled site photos contributed to delays in code-enforcement cases — a charge the Department of Building Inspection has not formally responded to as of publication. Accurate, deduplicated image records matter most when a landlord disputes a violation or a permit history is reconstructed for litigation.

The city's Department of Technology has not announced a formal deduplication program, but sources familiar with its current budget cycle say a request for proposals for a records-management platform upgrade — one that would include automated duplicate detection — is expected before the end of calendar year 2026. The DataSF team, which operates out of City Hall under the Chief Data Officer's office, has separately flagged image-record quality as a priority in its 2026 work plan, published in January.

Advocates say the window for action is now, while the digitization initiative still has momentum and before another round of fiscal tightening hits discretionary technology spending. The SF Digital Rights Coalition has urged residents to submit public comment through the City's 311 portal before the Board of Supervisors takes up the Department of Technology's supplemental budget request, expected in September 2026. Whether the Board treats image-record integrity as a funding priority or a line item to trim will likely define how quickly the duplicate backlog gets cleared.