San Francisco's Digital Archives Are Full of Duplicate Images — and Officials Say the Cleanup Is Overdue

San Francisco's municipal databases contain tens of thousands of duplicate digital images — repeated photographs, scanned permits, and redundant case files spread across at least a dozen city departments — and the people paid to manage them say the problem has reached a breaking point. The Department of Technology has flagged the issue in internal reviews, and advocates working on housing and homelessness services say the redundancy is creating real-world delays in case processing.

The timing matters. The city is mid-way through a court-monitored homelessness settlement that requires faster case documentation, and the Mayor's Office of Housing and Community Development is processing hundreds of applications monthly under its Small Sites Program, which helps nonprofits acquire rent-controlled buildings across neighborhoods like the Tenderloin and Excelsior. When intake workers upload photographs of units or client files and encounter duplicate-flagging errors, processing slows — sometimes by days.

What the Experts Are Saying

Database administrators and digital records specialists contacted by The Daily San Francisco describe the duplicate image problem as structural rather than accidental. The issue traces back to at least 2019, when multiple city departments migrated legacy systems onto a shared cloud infrastructure managed through a contract with a third-party vendor. At the time, deduplication protocols were not standardized across agencies. The result: departments including the Department of Building Inspection, the Human Services Agency, and the Office of Economic and Workforce Development each maintained separate image libraries that began generating overlapping records.

Digital records specialists who consult with Bay Area municipalities — though not the city directly — say the standard fix involves deploying perceptual hashing tools, software that identifies visually similar images even when file names differ. Several such tools are commercially available for under $50,000 annually for an enterprise license, a cost experts say is modest against the operational drag of leaving duplicates in place. San Francisco's Department of Technology did not respond to a request for comment by publication time.

At the ground level, staff at Tenderloin Housing Clinic on Turk Street — one of the largest providers of single-room occupancy housing management in the city — have flagged internal workflow slowdowns tied to document-management systems that share infrastructure with city portals. Separately, case managers at Episcopal Community Services, which operates multiple Navigation Centers including one near 8th and Brannan in SoMa, have described similar friction when uploading client intake photographs required under the city's Coordinated Entry system.

A Fix That's Been Promised Before

This is not the first time the city has acknowledged the problem. A 2023 audit by the Budget and Legislative Analyst's Office identified data redundancy as a contributing factor in delays across several permitting workflows, though duplicate images were not the sole cause cited. The city allocated $2.1 million in the fiscal year 2024-2025 budget for broader data infrastructure improvements, but sources familiar with the spending say the deduplication piece was deprioritized as the Department of Technology focused on cybersecurity hardening following a 2024 ransomware incident that affected several municipal systems.

Supervisor-level discussions at City Hall have touched on the issue in budget committee hearings this spring, with members of the Board of Supervisors pressing the Department of Technology on timelines for a comprehensive records modernization plan. No formal ordinance has been introduced, but the Budget and Legislative Analyst's Office confirmed it has been asked to prepare an updated assessment, expected before the end of the third quarter of 2026.

For residents and community organizations that interact daily with city digital systems — submitting housing applications through the DAHLIA portal, filing complaints with the Department of Building Inspection via its online interface on Sansome Street, or uploading documentation for small business permits — the practical advice from records management professionals is straightforward: submit images in standardized formats, specifically JPEG or PDF-A, and retain local copies of every file number and confirmation receipt. Until the city completes its cleanup, resubmission delays remain a real possibility, and having documentation on hand is the fastest way to resolve them.