San Francisco's Duplicate Image Problem: Why Bad Data Costs Real People Real Services

City departments across San Francisco are sitting on a growing backlog of duplicate photographs and image files embedded inside public-facing databases, a problem that is quietly degrading the speed and accuracy of services that low-income residents depend on most. The issue spans at least three major municipal systems, including the Department of Homelessness and Supportive Housing's client intake portal, the SF Human Services Agency's CalFresh enrollment pipeline, and the planning department's permit management software. Each has been flagged internally for carrying redundant image records that slow processing times and, in some cases, generate case-matching errors.

This is not an abstract technology complaint. When a caseworker at a Tenderloin navigation center tries to pull up a client's profile and the system loads four versions of the same intake photograph before resolving to the correct file, that delay ripples outward. Appointments get missed. Benefits renewals stall. In a city where the fentanyl crisis has compressed the window of time in which outreach workers can keep someone engaged, a thirty-second system lag can translate into a lost connection.

How Duplicate Images Enter the System

The problem typically begins at the point of intake. When a resident applies for emergency housing assistance through the Coordinated Entry System — administered locally by the Department of Homelessness and Supportive Housing out of its offices on Otis Street in SoMa — staff photograph applicants using tablets that sync to a central server. If the upload fails mid-transfer and the worker retries, the server frequently stores both the failed partial file and the completed image. Multiply that across dozens of daily intakes at sites from the Civic Center area to the Bayview, and the redundant files accumulate fast.

At the SF Human Services Agency, which processes CalFresh applications for hundreds of thousands of San Francisco residents, a similar dynamic plays out when applicants upload identification photos through the BenefitsCal portal. The agency's own technology staff have documented the issue in internal review cycles, though the department has not publicly committed to a timeline for a full deduplication sweep. The planning department faces a parallel headache: permit applicants routinely upload site photographs multiple times to navigate an upload interface that provides no clear confirmation, leaving planning staff to manually sort through stacks of redundant files before they can advance a review.

What It Costs Residents in Practice

Deduplication is solvable. Perceptual hashing — a technique that generates a compact numerical fingerprint from an image and flags near-identical copies — has been commercially available since the early 2010s and is now standard in major cloud storage platforms. Several Bay Area companies, including startups based in the Mid-Market tech corridor, offer enterprise-grade versions that can process millions of images for under $10,000 annually. The City of Seattle completed a comparable database cleanup for its permitting system in 2024, reducing average permit review times by roughly 18 percent, according to that city's published performance data.

San Francisco's Department of Technology, headquartered on Stevenson Street, has authority over citywide data infrastructure standards and has in recent years pushed agencies toward the DataSF platform for public-facing datasets. Whether that mandate extends to internal image stores within social services software is a question the department has not publicly resolved. Advocates at Tenderloin Housing Clinic and Causa Justa, both of which help clients navigate city benefit systems, have noted anecdotally that case-processing delays are frequent, though pinpointing duplicate images as a specific cause requires access to backend logs that advocates do not have.

For residents, the practical advice is limited but real: when uploading documents or photographs to any San Francisco city portal, wait for an explicit on-screen confirmation before clicking submit again. Resubmitting without confirmation is the single most common way duplicate files enter the system. Caseworkers at Mission Neighborhood Resource Center and Hamilton Family Center have begun walking clients through this step-by-step. City officials have not announced a formal remediation program, but the Department of Technology has indicated it plans to publish updated data quality standards for agency software procurement before the end of fiscal year 2026 — a deadline that lands in just a few weeks.