Duplicate Images in City Databases Are Costing San Francisco Residents Real Money and Real Services

San Francisco's network of city agencies is sitting on a largely invisible problem: thousands of duplicate images lodged inside municipal databases, from the Department of Homelessness and Supportive Housing's client intake files to BART's clipper card photo verification system. The redundancy sounds like a bureaucratic footnote. It isn't. It inflates storage costs, slows case processing, and — in the worst instances — causes workers to open the wrong resident's file entirely.

The issue landed back on the agenda this spring after the city's Department of Technology published its annual infrastructure audit, which found that unstructured data — a category that includes scanned documents and photographs — now accounts for a disproportionate share of server overhead across city departments. Duplicate image files are among the leading contributors to that load, according to the audit, which covers fiscal year 2025-26. The department did not publish a dollar figure for image-specific storage waste, but industry benchmarks suggest municipal governments routinely overspend by 15 to 30 percent on storage when deduplication protocols are absent.

Where the Problem Shows Up on the Ground

Residents dealing with the city's housing systems feel the friction most directly. The Tenderloin-based HomeFirst intake center on Turk Street and the SoMa navigation center on Bryant Street both run off digital case management platforms that pull resident photographs from city-linked databases. When a resident has been photographed at multiple intake points — a common occurrence given San Francisco's fragmented shelter network — duplicate images create mismatched records that staff must manually reconcile before a bed can be assigned or a benefits application can move forward.

At Glide Memorial Church on Ellis Street, which operates one of the Tenderloin's busiest daily service programs, volunteers and partner staff routinely encounter the downstream effects: a client whose file shows two different profile photos from two different intake events, flagged for manual review by an algorithm that can't determine which is current. Each manual review adds time. In a system already strained by caseload, that time compounds.

The San Francisco Human Services Agency, which administers CalFresh and Medi-Cal enrollment locally, has worked since 2023 to integrate a unified ID-photo system across its Mission Street offices. That effort reduced some duplicate-image creation at the front end, but legacy records pre-dating 2023 remain largely uncleaned. The department has not announced a remediation timeline.

Why Deduplication Matters Beyond Storage Costs

The practical stakes extend beyond server bills. When SFMTA's Clipper START discount transit program — which serves low-income riders with reduced fare access — enrolls a new applicant, staff cross-reference photo IDs against existing records. Duplicate or conflicting images can trigger a hold on an account while identity is re-verified, sometimes delaying card activation by a week or more. For a Bayview or Excelsior resident who depends on Muni to reach a job across the city, a week's delay is not an abstraction.

The technology to address this is not new or particularly expensive. Image hashing and perceptual deduplication tools have been widely deployed in the private sector for years. Several Bay Area nonprofits that manage large client databases — including those working under contract with the Department of Public Health — have already implemented basic deduplication pipelines at the application layer, cutting redundant records by roughly 40 percent in pilot programs run during 2024, according to information shared at a city vendor summit held in March 2025 at City Hall.

The city's Department of Technology has flagged image deduplication as a priority item in its current two-year digital services roadmap, which runs through June 2027. Residents who encounter processing delays they believe stem from duplicate or mismatched records can file a data correction request directly through SF.gov's resident services portal, or in person at the city's 311 service center on Frank Ogawa Plaza — though the latter requires a Muni or BART trip to Oakland, which is its own irony for transit program applicants caught in the loop. The more immediate fix, city officials have signaled, depends on departmental budgets surviving the next round of cuts — a question the Board of Supervisors is expected to take up in September.