SF's Duplicate Image Problem: What Happens Next and the Key Decisions Ahead

San Francisco's public agencies are sitting on a mounting crisis hiding in plain sight: tens of thousands of duplicate digital images clogging government servers, slowing down databases, and quietly inflating IT storage costs at a moment when the city's technology budget is already under strain. The San Francisco Department of Technology, which oversees citywide digital infrastructure, has been working through a backlog of redundant image files across multiple departments — a problem that has grown steadily since the city's pandemic-era push to digitize everything from permit records to homeless services case files.

The issue matters now because several major decisions are converging simultaneously. The city's fiscal year 2026–27 budget, which took effect July 1, includes consolidated IT line items under Mayor Daniel Lurie's administration following London Breed's departure from office. Department heads have until September 30 to submit proposals for new storage contracts, and the window for adopting a unified digital asset management platform — something city technology officials have studied for at least three years — is finally open. Miss it, and the next opportunity likely doesn't arrive until fiscal year 2028.

Where the Backlog Lives — and Why It Got This Bad

The problem is concentrated in a handful of departments. The San Francisco Planning Department, headquartered at 49 South Van Ness Avenue, maintains image libraries tied to tens of thousands of active permits. Staff upload site photos, architectural renderings, and inspection images through a legacy portal that has no automatic deduplication. The same image can exist in three or four project folders simultaneously. At the Department of Homelessness and Supportive Housing, which manages shelter intake records and Navigation Center documentation across sites including the 5th Street Navigation Center in SoMa, duplicate intake photos have created compliance headaches under state record-retention rules.

Nonprofits under city contract face the same tangle. Tenderloin Housing Clinic, which manages hundreds of units of supportive housing across the Tenderloin and Civic Center corridors, uses image documentation for unit inspections and tenant records. Staff have flagged that their city-required reporting templates force re-upload of images already housed in shared drives, creating redundancy by design rather than accident.

Cloud storage is not free. Enterprise-tier object storage through major vendors currently runs between $0.02 and $0.023 per gigabyte per month — modest per unit, but it compounds fast when duplicate files number in the hundreds of thousands. A 2024 audit by the Controller's Office found that the city's overall data storage costs had grown more than 40 percent over the prior three fiscal years, though that figure covers all data types and was not broken out by image files specifically.

The Decisions That Will Define the Fix

Three choices are now on the table, and city technology officials will likely need to signal their preference before the Board of Supervisors returns from recess in mid-August.

First: whether to run a retroactive deduplication sweep on existing archives or treat legacy files as a sunk cost and focus on preventing new duplicates going forward. The retroactive approach requires dedicated contractor hours and carries a risk of deleting files still needed for active litigation or open public records requests. The forward-only approach is cheaper short-term but leaves the backlog festering.

Second: whether to mandate a single citywide digital asset management platform or allow individual departments to procure their own solutions. The SF Digital Services team, based at City Hall, has piloted a shared platform with two smaller agencies. Scaling it to Planning or HSH — both of which have complex, legally sensitive image libraries — is a different order of magnitude.

Third: whether BART and Muni, which maintain their own image archives for infrastructure inspections along the Market Street corridor and beyond, are brought into any citywide solution or left to manage separately. Both agencies receive city funding but operate under distinct governance structures, complicating any unified mandate.

Advocates in the city's civic tech community, including groups that work with Code for San Francisco, have pushed for a public comment period before any vendor is selected. That call is unlikely to slow the procurement timeline — but it could shape which requirements end up in the final contract. Agencies that want their priorities included should be engaging with the Department of Technology before the August recess ends, not after.