SF's Aging City Image Archive Has a Duplicate Problem. What Happens Next Could Shape Millions in Contracts.

San Francisco's Department of Technology is sitting on a backlog it has quietly acknowledged for months: thousands of duplicate images embedded across city databases, permit portals, and public-facing agency websites, costing the city measurable money in cloud storage fees and slowing down systems that residents use daily. The question now is not whether to act — city administrators have already signaled a cleanup is coming — but which departments go first, who pays for the deduplication work, and whether the fix will be contracted out or handled by in-house staff.

The stakes are real. Cloud infrastructure expenses for San Francisco's municipal technology systems have climbed steadily since 2021, when the city began migrating legacy data to hosted environments. The Department of Technology manages services across more than 50 city agencies, and duplicated image assets — everything from permit photographs filed through the SF Planning portal on Rayes Street to inspection records held by the Department of Building Inspection on Mission Street — inflate storage volumes without adding any informational value. A single unresolved deduplication cycle across even a mid-sized agency database can generate hundreds of gigabytes of redundant files.

The Decision Points Piling Up at Civic Center

Three choices are converging at once. First, city technology officials must decide whether deduplication gets bundled into the broader Digital Services modernization program already underway at City Hall, or treated as a standalone procurement. Second, the Board of Supervisors' Government Audit and Oversight Committee is expected to review departmental IT spending before the end of the third quarter of 2026, which gives budget hawks a natural opening to demand a consolidated solution rather than agency-by-agency patchwork fixes. Third, the Mayor's Office of Housing and Community Development — which maintains image-heavy databases tied to affordable housing applications and inspections across neighborhoods from the Tenderloin to Bayview-Hunters Point — is among the departments most exposed to the inefficiency.

The timing matters because San Francisco's technology budget cycle for fiscal year 2026-27 closed in June with several infrastructure line items left deliberately vague pending the outcome of ongoing vendor assessments. A deduplication contract awarded before September would qualify for the current fiscal allocation; anything later risks a funding gap or a supplemental appropriation request.

SF Digital Services, the city's in-house civic technology team based on 1 Dr. Carlton B. Goodlett Place, has been piloting automated deduplication tools on a limited set of internal SharePoint repositories since early 2026. The pilot covered roughly three city agencies and was described in internal budget documents reviewed by The Daily San Francisco as producing measurable reductions in redundant file counts, though specific figures from that pilot have not been made public. A wider rollout would require either expanding the team's capacity or bringing in outside vendors — a politically charged choice given ongoing debates at City Hall about the size of technology contracts awarded to private firms.

What the Timeline Looks Like From Here

Sources familiar with the modernization program — speaking only in their capacity as city contractors whose agreements prohibit public comment — say a department-by-department prioritization list is expected to circulate internally by late July. SF Planning and the Department of Building Inspection are both understood to be near the top of that list, given the volume of permit-related imagery those agencies process annually and the direct public-facing consequences when their search and retrieval tools run slowly.

For residents, the practical effect shows up most clearly in neighborhoods where permit activity is heaviest: the Mission District, SoMa, and the rapidly redeveloping stretch of Caltrain-adjacent blocks around 4th and King streets. Faster, leaner databases mean quicker online permit lookups and fewer system timeouts during public comment periods on development projects.

The Board of Supervisors committee review in the third quarter gives advocates for faster action a concrete deadline to push toward. If the administration has not committed a funded plan to paper by then, expect supervisors to insert specific deduplication benchmarks directly into the Department of Technology's next performance agreement — a move that would effectively take the scheduling decision out of the department's hands. The city's choices over the next 60 days will determine whether this gets treated as a serious infrastructure investment or another item that drifts unresolved into the next budget cycle.