San Francisco's Digital Archive Problem: By the Numbers, Duplicate Images Are Costing City Agencies Millions in Storage

San Francisco's city government is sitting on a digital storage problem it has spent years ignoring. A review of municipal data practices, compiled by the city's Department of Technology in the spring of 2026, found that duplicate image files — photos, scanned documents, and surveillance stills stored repeatedly across departmental servers — account for an estimated 23 percent of total municipal cloud storage consumption, a share that translates directly into recurring annual contract costs that city budget analysts peg in the hundreds of thousands of dollars.

The timing matters because San Francisco is not flush. The city entered the current fiscal year carrying a projected deficit of roughly $800 million, forcing cuts across departments from the Department of Public Health to Recreation and Parks. Every dollar spent storing a photograph that already exists three times over in a different folder is a dollar that cannot go toward street-level services in the Tenderloin or shelter beds in SoMa.

Where the Redundancy Lives

Two agencies stand out in the Department of Technology's internal review as the heaviest contributors to duplicate image accumulation. The San Francisco Municipal Transportation Agency, which operates Muni bus cameras, rail platform surveillance, and traffic-monitoring equipment across roughly 700 route miles, generates image data at a scale that dwarfs most city departments. Separately, the Office of the Assessor-Recorder on Van Ness Avenue maintains digitized property records that stretch back decades, and staff there have long flagged that scanning workflows routinely create multiple copies of the same document page without any automated deduplication step.

The Planning Department, headquartered on Mission Street, faces a related challenge. Environmental review files for major development projects — think the proposed housing towers slated for Parkmerced and the ongoing Central SoMa build-out — contain hundreds of site photographs that get uploaded by multiple consultants, stored in project management software, and then re-uploaded into the city's public-facing portal. Each upload survives independently. No system currently checks whether the incoming file matches one already on the server.

The Department of Technology's review did not put a single department on notice, but it did identify deduplication software licensing as a gap. The city currently holds no enterprise-wide contract for image deduplication tools, relying instead on ad hoc storage purchased through agreements with vendors including Microsoft Azure and Amazon Web Services. Storage costs per terabyte have dropped considerably over the past decade, but volume growth has outpaced price declines — meaning the city's annual bill keeps rising even as the per-unit cost falls.

What the Data Actually Shows

Across San Francisco's 57 departments and offices, total cloud storage consumption crossed 4.2 petabytes as of the Department of Technology's March 2026 inventory — up from 2.8 petabytes in 2023. Image files, including still photographs, scanned PDFs, and video stills, represent the fastest-growing category, expanding at roughly 34 percent annually. If the 23 percent duplication estimate holds, the city is effectively paying to store approximately 966 terabytes of data it already has.

Enterprise deduplication solutions used by comparable large municipalities — San Jose has deployed one since 2024, and Denver rolled out a citywide image deduplication protocol in late 2023 — typically reduce redundant storage by 40 to 60 percent within 18 months of deployment. Applied to San Francisco's current figures, a conservative 40 percent reduction in duplicate image volume would free roughly 386 terabytes of storage, potentially saving the city between $150,000 and $280,000 annually depending on vendor pricing, according to cost modeling outlined in the Department of Technology review.

The Board of Supervisors Budget and Legislative Analyst's office is expected to take up the Department of Technology's findings during budget hearings scheduled for later this month at City Hall. Staff analysts are weighing whether to recommend a centralized procurement process for deduplication software or to push individual departments to fund their own solutions — a distinction that will determine how quickly, and how evenly, the cleanup happens.

For residents trying to make sense of the city's fiscal triage, the practical upshot is straightforward. San Francisco's ability to reduce waste in its own back office — the unglamorous, invisible kind measured in redundant gigabytes rather than ribbon cuttings — will factor into how much room the city has to restore services cut during deficit reduction. The audit gives the Board of Supervisors a concrete place to start.