The Victorian government has distanced itself from the stalled Williamstown Swimming and Life Saving Club redevelopment despite being responsible for the bulk of its funding.