Probably yes. Check BC Transit for bus routes, schedule and map.
http://www.bctransit.com/regions/vic/
We found Route 75 and Route 72 to go to Butchart Garden from the Inner Harbor for our trip next month.
http://www.bctransit.com/regions/vic/schedules/schedule.cfm?line=75
http://www.bctransit.com/regions/vic/schedules/schedule.cfm?line=72
We are goind to Victoria from Port Angeles (close to Olympic NP); we also decided to leave our car at Port Angeles and stay one night at Victoria Inner Harbor. Since we want to Butchart Garden in mid afternoon and stay there late, we don't want to be bounded by the Grayline Express Shuttle schedule. We found Route 75 which goes directly from the Inner Harbor to Butchart Garden. For our return trip at night, we will take Route 75 and then Route 72 to get back to Inner Harbor.