Plan to visit the Victoria Island in CA, need I to ferry the car

We plan to take BC ferry to visit Victoria Island, just checked the fee of the ferry, it seems it will take around $100 for two-way to take the car with us in the ferry.

So I want to know whether it deserved. Without car, can we visit the Victoria Island with the public transportation? Is it convenient? We plan to live in Victoria for one night.

Thanks,