The reason for F1 rejection?
If H4 I-94 still good, then you are still in valid H4 status (unless you had out-of-status that caused F1 rejection).
If USCIS did not say they found you have out-of-status, yes you can do F1 visa in Canada, but it really depend on why you are denied change of status, because US consulars in Canada also can look up your rejection and reject you for the same reason (if the reason of rejection still exists or carries panalties)
If H4 I-94 still good, then you are still in valid H4 status (unless you had out-of-status that caused F1 rejection).
If USCIS did not say they found you have out-of-status, yes you can do F1 visa in Canada, but it really depend on why you are denied change of status, because US consulars in Canada also can look up your rejection and reject you for the same reason (if the reason of rejection still exists or carries panalties)