Too small to fully experience the exciting college experience (~200 new students per year! - much smaller than a small high school)
Although the admission rate is historically low (even lower than MIT; but yield rate is much lower than MIT's. That means, Caltech is not the #1 choice for majority of the kids admitted by Caltech).