When we left EDI, we knew the street we needed to turn into, but we passed by it 3 times and still could not turn into - there were like 4 or 5 streets converged at an angle, and if you were not in the correct lane, impossible to turn... Finally we decided to just follow Airport as we know the general direction was correct, and got out of town.
We passed by the castle in your last picture.
P.S. Dont like EDI Castle. If you really enjoy old structures, you should visit York, Chester (they are in England) and Stirling (Scotland).