The Island of Destiny

You barely manage to grab the ledge again, but your hand is straining. You have to get back up on to that ledge. A rock falls down right next to your hand, smashing itself into pieces and almost making your hand flinch off the ledge. You manage to climb back up on to the ledge and lean against the rock face, breathing heavily and wondering how you survived.
After a while, you look around and notice down below an even smaller ledge, but this one is right in front of a cave. You look around some more. In the direct opposite direction is a comfortably large ledge with a note lying on it. Which one will you go to?