Probabilities if held through expiration yes. But the binaries can/will trade very differently before that.
I check the prices all the time. I like being able to just look at a number and say Oh, there's a 60% the S&P will be above 1100 in a month and a half, or a 20% chance it will be under 900 in three months. If I ever find those probabilities to be out of whack, they are going to be very easy to arb.
Before Taleb re-branded himself as a philosopher and his ego was still a manageable size, he wrote a pretty good book that covered some of the pricing issues of binary and other exotic options: Dynamic Hedging More recently, there has been a flurry of binary betting/binary options books over the last few years. A quick search on Amazon will reveal.