For the coin flip scenario, (without loss of generality suppose we got more heads than tails) what I would naturally do is calculate: the probability of getting at least that many heads, given: we know a certain number of coin flips occurred and assuming the probability of a getting a heads on a single coin flip to be .5.
Then if that probability is super low, I would suspect the coin is unfair, it wouldn't give me certainty that the coin is unfair (because after all, it's possible to have something unlikely occur), but this would give me reason to inspect the coin to know for sure. And if I wanted certainty, the only way to know (and be 100% certain) whether the coin is fair is to examine its shape and weight and the coin tossing mechanism.
I'm not sure if my natural way of reasoning above follows the logical probability or Bayesian way of doing it. That would be something like this:
The setup would be
M = "a coin flip must result in either heads or tails, the physical properties of the coin and the mechanism of flipping the coin determine the result of a flip. These properties can be designed in such a way that influences the result of the coin flip",
H = "the probability of a coin flip resulting in heads p = .5",
E = "An experiment is conducted consisting of flipping a coin some number of times, N, and the number of coin flips resulting in heads is counted, X, where N >= 0, and 0 <= X <= N",
C(n) = "number of coin flips conducted in the experiment N = n",
D(x) = "number of coin flips resulting in heads X = x".
The thing you really want to know is Pr(H | D(x) C(n) E M).
Applying Baye's theorem twice:
Pr(H | D(x) C(n) E M) = Pr(H D(x) | C(n) E M) / Pr(D(x) | C(n) E M)
= Pr(D(x) | H C(n) E M) * Pr(H | C(n) E M) / Pr(D(x) | C(n) E M).
Calculating Pr(D(x) | H C(n) E M) is done using the binomial distribution since each coin flip is a Bernoulli trial. Pr(D(x) | H C(n) E M) = (n choose x)*(p^x)*((1-p)^(n-x)) = (n choose x)*(p^n) (after simplifying since p = .5).
But how do we calculate Pr(H | C(n) E M)? I don't know. I believe it cannot be calculated.
Calculating Pr(D(x) | C(n) E M) also leaves me puzzled. Based on E, the number of heads obtained could be any integer on the inclusive interval from 0 up to n. Any value of x satisfying "0 <= x <= n" is equally possible given the setup of the experiment, so if you apply the statistical syllogism, then Pr(D(x) | C(n) E M) = 1/n. I'm not sure if this is right though.
So I'm left with: Pr(H | D(x) C(n) E M) = (n choose x)*(p^n) * Pr(H | C(n) E M) / Pr(D(x) | C(n) E M).
I think there is something wrong with the question in the first place if we can't actually calculate the probabilities. What exactly do we mean by a "fair coin"?
Please let me know if you think I'm right or wrong here or if you disagree with the propositions I chose for the setup.