Monday, April 16, 2012

Bayes Net Project Answers and Stats

Max Possible Points: 69

Undergrad:
 max 64.5
 min 42
 avg 56.5
 stddev ~7.2

Grad:
 max 67.5
 min 41.5
 avg 61
 std dev ~7

--

Check marks are one point, slashes are half points, and x's are no points. 
Plus and Minus are for exceptional answers.

Showing the screenshots is worth a point, except for part 3, which is worth two points, and part 2, which is worth no points.
Every `interesting' number in a CPT is worth a point.
Short answer questions are one point each.

The following is a detailed breakdown and meant only as a reference, not gospel --- it may contain errors since I wrote it from memory.

--

Question 1:
The correct numbers are straightforward, very few lost points. 
1 point for the diagram, 10 more for the CPTs, 5 for the calculations, 2 for the short answers.

Intuition:
Our belief in inferior plutonium should increase if we see that the slushies are liquified, as a possible cause of that is core meltdown, and in turn, a possible cause of core meltdown is inferior plutonium.
Of course if we directly observe a core meltdown then our belief should only be that much stronger in inferior plutonium.
At that point the state of the slushies is irrelevant, since slushie liquification is just a poor man's test for predicting whether core meltdown occurred.
Interestingly, if we know that a core meltdown has occurred and the water is poor quality, then we already have a good explanation for the core meltdown and so our belief should be somewhat lessened in the possible fault with the plutonium.
(For two things to both have gone wrong at the same time is harder to believe then just one thing going wrong!)

D-SEP: Regarding irrelevence of slushies once we know core meltdown: The formal test is to cut every path between the two nodes (with or without evidence depending on the type of path). 
There is just one path connecting IP and SL, going through CM, and the configuration is not -> CM <-.  Then knowing CM cuts that, and thus all, paths between IP and SL and we have that they are D-separated.

--

Question 2:

All but a handful got this wrong, so, a little theory first.
There are two key words: "perfect" and "exhaustive".
For a cause to be perfect it gives its effects with 100% probability, or, in propositional logic, a forward implication. 
For causes to be exhaustive means that the effect occurs with 0% probability in the absence of all causes.  In propositional logic, one could write a backwards implication (the converse), or the inverse.

So, encoding 1 (effects+converses):
 (LW or IP) <=> CM
 CM <=> GD
 CM <=> SL

Or encoding 2 (effects+inverses):
 (LW or IP) => CM
 CM => GD
 CM => SL
not (LW or IP) => not CM
 not CM => not GD
 not CM => not SL

There were 6 possible points for the encoding, 3 points for the calculations, and 2 points for the short answers.

Interpretation:
1) If SL, since causes are exhaustive, a cause had to be true, and there is only one: Core meltdown.  Again there must be a cause, but, there are two possible causes.  In the lack of any further evidence the best we can do is count those situations where IP is true versus false and infer the appropriate ratio, i.e., the best we can do is compute the probability.  Assuming you didn't change the priors, then, the number is 0.52.
2) As before, but now we have additional evidence ruling out bad water as a cause.  Left with only one possible explanation, again by the assumption of "exhaustive", it must be that the plutonium was bad.  So the probability is 1.0.
3) The probability is 0, because there is no possible world in which IP holds given the evidence.  This query is special though, because the probability of any and all queries given the evidence ~GD and SL is 0. 
By exhaustive causes and SL, CM must be true.  By perfect causes, then GD must also be true.  But since ~GD is given, there cannot be any world satisfying any property. 

Relation to Propositional Logic:
1) We could count possible worlds even in propositional logic, but, not in a `weighted' way (otherwise we are just doing straight up probabilistic propositional calculus already). 
One can get the tool to tell you what the right numbers are by giving priors on IP and LW that are fair coin flips.
2) Exactly the same as in the probabilistic case, formally: SL => CM and SL gives CM; CM => (LW or IP) and CM gives (LW or IP), finally (LW or IP) and ~LW gives IP.
3) Given a contradiction, we can infer anything we like, the opposite of the Bayes Net situation.  (In the net, IP either true or false is probability 0.  In the propositional encoding, one could infer that it is simultaneously true and false.)


One could of course still get points for shorter answers, I am just being verbose.  It was very unlikely to get points for either interpretation or relation to propositional logic if your encoding was wrong to begin with.

As a total freebie the CPTs were given 10 points for this question, where one was even permitted to change the priors around somewhat on IP and LW.

--

Question 3:

2 points is for giving GD just the parent CM, LW the parent CM, and IP the parents LW and CM.
1.5 points is for extra edges.
Less is for downright wrong networks. 

In the ideal network there are 11 probabilities that need to be assessed, and there are 11 points awarded if your network includes those values in the right places (duplicated however many times needed by redundant parents).
Note that you can use the original network as an oracle for more than just values --- you can also test conditional independence assumptions in order to figure out that GD does not need SL as a parent, yet, IP does need LW as a parent.

1 point if you said this network was worse than the original, or implied it well enough. 

1 point if you demonstrated that your network reproduced the right values.

--

Question 4:
part a) 1 point for the diagram; 5 more points for the `interesting' numbers, i.e., the CPTs that are new/different.
part b) 1 point for the diagram; 8 more points for the differing numbers. 
  I was kind here regarding rounding---not that there was a choice since only one student noticed that one needed to provide 8 digits of precision to do the question justice.
  (In general throughout the assignment any answer rounded consistently was acceptable.)

--

I was not kind regarding simple oversights, like swapping two numbers; the tool+assignment are too easy to use and complete and anyways there were tons of freebie points.


-Will


No comments:

Post a Comment