1Lecture 17 • 16.825 Techniques in Artificial IntelligenceWhere do Bayesian Networks Come From?So now we know what to do with a network if we have one; that is, how to answer queries about the probability of some variables given observations of others. But where do we get the networks in the first place?2Lecture 17 • 26.825 Techniques in Artificial IntelligenceWhere do Bayesian Networks Come From?• Human expertsWhen Bayesian networks were first developed for applications, it was in the context of expert systems. The idea was that “knowledge engineers”, who were familiar with the technology, would go and interview experts and elicit from them their expert knowledge about the domain they worked in. So the knowledge engineer would go talk to a physician about diagnosing diseases, or an engineer about building bridges, and together they would build a Bayes net.3Lecture 17 • 36.825 Techniques in Artificial IntelligenceWhere do Bayesian Networks Come From?• Human expertsIt turned out that this was pretty hard. We’ve already seen that humans aren’t very good at probabilistic reasoning. It turns out that it’s also hard for them to come up with good probabilistic descriptions of what they know and do. One way to make the problem easier for humans is to give them some fixed structures for the kinds of relationships they can express among the variables. We’ll talk a little bit about that in this lecture.4Lecture 17 • 46.825 Techniques in Artificial IntelligenceWhere do Bayesian Networks Come From?• Human experts• Learning from dataBut the big thing that has happened recently in the Bayes net world is a move toward learning from data. Given example cases in a domain: patients or loan applications or bridge designs, we can use learning techniques to build Bayesian networks that are good models of the domain.5Lecture 17 • 56.825 Techniques in Artificial IntelligenceWhere do Bayesian Networks Come From?• Human experts• Learning from data• A combination of bothOf course, there are other learning algorithms available. But one of the great strengths of Bayesian networks is that they give us a principled way to integrate human knowledge about the domain, when it’s present and easy to articulate, with knowledge extracted from data.6Lecture 17 • 6Human Experts• Encoding rules obtained from experts, e.g. physicians for PathFinderInterviewing humans and trying to extract their expert knowledge is very difficult. Even though we may be experts at a variety of tasks, it’s often hard to articulate the knowledge we have.7Lecture 17 • 7Human Experts• Encoding rules obtained from experts, e.g. physicians for PathFinder• Extracting these rules are very difficult, especially getting reliable probability estimatesHumans are reasonably good at specifying the dependency structure of a network, but they are particularly bad at specifying the probability distributions.8Lecture 17 • 8Human Experts• Encoding rules obtained from experts, e.g. physicians for PathFinder• Extracting these rules are very difficult, especially getting reliable probability estimates• Some rules have a simple deterministic form:AgeLegal DrinkerSometimes the relationships between nodes are really easy to articulate. One example is a deterministic form. So, whether someone can drink alcohol legally is a deterministic function of the person’s age (and perhaps their country or state of residence).9Lecture 17 • 9Human Experts• Encoding rules obtained from experts, e.g. physicians for PathFinder• Extracting these rules are very difficult, especially getting reliable probability estimates• Some rules have a simple deterministic form:AgeLegal Drinker• But, more commonly, we have many potential causes for a symptom and any one of these causes are sufficient for a symptom to be trueThere are other structured relationships that are relatively easy to specify. One is when there are many possible causes for a symptom, and it is sufficient for at least one of the causes to be true in order for the symptom to be true.10Lecture 17 • 10Multiple Independent CausesFeverFluMalariaColdImagine that there are three possible causes for having a fever: flu, cold, and malaria. This network encodes the fact that flu, cold, and malaria are mutually independent of one another.11Lecture 17 • 11Multiple Independent CausesFeverFluMalariaColdIn general, the table in the Fever node gives prob of fever given all combination of values of Flu, Cold and Malaria P(Fev | Flu, Col, Mal)Big, and hard to assessIn general, the conditional probability table for fever will have to specify the probability of fever for all possible combinations of values of flu, cold, and malaria. This is a big table, and it’s hard to assess. Physicians, for example, probably don’t think very well about combinations of diseases.12Lecture 17 • 12Multiple Independent CausesFeverFluMalariaColdP(Fever | Flu) = 0.6P(Fever | Cold) = 0.4P(Fever | Malaria) = 0.9In general, the table in the Fever node gives prob of fever given all combination of values of Flu, Cold and Malaria P(Fev | Flu, Col, Mal)Big, and hard to assessIt’s more natural to ask them individual conditional probabilities: what’s the probability that someone has a fever if they have the flu? We’re essentially ignoring the influence of Cold and Malaria while we think about the flu. The same goes for the other conditional probabilities. We can ask about P(fever | cold) and P(fever | malaria) separately.13Lecture 17 • 13Noisy Or ExampleP(Fever | Flu) = 0.6P(Fever | Cold) = 0.4P(Fever | Malaria) = 0.9FeverFluMalariaColdNow, the question is, what can we do with those independently specified conditional probabilities? They don’t by themselves, specify the whole CPT for the fever node.14Lecture 17 • 14Noisy Or ExampleP(Fever | Flu) = 0.6P(Fever | Cold) = 0.4P(Fever | Malaria) = 0.9We are assuming that the causes act independently, which reduces the set of numbers that we need to acquireFeverFluMalariaColdOne way to think about this is that P(fever | flu) is a probability that describes an unreliable connection between flu and fever. If the patient has flu, and the connection is on, then he will certainly have fever. Thus it is sufficient for one connection to be made from a positive variable into fever, from any of its causes. If none of the causes are true, then the probability of fever is assumed to be zero (though it’s always