Abstract
Inductive Logic Programming (ILP) is an area of Machine Learning which has now reached its twentieth year. Using the analogy of a human biography this paper recalls the development of the subject from its infancy through childhood and teenage years. We show how in each phase ILP has been characterised by an attempt to extend theory and implementations in tandem with the development of novel and challenging real-world applications. Lastly, by projection we suggest directions for research which will help the subject coming of age.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Bain, M., & Muggleton, S. H. (1991). Non-monotonic learning. In D. Michie (Ed.), Machine intelligence (Vol. 12, pp. 105–120). London: Oxford University Press.
Blockeel, H., & De Raedt, L. (1997). Lookahead and discretisation in ILP. In N. Lavrač & S. Džeroski (Eds.), LNAI: Vol. 1297. Proceedings of the seventh international workshop on inductive logic programming (pp. 77–84). Berlin: Springer.
Blockeel, H., De Raedt, L., Jacobs, N., & Demoen, B. (1999). Scaling up inductive logic programming by learning from interpretations. Data Mining and Knowledge Discovery, 3(1), 59–93.
Bratko, I. (2010). Discovery of abstract concepts by a robot. In LNAI: Vol. 6332. Proceedings of discovery science 2010 (pp. 372–379). Berlin: Springer.
Bratko, I., Muggleton, S. H., & Varsek, A. (1991). Learning qualitative models of dynamic systems. In Proceedings of the eighth international machine learning workshop, San Mateo, CA. San Mateo: Morgan-Kaufmann.
Bratko, I., Leban, G., & Žabkar, J. (2008). An experiment in robot discovery with ilp. In Proceedings of the 18th international conference on inductive logic programming (ILP 2008). Berlin: Springer.
Buntine, W. L. (1994). Operations for learning with graphical models. Journal of Artificial Intelligence Research, 2, 159–225.
Chen, J., Muggleton, S. H., & Santos, J. (2008). Learning probabilistic logic models from probabilistic examples. Machine Learning, 73(1), 55–85. doi:10.1007/s10994-008-5076-4.
Cohen, W. (1993). PAC-learning a restricted class of logic programs. In S. Muggleton (Ed.), Proceedings of the 3rd international workshop on inductive logic programming (pp. 41–72).
Corapi, D., Russo, A., & Lupu, E. (2010). Inductive logic programming as abductive search. In Technical communications of ICLP’10 (pp. 54–63).
Craven, M., & Slattery, S. (2001). Relational learning with statistical predicate invention: Better models for hypertext. Machine Learning, 43(1/2), 97–119.
Cussens, J. (2001). Parameter estimation in stochastic logic programs. Machine Learning, 44(3), 245–271.
Davis, J., & Domingo, P. (2009). Deep transfer via second-order markov logic. In Proceedings of the twenty-sixth international workshop on machine learning (pp. 217–224). San Mateo: Morgan Kaufmann.
De Raedt, L. (1997). Logical settings for concept-learning. Artificial Intelligence, 95(1), 197–201.
De Raedt, L. (2008). Logical and relational learning. Berlin: Springer.
De Raedt, L., & Bruynooghe, M. (1991). Clint: a multistrategy interactive concept-learner and theory revision system. In Proceedings of the 1st international workshop on multistrategy learning (pp. 175–191). San Mateo: Morgan Kaufmann.
De Raedt, L., & Kersting, K. (2004). Probabilistic inductive logic programming. In S. Ben-David, J. Case, & A. Maruoka (Eds.), Lecture notes in computer science: Vol. 3244. Proceedings of the 15th international conference on algorithmic learning theory. Berlin: Springer.
De Raedt, L., & Lavrač, N. (1996). Multiple predicate learning in two inductive logic programming settings. Journal on Pure and Applied Logic, 4(2), 227–254.
De Raedt, L., Kimmig, A., & Toivonen, H. (2007). ProbLog: a probabilistic Prolog and its application in link discovery. In R. Lopez de Mantaras & M.M. Veloso (Eds.), Proceedings of the 20th international joint conference on artificial intelligence (IJCAI-2007) (pp. 2462–2467).
De Raedt, L., Frasconi, P., Kersting, K., & Muggleton, S. H. (Eds.) (2008). LNAI: Vol. 4911. Probabilistic inductive logic programming. Berlin: Springer.
Dehaspe, L., & Toivonen, H. (2001). Discovery of relational association rules. In Džeroski, S., & Lavrač, N. (Eds.), Relational data mining (pp. 189–212). Berlin: Springer.
Dietterich, T., Domingos, P., Getoor, L., Muggleton, S. H., & Tadepalli, P. (2008). Structured machine learning: the next ten years. Machine Learning, 73(1), 3–23. doi:10.1007/s10994-008-5079-1.
Dolsak, B., & Muggleton, S. H. (1992). The application of Inductive Logic Programming to finite element mesh design. In S. H. Muggleton (Ed.), Inductive logic programming (pp. 453–472). London: Academic Press.
Domingos, P. S., Kok, S., Poon, H., Richardson, M., & Singla, P. (2006). Unifying logical and statistical ai. In Proceedings of the twenty-first national conference on artificial intelligence, AAAI06 (pp. 2–7). Menlo Park/Cambridge: AAAI Press/MIT Press.
Džeroski, S., & Lavrač, N. (Eds.) (2001). Relational data mining. Berlin: Springer.
Džeroski, S., Muggleton, S. H., & Russell, S. (1993). Learnability of constrained logic programs. In Proceedings of the European conference on machine learning (pp. 342–347). London: Springer.
Džeroski, S., De Raedt, L., & Driessens, K. (2001). Relational reinforcement learning. Machine Learning, 43(1/2), 5–52.
Emde, W., & Wettschereck, D. (1996). Relational instance-based learning. In Proceedings of the 13th international machine learning conference (pp. 122–130).
Esposito, F., Laterza, A., Malerba, D., & Semeraro, G. (1996). Refinement of Datalog programs. In Proceedings of the MLnet familiarization workshop on data mining with inductive logic programming (pp. 73–94).
Feng, C. (1992). Inducing temporal fault diagnostic rules from a qualitative model. In S. H. Muggleton (Ed.), Inductive logic programming. London: Academic Press.
Flach, P. (1993). Predicate invention in inductive data engineering. In P. B. Brazdil (Ed.), Lecture notes in artificial intelligence: Vol. 667. Machine learning: ECML-93 (pp. 83–94). Berlin: Springer.
Flach, P. A., & Kakas, A. C. (Eds.) (2000). Abduction and induction: essays on their relation and integration. Dordrecht: Kluwer Academic.
Getoor, L., & Taskar, B. (Eds.) (2007). An introduction to statistical relational learning. Cambridge: MIT Press.
Getoor, L., Friedman, N., Koller, D., & Pfeffer, A. (2001). Learning probabilistic relational models. In Džeroski, S., & Lavrač, N. (Eds.), Relational data mining (pp. 307–335). Berlin: Springer.
Graham, J. H., Page, C. D., & Kamal, A. H. (2003). Accelerating the drug design process through parallel inductive logic programming data mining. In Proceedings of the IEEE computer society bioinformatics conference—CSB (pp. 400–402). New York: IEEE Press.
Horsch, M., & Poole, D. L. (1990). A dynamic approach to probabilistic inference using Bayesian networks. In Proc. sixth conference on uncertainty in AI, Boston, July 1990 (pp. 155–161).
Inoue, K. (2004). Induction as consequence finding. Machine Learning, 55, 109–135.
Inoue, K., Furukawa, K., Kobayashiand, I., & Nabeshima, H. (2010). Discovering rules by meta-level abduction. In L. De Raedt (Ed.), LNAI: Vol. 5989. Proceedings of the nineteenth international conference on inductive logic programming (ILP09) (pp. 49–64). Berlin: Springer.
Kersting, K., & De Raedt, L. (2001). Towards combining inductive logic programming with bayesian networks. In LNAI: Vol. 2157. Proceedings of the eleventh international conference on inductive logic programming (pp. 118–131). Berlin: Springer.
Kersting, K., De Raedt, L., & Raiko, T. (2006). Logical Hidden Markov Models, 25, 425–456.
Khardon, R. (1998). Learning first order universal Horn expressions. In Proceedings of the eleventh annual ACM conference on computational learning theory (pp. 154–165). New York: ACM.
Kietz, J. U. (1993). Some lower bounds on the computational complexity of inductive logic programming. In P. Brazdil (Ed.), Lecture notes in artificial intelligence: Vol. 667. Proceedings of the 6th European conference on machine learning (pp. 115–123). Berlin: Springer.
King, R. D., Muggleton, S. H., Srinivasan, A., & Sternberg, M. J. E. (1996). Structure-activity relationships derived by machine learning: the use of atoms and their bond connectives to predict mutagenicity by inductive logic programming. Proceedings of the National Academy of Sciences, 93, 438–442.
King, R. D., Whelan, K. E., Jones, F. M., Reiser, P. K. G., Bryant, C. H., Muggleton, S. H., Kell, D. B., & Oliver, S. G. (2004). Functional genomic hypothesis generation and experimentation by a robot scientist. Nature, 427, 247–252.
King, R. D., Rowland, J., Oliver, S. G., Young, M., Aubrey, W., Byrne, E., Liakata, M., Markham, M., Pir, P., Soldatova, L. N., Aparkes, A., Whelan, K. E., & Clare, A. (2009). The automation of science. Science, 324(5923), 85–89.
Knobbe, A. J., Siebes, A., & Marseille, B. (2002). Involving aggregate functions in multi-relational search. In Proceedings of the 6th European conference on data mining principles and practice of knowledge discovery in databases (p. 1).
Kramer, S., Lavrač, N., & Flach, P. (2001). Propositionalization approaches to relational data mining. In S. Džeroski & N. Lavrač (Eds.), Relational data mining (pp. 262–291). Berlin: Springer.
Krogel, M.-A., & Wrobel, S. (2001). Transformation-based learning using multirelational aggregation. In LNCS: Vol. 2157. Inductive logic programming (pp. 142–155).
Landwehr, N., Kersting, K., & De Raedt, L. (2007). Integrating naive Bayes and Foil. Journal of Machine Learning Research, 8, 481–507.
Lavrač, N., & Džeroski, S. (1993). Inductive logic programming: techniques and applications. Chichester: Ellis Horwood.
Lavrač, N., Džeroski, S., & Grobelnik, M. (1991). Learning non-recursive definitions of relations with LINUS. In Y. Kodratoff (Ed.), Lecture notes in artificial intelligence: Vol. 482. Proceedings of the 5th European working session on learning. Berlin: Springer.
Lisi, F. A., & Malerba, D. (2003). Bridging the gap between horn clausal logic and description logics in inductive learning. In LNCS: Vol. 2829. AI*IA 2003: Advances in artificial intelligence. Berlin: Springer.
Lloyd, J. W. (2003). Logic for learning. Berlin: Springer.
Mihalkova, L., & Mooney, R. J. (2009). Transfer learning from minimal target data by mapping across relational domains. In IJCAI-09: Proceedings of the twentieth international joint conference on artificial intelligence (pp. 1163–1168). San Mateo: Morgan-Kaufmann.
Milch, B., Zettlemoyer, L. S., Kersting, K., Haimes, M., & Kaelbling, L. P. (2008). Lifted probabilistic inference with counting formulas. In Proceedings of the twenty third conference on artificial intelligence (AAAI).
Morik, K., Wrobel, S., Kietz, J., & Emde, W. (1993). Knowledge acquisition and machine learning: theory, methods and applications. London: Academic Press.
Moyle, S., & Muggleton, S. H. (1997). Learning programs in the event calculus. In N. Lavrač & S. Džeroski (Eds.), LNAI: Vol. 1297. Proceedings of the seventh inductive logic programming workshop (ILP97) (pp. 205–212). Berlin: Springer.
Muggleton, S. H. (1987). Duce, an oracle based approach to constructive induction. In IJCAI-87 (pp. 287–292). Los Altos: Kaufmann.
Muggleton, S. H. (1991). Inductive logic programming. New Generation Computing, 8(4), 295–318.
Muggleton, S. H. (Ed.) (1992). Inductive logic programming. San Diego: Academic Press.
Muggleton, S. H. (1995). Inverse entailment and Progol. New Generation Computing, 13, 245–286.
Muggleton, S. H. (1996). Stochastic logic programs. In L. de Raedt (Ed.), Advances in inductive logic programming (pp. 254–264). Amsterdam: IOS Press.
Muggleton, S. H. (2002). Learning structure and parameters of stochastic logic programs. In Proceedings of the 12th international conference on inductive logic programming (pp. 198–206). Berlin: Springer.
Muggleton, S. H., & Bryant, C. H. (2000). Theory completion using inverse entailment. In Proc. of the 10th international workshop on inductive logic programming (ILP-00) (pp. 130–146). Berlin: Springer.
Muggleton, S. H., & Buntine, W. (1988). Machine invention of first-order predicates by inverting resolution. In Proceedings of the 5th international conference on machine learning (pp. 339–352). Los Altos: Kaufmann.
Muggleton, S. H., & De Raedt, L. (1994). Inductive logic programming: Theory and methods. Journal of Logic Programming, 19–20, 629–679.
Muggleton, S. H., & Feng, C. (1990). Efficient induction of logic programs. In Proceedings of the first conference on algorithmic learning theory (pp. 368–381). Tokyo: Ohmsha.
Muggleton, S. H., & Feng, C. (1992). Efficient induction of logic programs. In S. H. Muggleton (Ed.), Inductive logic programming (pp. 281–298). London: Academic Press.
Muggleton, S. H., King, R. D., & Sternberg, M. J. E. (1992). Protein secondary structure prediction using logic-based machine learning. Protein Engineering, 5(7), 647–657.
Muggleton, S. H., Fidjeland, A., & Luk, W. (2002). Scalable acceleration of inductive logic programs. In IEEE international conference on field-programmable technology (pp. 252–259). New York: IEEE Press.
Nienhuys-Cheng, S.-H., & de Wolf, R. (1997). LNAI: Vol. 1228. Foundations of inductive logic programming. Berlin: Springer.
Otero, R. (2005). Induction of the indirect effects of actions by monotonic methods. In Proceedings of the fifteenth international conference on inductive logic programming (ILP05) (Vol. 3625, pp. 279–294). Berlin: Springer.
Passerini, A., Frasconi, P., & De Raedt, L. (2006). Kernels on Prolog proof trees: statistical learning in the ILP setting. Journal of Machine Learning Research, 7, 307–342.
Plotkin, G. D. (1969). A note on inductive generalisation. In B. Meltzer & D. Michie (Eds.), Machine intelligence (Vol. 5, pp. 153–163). Edinburgh: Edinburgh University Press.
Plotkin, G. D. (1971a). Automatic methods of inductive inference. Ph.D. thesis, Edinburgh University, August 1971.
Plotkin, G. D. (1971b). A further note on inductive generalization. In Machine intelligence (Vol. 6). Edinburgh: Edinburgh University Press.
Poole, D. L. (1991). Representing diagnostic knowledge for probabilistic Horn abduction (pp. 1129–1135). Sydney.
Poole, D. L. (1993). Probabilistic Horn abduction and Bayesian networks. Artificial Intelligence, 64(1), 81–129.
Poole, D. L. (1997). The independent choice logic for modelling multiple agents under uncertainty. Artificial Intelligence, 94, 7–56. Special issue on economic principles of multi-agent systems.
Poole, D. L. (2000). Abducing through negation as failure: stable models within the independent choice logic. Journal of Logic Programming, 44(1–3), 5–35.
Poole, D. L. (2003). First-order probabilistic inference. In Proc. eighteenth international joint conference on artificial intelligence (IJCAI-03), Acapulco, Mexico (pp. 985–991).
Poole, D. L. (2008). The independent choice logic and beyond. In L. De Raedt, P. Frasconi, K. Kersting, & S. Muggleton (Eds.), LNCS: Vol. 4911. Probabilistic inductive logic programming: theory and application. Berlin: Springer.
Poole, D. L., & Mackworth, A. K. (2010). Artificial intelligence: foundations of computational agents. New York: Cambridge University Press.
Quinlan, J. R. (1987). Generating production rules from decision trees. In Proceedings of the tenth international conference on artificial intelligence (pp. 304–307). Los Altos: Kaufmann.
Quinlan, J. R. (1990). Learning logical definitions from relations. Machine Learning, 5, 239–266.
Quinlan, J. R., & Cameron-Jones, R.M. (1993). FOIL: a midterm report. In P. Brazdil (Ed.), Lecture notes in artificial intelligence: Vol. 667. Proceedings of the 6th European conference on machine learning (pp. 3–20). Berlin: Springer.
Ray, O., Broda, K., & Russo, A. (2003). Hybrid abductive inductive learning: a generalisation of Progol. In Lecture notes in artificial intelligence: Vol. 2835. Proceedings of the 13th international conference on inductive logic programming (ILP’03) (pp. 311–328). Berlin: Springer.
Richardson, M., & Domingos, P. (2006). Markov logic networks. Machine Learning, 62, 107–136.
Rouveirol, C., & Puget, J.-F. (1989). A simple and general solution for inverting resolution. In EWSL-89 (pp. 201–210). London: Pitman.
Russell, S. J., & Norvig, P. (2010). Artificial intelligence: a modern approach (3rd ed.). New Jersey: Pearson.
Sammut, C., & Banerji, R.B. (1986). Learning concepts by asking questions. In R. Michalski, J. Carbonnel, & T. Mitchell (Eds.), Machine learning: an artificial intelligence approach (Vol. 2, pp. 167–192). Los Altos: Kaufmann.
Sammut, C., & Webb, G. (Eds.) (2010). Encyclopedia of machine learning. Berlin: Springer.
Sanner, S., & Kersting, K. (2010). Symbolic dynamic programming. In C. Sammut & G. Webb (Eds.), Encyclopedia of machine learning. Berlin: Springer.
Santos Costa, V., Page, D., Qazi, M., & Cussens, J. (2003). CLP(BN): Constraint logic programming for probabilistic knowledge. In Proceedings of the 19th conference on uncertainty in artificial intelligence (pp. 517–524).
Sato, T. (2005). Generative modeling with failure in prism. In International joint conference on artificial intelligence (pp. 847–852). San Mateo: Morgan Kaufmann.
Sato, T., & Kameya, Y. (1997). PRISM: a symbolic-statistical modeling language. In Proceedings of the 15th international joint conference on artificial intelligence (IJCAI-97) (pp. 1330–1335).
Sato, T., & Kameya, Y. (2008). New advances in logic-based probabilistic modeling by PRISM. In L. De Raedt, P. Frasconi, K. Kersting, & S. Muggleton (Eds.), LNCS: Vol. 4911. Probabilistic inductive logic programming (pp. 118–155). Berlin: Springer.
Shapiro, E. Y. (1983). Algorithmic program debugging. Cambridge: MIT Press.
Stahl, I. (1992). Constructive induction in inductive logic programming: an overview (Technical report). Fakultat Informatik, Universitat Stuttgart.
Stahl, I. (1996). Predicate invention in inductive logic programming. In L. De Raedt (Ed.), Advances in inductive logic programming (pp. 34–47). Amsterdam: IOS Press.
Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: an introduction. Cambridge: MIT Press.
Synnaeve, G., Inoue, K., Doncescu, A., Kameya, Y., Sato, T., Ishihata, M., & Nabeshima, H. (2011). Kinetic models and qualitative abstraction for relational learning in systems biology. In Proceedings of the international conference on bioinformatics models, methods and algorithms.
Tamaddoni-Nezhad, A., Chaleil, R., Kakas, A., & Muggleton, S. H. (2006). Application of abductive ILP to learning metabolic network inhibition from temporal data. Machine Learning, 64, 209–230. doi:10.1007/s10994-006-8988-x.
Tamaddoni-Nezhad, A., Chaleil, R., Kakas, A., Sternberg, M. J. E., Nicholson, J., & Muggleton, S. H. (2007). Modeling the effects of toxins in metabolic networks. IEEE Engineering in Medicine and Biology, 26, 37–46. doi:10.1109/MEMB.2007.335590.
Torrey, L., & Shavlik, J. W. (2010). Policy transfer via Markov logic networks. In L. De Raedt (Ed.), LNAI: Vol. 5989. Proceedings of the nineteenth international conference on inductive logic programming (ILP09) (pp. 234–248). Berlin: Springer.
Van den Broeck, G., Thon, I., van Otterlo, M., & De Raedt, L. (2010). DTProbLog: A decision-theoretic probabilistic prolog. In Proceedings of the AAAI conference on artificial intelligence (AAAI 2010).
van Otterlo, M. (2009). The logic of adaptive behavior—knowledge representation and algorithms for adaptive sequential decision making under uncertainty in first-order and relational domains. Amsterdam: IOS Press.
Vens, C., Ramon, J., & Blockeel, H. (2006). Refining aggregate conditions in relational learning. In J. Fürnkranz, T. Scheffer, & M. Spiliopoulou (Eds.), Lecture notes in computer science: Vol. 4213. Proceedings of the 10th European conference on principles and practice of knowledge discovery in databases (pp. 383–394). Berlin: Springer.
Vere, S. A. (1975). Induction of concepts in the predicate calculus. In Proceedings of the 4th international joint conference on artificial intelligence (pp. 282–287). San Mateo: Morgan Kaufmann.
Wrobel, S. (1994). Concept formation during iterative theory revision. Machine Learning, 14, 169–191.
Yamamoto, Y., Inoue, K., & Iwanuma, K. (2010). From inverese entailment to inverese subsumption. In Proceedings of the 20th international conference on inductive logic programming (ILP’10).
Author information
Authors and Affiliations
Corresponding author
Additional information
Editors: Paolo Frasconi and Francesca Lisi.
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Muggleton, S., De Raedt, L., Poole, D. et al. ILP turns 20. Mach Learn 86, 3–23 (2012). https://doi.org/10.1007/s10994-011-5259-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10994-011-5259-2