Monday, August 11, 2014

Probabilistic Graphical Models (PGM)

Recently, I discussed probabilistic graphical Models (PGM) with several people. Everyone has a link here and a link there. So, I decided to put the references in one place. See list below. As expected, people will disagree about what should or should not be included. This is a good thing. So comment away.

Similarly, people will disagree as to what topics belong under the PGM umbrella. Furthermore, some people will want to format the topics in an alphabetic linear list while others will want a hierarchy. Attached to this blog are pdf files that reflect my attempt at both. Disagreemnt is a good thing. So comment away.

By the way, I am at best a novice at this type of stuff.
  1. Books
    1. Advanced Data Analysis from an Elementary Point of View by Cosma Rohilla Shalizi
      1. Algorithms for Graphical Models by James Cussens
        1. To download the book, click here.
        2. Artificial Intelligence: A Modern Approach 2e, by S. Russell & P. Norvig, Prentice-Hall, 2003
          1. Bayesian Artificial Intelligence, Second Edition by Kevin B. Korb, Ann E. Nicholson
            1. Amazon
            2. Bayesian Networks and Decision Graphs by Thomas Dyhre Nielsen, FINN VERNER JENSEN
              1. Amazon
              2. Bayesian Networks and Influence Diagrams: A Guide to Construction and Analysis by Uffe B. Kjærulff, Anders L. Madsen
                1. Amazon
                2. Bayesian Networks for Probabilistic Inference and Decision Analysis in Forensic Science by Franco Taroni, Alex Biedermann, Silvia Bozza
                  1. Amazon
                  2. Bayesian Networks in R: with Applications in Systems Biology (Use R!) by Radhakrishnan Nagarajan, Marco Scutari
                    1. Amazon
                    2. Bayesian Networks: A Practical Guide to Applications by Olivier Pourret, Patrick Naïm, Bruce Marcot
                      1. Amazon
                      2. Bayesian Networks: With Examples in R by Marco Scutari, Jean-Baptiste Denis
                        1. Amazon
                        2. Bayesian Reasoning and Machine Learning (BRML) by David Barber
                          1. Author's Web Page for the Book
                            1. Free Legal PDF Version of Book & Errata
                              1. Associated MatLab Code
                              2. Amazon
                                1. ... only criticism (a mild one) is that, when applying Barber's examples to Bodies of Knowledge and data mining, he skips Prolog, backward chaining, predicate calculus and other techniques that are the foundation of automated inference systems (systems that extend knowledge bases automatically by checking whether new propositions can be inferred from the KB as consistent, relevant, etc.) ...
                                2. Cambridge.Org
                                  1. Git Hub
                                  2. Building Probabilistic Graphical Models with Python by Kiran R Karkera
                                    1. CERN Document Server
                                      1. Amazon
                                        1. ... All is illustrated with examples and code snippets using mostly the libpgm package. PyMC is used for Bayesian parameter estimation ...
                                        2. PacktPub.com
                                        3. Causality: Models, Reasoning and Inference by Judea Pearl
                                          1. Amazon
                                          2. Elements of Statistical Learning by Trevor Hastie
                                            1. Amazon
                                            2. Graphical Models by Steffen L. Lauritzen
                                              1. Amazon
                                              2. Graphical Models: Foundations of Neural Computation by Michael I. Jordan, Terrence J. Sejnowski
                                                1. Amazon
                                                2. Graphical Models: Representations for Learning, Reasoning and Data Mining by Christian Borgelt, Matthias Steinbrecher
                                                  1. Amazon
                                                  2. Graphical Models for Machine Learning and Digital Communication by Brendan J. Frey
                                                    1. Amazon
                                                    2. Graphical Models in Applied Multivariate Statistics Paperback by Joe Whittaker
                                                      1. Amazon
                                                      2. Graphical Models with R (Use R!) by Søren Højsgaard, David Edwards, Steffen Lauritzen
                                                        1. Amazon
                                                        2. Graphical Models, Exponential Families, and Variational Inference by Martin J Wainwright, Michael I Jordan
                                                          1. Amazon
                                                          2. Inference in Hidden Markov Models by Olivier Cappé, Eric Moulines, Tobias Ryden
                                                            1. Amazon
                                                            2. Introduction to Graphical Modelling by David Edwards
                                                              1. Amazon 1
                                                                1. Amazon 2
                                                                2. Introduction to Statistical Learning: with Applications in R by Gareth James, Daniela Witten, Trevor Hastie
                                                                  1. Amazon
                                                                  2. Large-Scale Inference: Empirical Bayes Methods for Estimation, Testing, and Prediction by Bradley Efron
                                                                    1. Amazon
                                                                    2. Learning Bayesian Networks by Richard E. Neapolitan
                                                                      1. Amazon
                                                                      2. Learning in Graphical Models by Michael I. Jordan
                                                                        1. Amazon
                                                                        2. Machine Learning: a Probabilistic Perspective by Kevin Patrick Murphy
                                                                          1. Author's Web Page for the Book
                                                                            1. Free Legal PDF Version of Book & Errata
                                                                              1. Associated MatLab Code
                                                                            2. Mind's Arrows: Bayes Nets and Graphical Causal Models in Psychology by Clark Glymour
                                                                              1. Amazon
                                                                              2. Modeling and Reasoning with Bayesian Networks by Adnan Darwiche
                                                                                1. Amazon
                                                                                2. Pattern Recognition & Machine Learning, by C.M. Bishop, Springer, 2007.
                                                                                  1. Probabilistic Graphical Models: Principles and Techniques by Daphne Koller, Nir Friedman
                                                                                    1. Amazon
                                                                                      1. ... Her course for probabilistic models is available online, and watching the videos alongside the book really helps sometimes ...
                                                                                        1. ... authors painstakingly provided necessary background materials from both probability theory and graph theory ...
                                                                                        2. Associated Coursera Course
                                                                                          1. Google
                                                                                            1. mitpress.mit.edu
                                                                                              1. Stanford.Edu
                                                                                              2. Probabilistic Networks and Expert Systems - Exact Computational Methods for Bayesian Networks by Cowell, R.G., Dawid, P., Lauritzen, S.L., Spiegelhalter, D.J.
                                                                                                1. Springer
                                                                                                2. Probabilistic Programming & Bayesian Methods for Hackers = Using Python and PyMC
                                                                                                  1. Probability on Graphs: Random Processes on Graphs and Lattices by Geoffrey Grimmett
                                                                                                    1. Amazon
                                                                                                  2. Classes
                                                                                                    1. Advanced Topics in Machine Learning - Instructor: Milos Hauskrecht
                                                                                                    2. Web Pages
                                                                                                      1. BAYES NET BY EXAMPLE USING PYTHON AND KHAN ACADEMY DATA by DeRandomized.com
                                                                                                        1. Bayes Net Toolbox for Matlab
                                                                                                          1. Code.Google.com
                                                                                                            1. GitHub
                                                                                                            2. Brief Introduction to Graphical Models and Bayesian Networks by Kevin Murphy
                                                                                                              1. Building Probabilistic Graphical Models with Python by Steve Lott
                                                                                                                1. Carmen: An open source project for probabilistic graphical models by Manuel Arias and Francisco J. D´ıez
                                                                                                                  1. Comparison between igraph and networkx by Google Group: NetworkX Discussion
                                                                                                                    1. Data Structures and Algorithms with Object-Oriented Design Patterns in Python by Bruno R. Preiss - Graphs and Graph Algorithms
                                                                                                                      1. Daft: Python package that uses matplotlib to render pixel-perfect probabilistic graphical models
                                                                                                                        1. ECE 175B: Prob. Reasoning & Graphical Models by Ken Kreutz-Delgado, Spring 2014
                                                                                                                          1. FACTORIE
                                                                                                                            1. FACTORIE is a toolkit for deployable probabilistic modeling, implemented as a software library in Scala.
                                                                                                                              1. FACTORIE aims to provide a full-featured framework for probabilistic graphical models (both directed and undirected)
                                                                                                                                1. SciKit-Learn provides an extensive set of tools for classification, regression, clustering, dimensionality reduction and model selection. It is more mature than FACTORIE, and it is easily integratable with matplotlib, a graphics package, so it can produce high-quality graphs directly. However, it does not support graphical models. SciKit-Learn is implemented in Python; in many cases it is less efficient than FACTORIE, which is JVM-based.
                                                                                                                                2. gPy
                                                                                                                                  1. graph-tool: Graph-tool is an efficient Python module for manipulation and statistical analysis of graphs (a.k.a. networks). Contrary to most other python modules with similar functionality, the core data structures and algorithms are implemented in C++, making extensive use of template metaprogramming, based heavily on the Boost Graph Library. This confers it a level of performance that is comparable (both in memory usage and computation time) to that of a pure C/C++ library.
                                                                                                                                    1. igraph: The network analysis package igraph is a collection of network analysis tools with the emphasis on efficiency, portability and ease of use. igraph is open source and free.
                                                                                                                                      1. Implementing a Principal Component Analysis (PCA) in Python step by step by Sebastian Raschka on April 13, 2014
                                                                                                                                        1. Daft: Python package that uses matplotlib to render pixel-perfect probabilistic graphical models
                                                                                                                                          1. NetworkX: NetworkX is a Python language software package for the creation, manipulation, and study of the structure, dynamics, and functions of complex networks.
                                                                                                                                            1. Modular toolkit for Data Processing (MDP): Python data processing framework
                                                                                                                                              1. Pebl
                                                                                                                                                1. Code.Google.Com
                                                                                                                                                  1. GitHub
                                                                                                                                                  2. pgmpy
                                                                                                                                                    1. GitHub
                                                                                                                                                    2. Probabilistic Modeling Toolkit for Matlab/Octave
                                                                                                                                                      1. GitHub
                                                                                                                                                      2. Probabilistic Programming & Bayesian Methods for Hackers by Cam Davidson-Pilon
                                                                                                                                                        1. GitHub
                                                                                                                                                          1. iPython
                                                                                                                                                          2. PyDot: Python interface to Graphviz's Dot language. Pydot allows to easily create both directed and non directed graphs from Python.
                                                                                                                                                            1. pymc
                                                                                                                                                              1. Python.org
                                                                                                                                                                1. GitHub
                                                                                                                                                                2. PyNFG: PyNFG is designed to make it easy for researchers to model strategic environments using the Network Form Game (NFG) formalism developed by David Wolpert with contributions from Ritchie Lee, James Bono and others. The main idea of the NFG framework is to translate a strategic environment into the language of probabilistic graphical models.
                                                                                                                                                                  1. python-graph: Library for working with graphs in Python. This software provides a suitable data structure for representing graphs and a whole set of important algorithms.
                                                                                                                                                                    1. scikit-learn
                                                                                                                                                                      1. Software by Kevin Murphy and students
                                                                                                                                                                        1. Software Packages for Graphical Models by Kevin Murphy
                                                                                                                                                                          1. Two Node Directed Graphical Model Example by Jascha Sohl-Dickstein
                                                                                                                                                                            1. Understanding your data w/ Bayesian networks (in python) by Bartek Wilczunski (bartek@mimuw.edu.pl)
                                                                                                                                                                              1. slideshare.net
                                                                                                                                                                              2. Wikipedia: Bayesian Network
                                                                                                                                                                                1. Wikipedia: Graphical Model
                                                                                                                                                                              Posted by at

                                                                                                                                                                              3 comments:

                                                                                                                                                                              1. Ned PolianApril 9, 2015 at 10:21 AM

                                                                                                                                                                                This is so beautiful and creative. I just love the colors and whoever gets it in the mail will be smiling. electric hot water heater

                                                                                                                                                                                ReplyDelete
                                                                                                                                                                              2. UnknownJuly 30, 2017 at 4:03 AM

                                                                                                                                                                                Thanks for taking the time to discuss this, I feel strongly about it and love learning more on this topic. If possible, as you gain expertise, would you mind updating your blog with more information? It is extremely helpful for me.
                                                                                                                                                                                http://www.anixusa.com/

                                                                                                                                                                                ReplyDelete
                                                                                                                                                                              3. UnknownNovember 28, 2018 at 4:58 AM

                                                                                                                                                                                Love to read it,Waiting For More new Update and I Already Read your Recent Post its Great Thanks.
                                                                                                                                                                                wholesale strainers

                                                                                                                                                                                ReplyDelete

                                                                                                                                                                              Subscribe to: Post Comments (Atom)