Uncertainty - The Bayesian  Network

  

Bayesian Networks are a type of Probabilistic Graphical Model that can be used to build models from data and/or expert opinion. Bayesian networks are probabilistic because they are built from probability distributions and also use the laws of probability for prediction and anomaly detection, for reasoning and diagnostics decision making under uncertainty and times series prediction.

They can be used for a wide range of tasks including prediction, anomaly detection, diagnostics, automated insight, reasoning, time series prediction and decision making under uncertainty.

Real world applications are probabilistic in nature, and to represent the relationship between multiple events, we need a Bayesian network. 

In Bayesian network, we deal with a number of interrelated (random) variables. We explore how the joint distribution of the variables can be described by exploiting what we know about their natural interrelationships via conditional distributions. We use graph theory to explain their interrelationship. If data are available on the random variables, we fit a Bayesian network model which describes their relationship in a succinct way. Bayesian networks are a marriage between probability theory and graphs. One of the main goals in Bayesian networks is prediction.

In this blog  we will train the Bayesian Network in Python.  My self Happy khatun. I am a student of city university. And this blog is a part of our AI (Artificial Intillgecne) Lab Course conducted by our most honorable teacher Nuruzzaman Faruqui sir.

Here is an example of a Bayesian network that involves variables that detect whether we get to our appointment on time.

According to solve this problem we have to build up a python code to train the Bayesian Network. The code is given below:

'''

pomegranate is a python package that implements fast, efficient, and extremely flexible probabilistic models

ranging from probability distributions to Bayesian networks to mixtures of hidden Markov models.

'''

from pomegranate import *

from pomegranate.base import Node

# Rain has no parent Node. So it's of DiscreteDistribution

rain = Node(DiscreteDistribution(

    {

        'none': 0.7,

        'light': 0.2,

        'heavy': 0.1

    }

), name='rain')

# Maintenance Node is conditional on rain

maintanence = Node(ConditionalProbabilityTable(

    [

        ['none', 'yes', 0.4],

        ['none', 'no', 0.6],

        ['light', 'yes', 0.2],

        ['light', 'no', 0.8],

        ['heavy', 'yes', 0.1],

        ['heavy', 'no', 0.9],

    ], [rain.distribution]), name='maintanence')

# train Node is conditional on rain,maintanence

train = Node(ConditionalProbabilityTable(

    [

        ['none', 'yes', 'ontime', 0.8],

        ['none', 'yes', 'delayed', 0.2],

        ['none', 'no', 'ontime', 0.9],

        ['none', 'no', 'delayed', 0.1],

        ['light', 'yes', 'ontime', 0.6],

        ['light', 'yes', 'delayed', 0.4],

        ['light', 'no', 'ontime', 0.7],

        ['light', 'no', 'delayed', 0.3],

        ['heavy', 'yes', 'ontime', 0.4],

        ['heavy', 'yes', 'delayed', 0.6],

        ['heavy', 'no', 'ontime', 0.5],

        ['heavy', 'no', 'delayed', 0.5],

    ], [rain.distribution, maintanence.distribution]),

    name='train')

# Appointment Node is conditional on train

appointment = Node(ConditionalProbabilityTable(

    [

        ['ontime', 'attend', 0.9],

        ['ontime', 'miss', 0.1],

        ['delayed', 'attend', 0.6],

        ['delayed', 'miss', 0.4],

    ], [train.distribution]), name='appointment')

# create a Bayesian Network to add states or Nodes

model = BayesianNetwork()

# Add Nodes

model.add_states(rain, maintanence, train, appointment)

# Add edges connecting Nodes

model.add_edge(rain, maintanence)

model.add_edge(rain, train)

model.add_edge(maintanence, train)

model.add_edge(train, appointment)

# Finalize the model

model.bake()

By doing this code we made our Bayesian Network model. And now from this model we will detect the probability distributions of the variables Rain, Maintenance, and Appointment.
Now we have to implement another following code:

from model import model

# calculate predictions based on the evidence that the train was delayed

predictions = model.predict_proba(

    {

        "train": "delayed"

    }

)

# print predictions for each node


for node, prediction in zip(model.states, predictions):

    if isinstance(prediction, str):

        print(f"{node.name} : {prediction} ")



    else:

        print(f" {node.name} ")

        for value, probability in prediction.parameters[0].items():
            print(f"     {value}: {probability: .4f} ")

After running this code we get following result:

Bayesian Networks are very convenient for representing systems of probabilistic causal relationships.

A  Bayesian Network, provides a way of defining a probabilistic model for a complex problem by stating all of the conditional independence assumptions for the known variables, whilst allowing the presence of unknown (latent) variables.

Here we describe Bayesian Networking in a easiest way .This is the easiest source to learn  

python programming.

 Knowledge Representation in AI

Knowledge representation plays a crucial role in artificial Intelligence. It has to do with the ‘thinking’ of AI systems and contributes to its intelligent behavior. Knowledge Representation is a radical and new approach in AI that is changing the world. is concerned with presenting real-world information in a form that the computer can ‘understand’ and use to ‘solve’ real - life problem or ‘handle’ real-life task. The ability of machines to think and act like humans such as understanding, interpreting and reasoning constitute knowledge representation. It is related to designing agents that can think and ensure that such thinking can constructively contribute to the agent’s behavior. 

In simple words, knowledge representation allows machines to behave like humans by empowering an AI machine to learn from available information, experience or experts.

My self Happy khatun. I am a student of city university. And this blog is a part of our AI (Artificial Intilegent) Lab Course conducted by our most honorable teacher Nuruzzaman Faruqui.

In this blog, we will develop a game engine that will detect a murder based on its knowledge base.

Here is a problem which we will solve. Suppose there is a dead body of a Doctor called Doctor Black .It has been found Yesterday. There were only three people in Doctor Black’s mansion.  Now we have to detect the murderer using AI. We have the following information in our hand regarding to start our investigation.

Three prime suspects:

1. Col. Mustard 

2. Prof. Plum

3. Ms. Scarlet

Three weapons (Founded by the police in the mansion)

1. Knife

2. Revolver

3. Wrench

There is three rooms in his mansion

1. Ballroom

2. Kitchen

3. Library

Now we will find find the murderer based on this above information. For this we have implement the following code:

# we import termcolor to change the colour in terminal window
import termcolor

# we have import the logic file
from logic import *

#Now we are symboling all of the charecters,rooms,weapons
mustard=Symbol("col.mustard")
plum=Symbol("ProfPlum")
scarlet=Symbol("MsScarlet")

charecters=[mustard,plum,scarlet]

ballroom=Symbol("ballroom")
kitchen=Symbol("kitchen")
library=Symbol("library")

rooms=[ballroom,kitchen,library]

revolber=Symbol("revolber")
knife=Symbol("knife")
wrench=Symbol("wrench")

wreapons=[revolber,knife,wrench]

# Now we are concating characters , rooms and weapons in symbols.

symbols=charecters+rooms+wreapons

# we are checking the model and get some truth value
def check_knowledge(knowledge_base):
    for symbol in symbols:
         if model_check(knowledge_base,symbol):
             termcolor.cprint(f"{symbol}:YES","green")

         elif not model_check(knowledge_base,Not(symbol)):
             print(f"{symbol}:Maybe")


# Createing knowledge base
knowledge_base=And(

    Or(mustard,plum,scarlet),
    Or(ballroom,kitchen,library),
    Or(knife,revolber,wrench)
)

# They are clue

knowledge_base.add(And(
    Not(mustard),Not(kitchen),Not(wrench)
))
knowledge_base.add(Or(
    Not(scarlet),Not(library),Not(wrench)
))

knowledge_base.add(Not(plum))
knowledge_base.add(Not(ballroom))
knowledge_base.add(Not(revolber))



check_knowledge(knowledge_base)

After running this code we get the following result:

In artificial intelligence, knowledge can be represented in various ways depending on the structure of the knowledge or the perspective of the designer or even the type of internal structure used. An effective knowledge representation should be rich enough to include the knowledge required to solve the problem. It should be natural, compact and maintainable. it is important to choose the right type of knowledge representation .Knowledge and logical reasoning play a huge role in artificial intelligence. However, you often require more than just general and powerful methods to ensure intelligent behavior. And this blog is the easiest solution to prepare oneself in order to learn knowledge representation through AI.

 

Classifying Authenticity of Banknotes using Machine Learning

Machine learning is the art of giving a computer data, and having it learn trends from that data and then make predictions based on new data. Machine learning is a subfield of artificial intelligence. Its goal is to enable computers to learn on their own.  Machine learning is an application of artificial intelligence (AI) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed. Machine  learning focuses on the development of computer programs that can access data and use it to learn for themselves.The process of learning begins with observations or data, such as examples, direct experience, or instruction, in order to look for patterns in data and make better decisions in the future based on the examples that we provide. 

Myself  Happy khatun. I am a student of city university. And this blog is a part of our AI (Artificial Intelligence) Lab Course conducted by our most honorable teacher Nuruzzaman Faruqui.
Here we will discuss and implement machine learning algorithm in order to detecting fake or real Banknotes.

suppose we have the following data set

Now  we are going to train a machine on a dataset and then put it on testing.There are five-section and the last section depends on the first four sections. In the last sections, it will find the notes is real or fake. The first section is variance, the second is skewness, the third is kurtosis, fourth is entropy. we'll use 40% of our dataset in our programme.we will use perceptron learning, SVM, K-nearest neighbor, and GaussianNB to check which classification model performs well on our data. 

For this we will need this following code

# pip install scikit-learn
import csv
import random

from sklearn import svm
from sklearn.linear_model import Perceptron
from sklearn.naive_bayes import GaussianNB
from sklearn.neighbors import KNeighborsClassifier

model = Perceptron()
# model = svm.SVC()
# model = KNeighborsClassifier(n_neighbors=1)
# model = GaussianNB()

# Read data in from file
with open("banknotes.csv") as f:
    reader = csv.reader(f)
    next(reader)

    data = []
    for row in reader:
        data.append({
            "evidence": [float(cell) for cell in row[:4]],
            "label": "Authentic" if row[4] == "0" else "Counterfeit"
        })

# Separate data into training and testing groups
holdout = int(0.40 * len(data))
random.shuffle(data)
testing = data[:holdout]
training = data[holdout:]

# Train model on training set
X_training = [row["evidence"] for row in training]
y_training = [row["label"] for row in training]
model.fit(X_training, y_training)

# Make predictions on the testing set
X_testing = [row["evidence"] for row in testing]
y_testing = [row["label"] for row in testing]
predictions = model.predict(X_testing)

# Compute how well we performed
correct = 0
incorrect = 0
total = 0
for actual, predicted in zip(y_testing, predictions):
    total += 1
    if actual == predicted:
        correct += 1
    else:
        incorrect += 1

# Print results
print(f"Results for model {type(model).__name__}")
print(f"Correct: {correct}")
print(f"Incorrect: {incorrect}")
print(f"Accuracy: {100 * correct / total:.2f}%")

After running this code we get the following result

Model	correct	Incorrect	Accuracy
Perceptron	539	9	98.36%
SVM	545	3	99.45%
KNN	548	0	100%
GaussianNB	457	91	83.39%

Here we can see that KNN has the hihgest accuracy among Perceptron, SVM, GaussianNB. And GaussianNB has the lowest accuracy.

Modern day machine learning has two objectives, one is to classify data based on models which have been developed, the other purpose is to make predictions for future outcomes based on these models.
In this blog we discuss about the implementation of machine learning in detecting Banknotes in a easiest way.so, this is the easiest way to learn python programming.

 

Hill climbing algorithm

 

Hill climbing is a mathematical optimization algorithm, which means its purpose is to find the best solution to a problem which has a (large) number of possible solutions. Explaining the algorithm (and optimization in general) is best done using an example.Hill climbing tries to find the best solution to this problem by starting out with a random solution, and then generate neighbours: solutions that only slightly differ from the current one. If the best of those neighbours is better (i.e. shorter) than the current one, it replaces the current solution with this better solution. It then repeats the pattern by again creating neighbours. If at some point no neighbour is better than the current solution, it returns the then current solution.

In this blog we will optimize Hill climbing algorithm with Random Re Start Varient.  My self Happy khatun. I am a student of city university. And this blog is a part of our AI (Artificial Intilegent) Lab Course conducted by our most honourable teacher Nuruzzaman Faruqui.

 

Now we will solve the following problem

 

In this problem, there are hospitals and houses in different places. We have to find the best cost path between the hospital and the house. When the cost will less than the previous then it will be the best cost. In order to finding this we need to implement following code :

import random  # pseudo-random number generators


class Space():

    def __init__(self, height, width, num_hospitals):
        """Create a new state space with given dimensions."""
        self.height = height
        self.width = width
        self.num_hospitals = num_hospitals
        self.houses = set()
        self.hospitals = set()

    def add_house(self, row, col):
        """Add a house at a particular location in state space."""
        self.houses.add((row, col))

    def available_spaces(self):
        """Returns all cells not currently used by a house or hospital."""

        # Consider all possible cells
        candidates = set(
            (row, col)
            for row in range(self.height)
            for col in range(self.width)
        )

        # Remove all houses and hospitals
        for house in self.houses:
            candidates.remove(house)
        for hospital in self.hospitals:
            candidates.remove(hospital)
        return candidates

    def hill_climb(self, maximum=None, image_prefix=None, log=False):
        """Performs hill-climbing to find a solution."""
        count = 0

        # Start by initializing hospitals randomly
        self.hospitals = set()
        for i in range(self.num_hospitals):
            self.hospitals.add(random.choice(list(self.available_spaces())))
        if log:
            print("Initial state: cost", self.get_cost(self.hospitals))
        if image_prefix:
            self.output_image(f"{image_prefix}{str(count).zfill(3)}.png")
            '''zfill() method adds zeros (0) at the beginning of the string'''

        # Continue until we reach maximum number of iterations
        while maximum is None or count < maximum:
            count += 1
            best_neighbors = []
            best_neighbor_cost = None

            # Consider all hospitals to move
            for hospital in self.hospitals:

                # Consider all neighbors for that hospital
                for replacement in self.get_neighbors(*hospital):

                    # Generate a neighboring set of hospitals
                    neighbor = self.hospitals.copy()  # Slide 28
                    neighbor.remove(hospital)  # slide 29
                    neighbor.add(replacement)  # Slide 30

                    # Check if neighbor is best so far
                    cost = self.get_cost(neighbor)
                    if best_neighbor_cost is None or cost < best_neighbor_cost:
                        best_neighbor_cost = cost
                        best_neighbors = [neighbor]
                    elif best_neighbor_cost == cost:
                        best_neighbors.append(neighbor)

            # None of the neighbors are better than the current state
            if best_neighbor_cost >= self.get_cost(self.hospitals):
                return self.hospitals

            # Move to a highest-valued neighbor
            else:
                if log:
                    print(f"Found better neighbor: cost {best_neighbor_cost}")
                self.hospitals = random.choice(best_neighbors)

            # Generate image
            if image_prefix:
                self.output_image(f"{image_prefix}{str(count).zfill(3)}.png")

    def random_restart(self, maximum, image_prefix=None, log=False):
        """Repeats hill-climbing multiple times."""
        best_hospitals = None
        best_cost = None

        # Repeat hill-climbing a fixed number of times
        for i in range(maximum):
            hospitals = self.hill_climb()
            cost = self.get_cost(hospitals)
            if best_cost is None or cost < best_cost:
                best_cost = cost
                best_hospitals = hospitals
                if log:
                    print(f"{i}: Found new best state: cost {cost}")
            else:
                if log:
                    print(f"{i}: Found state: cost {cost}")

            if image_prefix:
                self.output_image(f"{image_prefix}{str(i).zfill(3)}.png")

        return best_hospitals

    def get_cost(self, hospitals):
        """Calculates sum of distances from houses to nearest hospital."""
        cost = 0
        for house in self.houses:
            cost += min(
                abs(house[0] - hospital[0]) + abs(house[1] - hospital[1])
                for hospital in hospitals
            )
        return cost

    def get_neighbors(self, row, col):
        """Returns neighbors not already containing a house or hospital."""
        candidates = [
            (row - 1, col),
            (row + 1, col),
            (row, col - 1),
            (row, col + 1)
        ]
        neighbors = []
        for r, c in candidates:
            if (r, c) in self.houses or (r, c) in self.hospitals:
                continue
            if 0 <= r < self.height and 0 <= c < self.width:
                neighbors.append((r, c))
        return neighbors

    def output_image(self, filename):
        """Generates image with all houses and hospitals."""
        from PIL import Image, ImageDraw, ImageFont
        cell_size = 100
        cell_border = 2
        cost_size = 40
        padding = 10

        # Create a blank canvas
        img = Image.new(
            "RGBA",
            (self.width * cell_size,
             self.height * cell_size + cost_size + padding * 2),
            "white"
        )
        house = Image.open("assets/images/House.png").resize(
            (cell_size, cell_size)
        )
        hospital = Image.open("assets/images/Hospital.png").resize(
            (cell_size, cell_size)
        )
        font = ImageFont.truetype("assets/fonts/OpenSans-Regular.ttf", 30)
        draw = ImageDraw.Draw(img)

        for i in range(self.height):
            for j in range(self.width):

                # Draw cell
                rect = [
                    (j * cell_size + cell_border,
                     i * cell_size + cell_border),
                    ((j + 1) * cell_size - cell_border,
                     (i + 1) * cell_size - cell_border)
                ]
                draw.rectangle(rect, fill="black")

                if (i, j) in self.houses:
                    img.paste(house, rect[0], house)
                if (i, j) in self.hospitals:
                    img.paste(hospital, rect[0], hospital)

        # Add cost
        draw.rectangle(
            (0, self.height * cell_size, self.width * cell_size,
             self.height * cell_size + cost_size + padding * 2),
            "black"
        )
        draw.text(
            (padding, self.height * cell_size + padding),
            f"Cost: {self.get_cost(self.hospitals)}",
            fill="white",
            font=font
        )

        img.save(filename)


# Create a new space and add houses randomly
s = Space(height=10, width=20, num_hospitals=3)
for i in range(15):
    s.add_house(random.randrange(s.height), random.randrange(s.width))

# Use local search to determine hospital placement
hospitals = s.hill-climb(image_prefix="hospitals", log=True)

After running this code we get the following result

For Hill climbing to work, it has to start with a random solution to our Travelling salesman problem. From there, it can generate neighbouring solutions and start the optimization process.

In this blog  we learned how to implement Hill Climbing Algorithm & Random Restart variants to optimize solutions in a easiest way.