{"id":15585,"date":"2020-06-05T16:44:22","date_gmt":"2020-06-05T11:14:22","guid":{"rendered":"https:\/\/www.mygreatlearning.com\/blog\/bernoulli-distribution-explained\/"},"modified":"2024-12-16T10:12:06","modified_gmt":"2024-12-16T04:42:06","slug":"bernoulli-distribution-explained","status":"publish","type":"post","link":"https:\/\/www.mygreatlearning.com\/blog\/bernoulli-distribution-explained\/","title":{"rendered":"What is Bernoulli distribution? Bernoulli Distribution Explained"},"content":{"rendered":"\n

Contributed by: Shailendra Singh
LinkedIn Profile: https:\/\/www.linkedin.com\/in\/shailendra-singh-a817802\/ <\/a><\/p>\n\n\n\n

An important skill for people working in Data Science is to have a good understanding of the fundamental concepts of descriptive statistics and probability theory. This includes the key concepts of probability distribution<\/a><\/strong>, statistical significance, hypothesis testing, and regression. In practice, a simple analysis using R or scikit-learn in python<\/a>, without quite understanding the probability distributions, often ends in errors and wrong results.<\/p>\n\n\n\n

There are many probability distributions, but in this article, we will be talking about the simplest probability distribution called Bernoulli distribution. It is considered to be a building block for other more complicated discrete distributions. Before proceeding on to explaining Bernoulli distribution, we first need to understand some of the basic concepts used in probability distributions. Let\u2019s get started.<\/p>\n\n\n\n

Random Variables<\/strong><\/h2>\n\n\n\n

In statistics and probability, random variable, random quantity, or stochastic variable are described as those variables whose values depend on the outcomes of an experiment (i.e. a random process). Random variables are of two types, discrete and continuous. In this text, we will cover a distribution type concerning discrete random variables.<\/p>\n\n\n\n

To understand random variables with a simple example, assume that we execute a random experiment of rolling a dice. The possible outcome we could get from this experiment could be any number between 1 to 6. If X denotes the random variable that represents the outcome of such a random process, the sample space of this experiment consists of the outcomes {1, 2, 3, \u00b7 \u00b7 \u00b7, 6}.<\/p>\n\n\n\n

So X=1, if the outcome of the dice roll is 1, X=2, if the outcome of the dice roll is 2 and so on till X=6 if the outcome of the dice roll is 6. <\/p>\n\n\n\n

Taking a mathematical approach to simplify and generalize the problem, we can represent a single random event of rolling a dice as shown in a single box in the figure below. Extending the random event to n trials, shown as separate boxes in the figure below, would represent the outcome from n such random events.<\/p>\n\n\n\n

Probability distribution<\/strong><\/h2>\n\n\n\n

With the understanding of random variables, we can define a probability distribution to be a list of all the possible outcomes of a random variable, along with their corresponding probability values.<\/p>\n\n\n\n

Considering our earlier example of a dice roll, we can represent the probability distribution of a 6 sided dice as given below.<\/p>\n\n\n\n

Outcome <\/td>1<\/td>2<\/td>3<\/td>4<\/td>5<\/td>6<\/td><\/tr>
Probability<\/td>1\/6<\/td>1\/6<\/td>1\/6<\/td>1\/6<\/td>1\/6<\/td>1\/6<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

Table 1: Probability Distribution<\/strong><\/p>\n\n\n\n

We can represent the dice roll example graphically as shown below:<\/p>\n\n\n\n

We can state the following in regards to the probability distribution table shown above-<\/p>\n\n\n\n

    \n
  1. In the case of an experiment to roll a six-sided dice where the values lie in the set {1,2,3,4,5,6}. The outcome variable would always have a discrete<\/strong> value (between 1-6). <\/li>\n\n\n\n
  2. This is a univariate<\/strong> distribution since there is just one random variable i.e., the outcome of the dice roll. <\/li>\n<\/ol>\n\n\n\n

    Therefore the distribution shown in the table above can be termed as a discrete univariate probability distribution.<\/strong> <\/p>\n\n\n\n

    Also Read: What is Gradient Boosting?<\/a><\/p>\n\n\n\n

    Probability Functions<\/strong><\/h2>\n\n\n\n

    If we represent the probability in machine learning<\/a> graphically, it will look like this-<\/p>\n\n\n\n

    The figure above represents a single trial(x1) experiment where n = 1. We could repeat the experiment n number of times for X={x1, x2,..xn } to get n outcomes. <\/p>\n\n\n\n

    Discrete Probability Distribution: (Probability Mass Function)<\/strong><\/h2>\n\n\n\n

    When we use a probability function (which is described above) to describe a discrete distribution we call this function a probability mass function (pmf).<\/p>\n\n\n\n

    By a discrete distribution, we mean that the random variable of the underlying distribution can take on only finitely many different values (or it can be said that the outcome space is finite).<\/p>\n\n\n\n

    To define a discrete distribution, we can simply enumerate the probability of the random variable taking on each of the possible values. This enumeration is known as the probability mass function, as it divides up a unit mass (the total probability) and returns the probability of different values a random variable can take.<\/p>\n\n\n\n

    Generally, we can represent a probability mass function as below.
    <\/p>\n\n\n\n

    f(x) = P(X = x), <\/strong>for e.g. Taking the dice roll as a random variable, we can write the probability of the dice landing on the number 2 as   f(2) = P(X=2) = 1\/6<\/strong>.<\/p>\n\n\n\n

    The probability mass function must follow the rules of probability, therefore-<\/p>\n\n\n\n

      \n
    1. 0<=f(x)<=1<\/li>\n\n\n\n
    2. \u2211f(xi) = f(x1) + f(x2) + \u2026 = 1<\/li>\n<\/ol>\n\n\n\n

      Some of the examples of discrete events could be rolling a dice or tossing a coin, counts of events are discrete functions. As there are no in-between values therefore these can be called as discrete distributions. For example, we can only get heads or tails in a coin toss and a number between (1-6) in a dice roll. Similarly, in a count of the number of books issued by a library per hour, you can count something like 10 or 11 books, but nothing in between.<\/p>\n\n\n\n

      In the dice roll example, the dice roll is a random variable, The probability of the dice landing on a number 2 can be written as P(X=2) = 1\/6. Where (capital letter), X, denotes the random variable and 2 is the outcome value. <\/p>\n\n\n\n

      Bernoulli Distribution<\/strong><\/h1>\n\n\n\n

      Before defining Bernoulli distribution let us understand some basic terms:<\/p>\n\n\n\n

      Bernoulli event:<\/strong> An event for which the probability of occurrence is p and the probability of the event not occurring is 1-p i.e., the event has only two possible outcomes (these can be viewed as Success or Failure, Yes or No and Heads or Tails). The event occurs with a probability p and 1-p respectively. <\/p>\n\n\n\n

      Bernoulli trial: <\/strong>A Bernoulli trial is an instantiation of a Bernoulli event. It is one of the simplest experiments that can be conducted in probability and statistics. It\u2019s an experiment where there are two possible outcomes (Success and Failure).<\/p>\n\n\n\n

      Examples of Bernoulli trials:<\/p>\n\n\n\n