General.   In daily life calculating a proportion is a matter of routine. In nearly every newspaper or TV-news there are proportions stated concerning very different matters. Nothing can be easier – there are a number of 'events' amongst a number of 'reported' and from this info a simple calculation gives a proportion, often expressed as a percentage.
Very often there are two (or more) percentages that are to be compared. In the newspaper or the TV-show this is very simple: a simple glance will tell that one is larger than the other and based upon this the story is told.

Not so easy.   However, this situation is not as simple as it looks. The confidence of the two percentages depends on how big the 'reported' number is. And this fact is never included when media draws their conclusions. It turns out that mathematically the comparison is rather complicated and needs computer support in order to perform the calculations. This simulation does not contain any formal discussion of the difficulties but offers only an illustration. Use also the exercises under the [ Exercises ]-button.
(See http://www.indstat.se/simuleringar/simuleringar.php and "Ett antal...".)

The so-called "Fisher´s exact method" is one way to perform a formal test of the data.

Background.   This simulation is triggered by some TV-news on the 3rd of June 2012. It was stated that 22 of 164 (13%) of one type of doctors had a remark. For another type of doctors it was 8 of 109 (7%). The TV-conclusion was obvious, there is a substantial difference.
But a formal test shows that there is a 17% risk of being wrong in this statement. This is a risk that no scientist or quality engineer would accept.
During the same weeks the news about the Higg´s particle was released. The scientists were 99.99994% sure of being correct. A simple comparison: 17/(100-99.99994) = 280000, i.e. there was a willingness of taking a nearly 300000 higher risk with people´s daily life and health compared to identifing a particle that was reported to be of no practical value!

••••

Below there are four slides that can be used to change the parameters. The slides can either be dragged using the cursor or by the arrows on the keyboard. It is also possible to click on the horisontal scale to quickly move the slide to a new position. (The endpoints of the scales cannot be changed.)

NB that below only the very last simulated result is presented.

Exercise 1 – the inital parameter values
The inital parameters are the two fault rates pA = pB = 0.05 and nA = nB = 200 (dots). This means that we expect the number of faults in one such sample of 200 dots to be 0.05*200=10 dots and that the expected difference between the fault rates is 0.000 (see the third histogram.)
However, because of random reasons, the difference betwee the calculated fault rates can be positive or negative. Open the parameter window [Change parameters] and then click [Repeat simulation] several times. Notice that 'Number of events' is around 10.
More important, notice how the results in the third histogram vary from, say, -0.06 to +0.06 (sometimes even more). Thus there is a large risk that e.g. a newsdesk or a newspaper draws the conclusion that there is a difference in fault rates when in fact there is no such difference!

Exercise 2 – change the parameters
Change the four parameters to the following:
              - 'Number of type A' to 300
              - 'pA' to 0.05
              - 'pA' to 0.08
              - 'Number of simulations' to 240

Click [Repeat simulation] several times. Notice that even if there is a negative difference in fault rates, some results are showing a positive result (i.e. right of the blue vertical line). Thus there is a probability that a decision based on a sample is completetly wrong!
Note also that the middle histogram shows a larger variation, compared to the top one. Each calculated 'B'-result is based on fewer values and this leads to a larger variation, and this also has an influence of the interpretation of the result.

••••

Some conclusions

After reading the 'info'-fields and performing the exercises, it is obvious that a comparison of two (or more) proportions is not as easy at it looks. This simulation does not discuss the theoretical issues involved. However, in order to do a reliable analysis it is necessary to know the basic about the situation at hand.

The distributions. The 'number of A'-events is a so-called binomial distribution with the parameters (pA, nA) and corresponding for 'number of B'-events. The difference, which is of central interest, has no standard form and the analysis really need a computer support.

Inference. The situation involves also the notion of testing of a hypothesis and a so-called confidence interval which is a very practical tool as it exactly states what conclusions can or cannot be drawn. These tools belong to the area of Statistical inference. (See e.g. the simulation of 'Konfidensintervall' for more info.)

••••

The dots are randomly distributed across the square and might represent a number of installed components or a number of animals or a number of trees or a number or medical centers, etc. (The very position in 'X' or 'Y' has no meaning here.)

Every dot is an 'item' and there are two types of 'items' (different colours). An 'item' can be either OK/notOK, an animal can be healthy or not helathy, etc. Incorrect items have a red circle around the dot.

Two buttons. The upper left corner of the square contains two buttons. One is labeled [Repeat simulation] and will repeat a simulation with the chosen values of the parameters. The other button is labeled [Change parameters] and can be used to change the parameters (when clicked, a new window will open.)

••••

The histogram shows the simulated 'pA'-results, i.e. calculated fault rates using the number of 'faults found' divided by 'number of inspected items'. The upper left corner shows the parameters used for this particular simulation.

The number of values making the histogram is the 'number of simulations', shown under the button [Change parameters].
The X-axis contains two triangles: the red one shows the expected value (i.e. pA) while the black one indicates the average of the calculated fault rates.
If the histogram is drawn outside the X-axis, new values of x-max (or x-min) can be entered by clicking current values.
(See also the histogram over 'pB-results'.)

••••

The histogram shows the simulated 'pB'-results, i.e. calculated fault rates using the number of 'faults found' divided by 'number of inspected items'. The upper left corner shows the parameters used for this particular simulation.

The number of values making the histogram is the 'number of simulations', shown under the button [Change parameters].
The X-axis contains two triangles: the red one shows the expected value (i.e. pB) while the black one indicates the average of the calculated fault rates.
If the histogram is drawn outside the X-axis, new values of x-max (or x-min) can be entered by clicking current values.
(See also the histogram over 'pA-results'.)

••••

Each simulation gives a 'pA'- and a 'pB'-value (giving the two histograms above). The differences 'pA-pB' are shown here. The red triangle shows the expected value, the blue line shows x=0. These differences are of main interest when comparing two proportions. Using the initial parameter settings (pA = pB = 0.05, nA = 200, nB = 200) it is very obvious that a calculated difference can be either negative or positive!

If reported in e.g. TV or a daily newspaper this result is often understood as a truth without any hesitation and thus a common source for incorrect decisions and creations of rumours. With knowledge in basic statistics it is clear that this data needs a more formal analysis.

••••

Info om in/utdata