# Unforeseen behaviour of fix() in Octave and Scilab

I encountered some odd results when using the function fix() in Octave and Scilab. The following is input and output of the Scilab console but the exact same thing happens in Octave.

A=[1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0;
0,0,0,1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0;
0,0,0,0,0,0,0,1,1,1,1,1,0,0,0,0,0,0,0;
0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,0,0,0;
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1;
1,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0;
0,1,0,0,1,0,0,0,1,0,0,0,1,0,0,0,0,0,0;
0,0,1,0,0,1,0,0,0,1,0,0,0,1,0,0,1,0,0;
0,0,0,0,0,0,1,0,0,0,1,0,0,0,1,0,0,1,0;
0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,1;
0,0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,0;
0,1,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0;
1,0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,1;
0,0,0,1,0,0,0,0,1,0,0,0,0,1,0,0,0,1,0]


I want to take it to the reduced row echelon form:

-->rref(A)
ans  =

1.   0.   0.   0.   0.   0.   0.   0.   0.   1.   1.   0.   0.   1.   2.   1.   0.   1.          1.
0.   1.   0.   0.   0.   0.   0.   0.   0.  -1.   0.   0.   0.  -1.  -1.   0.   0.   0.          0.
0.   0.   1.   0.   0.   0.   0.   0.   0.   0.  -1.   0.   0.   0.  -1.  -1.   0.  -1.         -1.
0.   0.   0.   1.   0.   0.   0.   0.   0.  -1.  -1.   0.   0.   0.  -1.   0.   0.  -1.388D-16   0.
0.   0.   0.   0.   1.   0.   0.   0.   0.   0.  -1.   0.   0.  -1.  -1.  -1.   0.  -1.          0.
0.   0.   0.   0.   0.   1.   0.   0.   0.   1.   1.   0.   0.   1.   1.   1.   0.   0.          0.
0.   0.   0.   0.   0.   0.   1.   0.   0.   0.   1.   0.   0.   0.   1.   0.   0.   1.          0.
0.   0.   0.   0.   0.   0.   0.   1.   0.   0.   0.   0.   0.  -1.  -1.  -1.   0.  -1.         -1.
0.   0.   0.   0.   0.   0.   0.   0.   1.   1.   1.   0.   0.   1.   1.   0.   0.   1.          0.
0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   1.   0.   0.   0.   1.   0.   0.          1.
0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   1.   1.   1.   1.   0.   0.          0.
0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   1.   1.          1.
0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.          0.
0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.          0.


So far so good, I guess, but when I now use fix() on the last obtained matrix to get integer entries, I get zeros at (6,10) and (6,11), where I expected ones:

-->fix(rref(A))
ans  =

1.   0.   0.   0.   0.   0.   0.   0.   0.   0.   1.   0.   0.   1.   2.   1.   0.   1.   1.
0.   1.   0.   0.   0.   0.   0.   0.   0.  -1.   0.   0.   0.  -1.  -1.   0.   0.   0.   0.
0.   0.   1.   0.   0.   0.   0.   0.   0.   0.  -1.   0.   0.   0.  -1.  -1.   0.  -1.  -1.
0.   0.   0.   1.   0.   0.   0.   0.   0.  -1.  -1.   0.   0.   0.  -1.   0.   0.   0.   0.
0.   0.   0.   0.   1.   0.   0.   0.   0.   0.  -1.   0.   0.  -1.  -1.  -1.   0.  -1.   0.
0.   0.   0.   0.   0.   1.   0.   0.   0.   0.   0.   0.   0.   1.   1.   1.   0.   0.   0.
0.   0.   0.   0.   0.   0.   1.   0.   0.   0.   1.   0.   0.   0.   1.   0.   0.   1.   0.
0.   0.   0.   0.   0.   0.   0.   1.   0.   0.   0.   0.   0.  -1.  -1.  -1.   0.   0.  -1.
0.   0.   0.   0.   0.   0.   0.   0.   1.   0.   0.   0.   0.   1.   1.   0.   0.   0.   0.
0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   1.   0.   0.   0.   1.   0.   0.   1.
0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   1.   1.   1.   1.   0.   0.   0.
0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   1.   1.   1.
0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.
0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.   0.


What happened there?

• I get 1's using MATLAB. But perhaps the (6,10) and (6,11) elements of rref(A) as computed in Octave and Scilab are computed as being slightly less than 1, hence fix to 0. – Mark L. Stone Jan 5 at 13:42
• ^^ this is the reason. You probably want to use round, not fix here. – Federico Poloni Jan 5 at 14:30
• Thanks! That is what I suspected, too. But why don't they display it in a way more indicative of that like "0.9999"? Can I force them to do that? – Christian Sander Jan 5 at 16:17
• @MarkL.Stone You are right, it is somewhat surprising. After some thought I figured out exactly why. eps, or eps(1), is, by definition, the distance between 1 and the next floating-point number. However, since the exponent of the floating-point representation changes exactly at 1, the previous floating-point number is 1-eps/2. So when the result of an operation is smaller than 1-eps/4, it rounds down to 1-eps/2, when it is larger it rounds up to 1. IEEE specifies round-to-even, so when it is exactly 1-eps/4 it rounds to 1, because the least significant bit of 1 is zero. – Federico Poloni Jan 5 at 22:03
• @Federico Poloni Yes, and as it turns out , 1+eps/2 is the boundary for being > 1. – Mark L. Stone Jan 5 at 22:23