The Sunday Blog: Dealing with numbers – Integer, Float, or, wait…

The Sunday Blog: Dealing with numbers

Integer, Float, or, wait…

Reading through the posts in the Nextion user forums or on social networks, I see that there are often questions about how to deal with float (not integer) numbers in Nextion code. Sometimes, people even complain and ask why the Nextion can’t handle directly floats while even this old and asthmatic 8bit AVR 328p MCU seems to be able to do so.

The answer is that your Arduino UNO can’t neither. You can declare float variables and do computations with them, but it’s not the AVR 8bit MCU which does this, it’s the C/C++ compiler which adds sometimes hundreds of instructions to your binary hex file to emulate this float processing. You can check this out by yourself in the Arduino IDE. First, compile this sketch, using integer variables:

int a, result;

void setup() {
  a = 1;
  Serial.begin(57600);
}

void loop() {
  result = a * a;
  Serial.print(result);
  delay(1000);
}

It will use 1810 bytes of program memory and 186 bytes of dynamic memory.

Now, do exactly the same with float variables:

float a, result;

void setup() {
  a = 1.0;
  Serial.begin(57600);
}

void loop() {
  result = a * a;
  Serial.print(result);
  delay(1000);
}

Now, it will use 3076 bytes of program memory and 200 bytes of dynamic memory.

Our program code has been blown up by +70%! And our dynamic storage which we’d expect to grow by 4 bytes for the two variables (floats take 4 bytes, integers 2 bytes on the AVR) has grown by 10 more bytes which the compiler reserved internally to store intermediate results during the emulated float multiplication.

And now imagine you are running a more complex program with more than only two float variables – you’ll quickly saturate the Arduino’s memory and have a slowly running code due to all the emulation stuff which the compiler adds in the background.

The solution is using a fixed decimal format

What is the fixed decimal or sometimes called fixed point format? It consists of reserving a fixed number of digits as decimals while continuing to compute and to deal with integers. You may read more about this technique in this article which I published in Summer 2020. And our Nextion HMI helps us greatly with the Xfloat component. Setting its .val attribute to 314159 and setting its .ws1 attribute to 5 will display 3.14159. Even if you have tens of Xfloat components to update, for example to display running motor data from your CAN bus, there is no need to have true floats.

Thus, by thoughtful design, you can not only display decimal numbers, you can also save much program and variable space on your Arduino and accelerate its code.

Again, an Arduino example

Let’s assume that we want to display a sensor value on our HMI. The ADC reads a value of two, but the result had to be scaled by 10/7, or 1.42857142857 to match the sensor’s physical properties. We need to display 4 decimals. This can be done in the classic way, computing the result and transforming it into a string on Arduino side before we display the result in a Text component on our HMI:

void setup() {
  Serial.begin(57600);
}

void loop() {
  String foo = String(float(2.0 * 1.42857142857),4);
  String bar = "t0.txt=\""+foo+"\"";
  Serial.print(bar);
  Serial.print("\xFF\xFF\xFF");
  delay(1000);
}

This “tiny” sketch compiles to a 4722 bytes tall hex file and will use 208 bytes of RAM on the Arduino. On the Nextion side, we have text which we could not simply re-use for further computations.

Now, let’s do the same thing with integers. Since we know that the last 4 digits will serve as decimals, we have to scale everything up by 10000 on Arduino side, while, already at design time, we set the ws1 attribute of our Xfloat component to 4 in order to compensate for this upscaling.

void setup() {
  Serial.begin(57600);
}

void loop() {
  int32_t foo = 2 * 14285;
  Serial.print("x0.val=");
  Serial.print(foo); 
  Serial.print("\xFF\xFF\xFF");
  delay(1000);
}

And, ho-ho-ho,  1814 instead of 4722 bytes – 2908 bytes saved already on such a tiny sketch! And we have the advantage that our Xfloat holds a numeric value which can be re-used in Nextion code.

A last, more practical, example

Let’s take the standard Arduino application for reading a LM35 temperature sensor. The example code (found on the Arduino website) uses float variables and constants to scale the ADC readings (which happen in steps of 5V/1024 = 4.8828mV) to the “true” temperature, since the LM35 outputs 10mV/°C. Thus, at 20°C, the LM35 outputs 200mV. Our ADC will read 41 because it can’t convert the true value (equivalent to a reading of 40.96) with enough precision. From this, that code will display 20.01953125°C which is misleading and shows a so-called wrong accuracy, since the ADC quantizes in steps slightly less than 0.5°C:

float temp;
int tempPin = 0;

void setup() {
   Serial.begin(9600);
}

void loop() {
   temp = analogRead(tempPin);
   // read analog volt from sensor and save to variable temp
   temp = temp * 0.48828125;
   // convert the analog volt to its temperature equivalent
   Serial.print("TEMPERATURE = ");
   Serial.print(temp); // display temperature value
   Serial.print("*C");
   Serial.println();
   delay(1000); // update sensor reading each one second
}

It compiles to 3202 + 222 bytes.  Let’s now, in a first step, “Nextionify” it, by modifying the Serial.print stuff to display the result in a Text component as in the example above, already leaving only four decimals to eliminate that the impression of wrong accuracy:

float temp;
int tempPin = 0;

void setup() {
   Serial.begin(9600);
}

void loop() {
   temp = analogRead(tempPin);
   // read analog volt from sensor and save to variable temp
   String foo = String(float(temp * 0.48828125),4);
   // convert the analog volt to its temperature equivalent
   Serial.print("t0.txt=\""+foo+"\""); // display temperature value
   Serial.print("\xFF\xFF\xFF");
   delay(1000); // update sensor reading each one second
}

Now, with the explicit conversion of the float to a String with only 4 decimals, the code blows up to 5198 + 206 bytes, but it is Nextionified. And I bet it is still smaller as if we included whatever Nextion library to handle the communication.

Let’s now modify everything to work with integers and the Xfloat component:

int32_t temp;
int tempPin = 0;

void setup() {
   Serial.begin(9600);
}

void loop() {
   temp = analogRead(tempPin);
   // read analog volt from sensor and save to variable temp
   temp*=4883;
   // convert the analog volt to its temperature equivalent
   Serial.print("x0.val=");
   Serial.print(temp); // display temperature value
   Serial.print("\xFF\xFF\xFF");
   delay(1000); // update sensor reading each one second
}

Oh, surprise! It compiles to 1996 + 202 bytes, much less than the original sketch but with the Nextion communication added. And the displayed results are identical, at least if you do not forget to set the ws1 attribute of x0 to 4 at design time.

We learn that working with fixed decimal formats instead of floats brings huge advantages, not only on Nextion but even more on the MCU (Arduino) side.

Happy Nextioning!