'Use of real types can result in precision problems'

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2004-11-16 08:35 PM
The line where the warning points to have IF statements like these:
IF B=0 THEN ....
IF anyparameter<>0 THEN...
Despite the warnings, the object seems to work perfectly.
What 'real types' is AC talking about??

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2004-11-17 04:32 AM
I answered this here:
AFAIK, this is only a checkscript warning/aid ... and will not affect the running of an otherwise correct script.
Karl
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2004-11-17 09:34 AM
I have had code like this fail:
IF angle = 90 THEN......and replaced it with the following which worked:
IF angle > 89.99 & angle < 90.01 THEN...

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2004-11-17 03:05 PM
Karl wrote:
AFAIK, this is only a checkscript warning/aid ... and will not affect the running of an otherwise correct script.
In the referred script, I have tens of IFs triggering that warning. So, I wouldn't call it an 'aid', because it is very annoying having to click again and again the 'continue' button every time I need to check for actual errors. I must find a way to get reed of the 'aid', to preserve my finger and my mental health...
I found out that replacing 'IF parameter=0 THEN...' by 'IF NOT(parameter) THEN...' removes the warning. But in other cases, I still didn't figure an efficient alternative.
Matthew wrote:
I have had code like this fail:IF angle = 90 THEN...
...and replaced it with the following which worked:IF angle > 89.99 & angle < 90.01 THEN...
What if your angle is 89.991? IMHO, this is actually the one that 'can result in precision problems'... Maybe 'IF NOT(angle-90) THEN...' is a better option. I will test it now.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2004-11-18 01:19 AM
Durval wrote:If you need tighter precision use 89.9999. I can't believe that a accuracy to 1/10,000 of a degree can possibly matter to you.Matthew wrote:
I have had code like this fail:IF angle = 90 THEN...
...and replaced it with the following which worked:IF angle > 89.99 & angle < 90.01 THEN...
What if your angle is 89.991? IMHO, this is actually the one that 'can result in precision problems'... Maybe 'IF NOT(angle-90) THEN...' is a better option. I will test it now.
The NOT() function may only work with boolean (true/false, checkbox) parameters. I'd be curious to know if it works though.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2004-11-18 06:02 AM
Durval wrote:I agree. There should be some preference settings for Check Script. Want to post it to the wish list?
In the referred script, I have tens of IFs triggering that warning. So, I wouldn't call it an 'aid', because it is very annoying having to click again and again the 'continue' button every time I need to check for actual errors. I must find a way to get reed of the 'aid', to preserve my finger and my mental health...
Karl

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2004-11-18 06:10 AM
Matthew wrote:It is perhaps better to use:IF angle = 90 THEN...
epsilon = 0.001 ... IF ABS(angle-90) < epsilon THENwhere you can specify epsilon once in the master script, for example.
Durval's
IF NOT(angle-90) THENis no different than saying angle=90, and so would still have potential representational/precision errors and fail (unless the NOT operator does more than zero/non-zero testing and tests for an epsilon factor).
Karl

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2004-11-18 07:47 PM
Matthew wrote:
If you need tighter precision use 89.9999. I can't believe that a accuracy to 1/10,000 of a degree can possibly matter to you.
The accuracy should suffice if value of 'angle' is directly typed by user. But if 'angle' stores a results from a formula in the script, even 1/1,000,000 could lead to 'precision problems', couldn't it?
Matthew wrote:
The NOT() function may only work with boolean (true/false, checkbox) parameters. I'd be curious to know if it works though.
Yes, it works, I just tested. 'NOT(x)' results true if x=0, and results false if x is anything but zero (not necessarily 1).

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2004-11-18 07:57 PM
Karl wrote:...but NOT(angle-90) doesn't trigger the precision warning, while angle=90 does, god knows why....
Durval'sIF NOT(angle-90) THENis no different than saying angle=90,...

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2004-11-18 08:10 PM
Durval wrote:It sort of makes sense to not trigger the warning since the result is to be used for simply a zero/non-zero test ... sort of a boolean conversion. Still subject to problems, but I imagine that's their logic.
...but NOT(angle-90) doesn't trigger the precision warning, while angle=90 does, god knows why....
The point I was trying to make is that the warning could be real, and I wanted to make sure that less experienced GDL programmers didn't use your 'trick' of NOT(expression) to make the message go away, when really they need the 'epsilon' code for the code to work properly.
For example, in you case, if you KNOW that angle is going to 'snap' to 90, 180, 270, and 0, then your code is fine. The value of 90 may not really be 90 exactly (binary rep), but the representation of 90 will always equal the representation of 90. Right?
But, if the angle is
As an addition to the wish for always turning the warning on or off globally, we could ask that GDL include PRAGMAS as in other languages, so that we could write something like:
#PRAGMA noerrorchecking IF angle=90 THEN ... #PRAGMA errorcheckingTo turn the warning off on code that we know is fine.
Karl