problems with string conversions

[jpc]

I am so lucky to have a client that wants his programs to be completely idiot proof ( we will never get there unfortunately) and he is absolutely fabulous where it comes to testing.
In the current situation he insist in trying to input the least likely data where we need a numerical input so where i would expect input like say '15' he will try '££'.
Now here is the problem, after studying a bit the issue i noticed that the very simple conversions like strtoword and strtoint behave very bad in such situations.
Would it be asked too much that these functions return zero in such cases?

as example perhaps try this :

Code: Select all

number := strtoint('aa');
number:= strtoword('aa');

i personally would not think of inputting such data but the result is in fact frightening.

[filip]

Hi,

I can make this change for you if you want, and attach the EMCL file here.

Can you give more directions, so I do this according to your needs ?

Regards,
Filip.

[jpc]

hello Filip,

i do not exactly understand your proposal, it seems to me that any string that would not contain numbers should return a zero value.

[janni]

If I may suggest, it's better to have the wrong string signaled with maximum value - it's usually much less common than zero and thus easier to isolate for diagnostics. That's how I solved it in my version of Conversions library (for PIC18s only, though) and never regretted it.

[jpc]

Janni,

i fully agree that this would be a better solution that would allow for easy validating of any string to number conversion.
the current situation however is that we can expect all sorts of results from illegal input , the only solution i have at the moment is to parse the string for any character other than 0..9 and generate a warning for the user if he is so foolish to provide that input.

[jpc]

ok, did some more research ( in fact this time on small pic but it looks like this is one and the same in all compilers.
i ran this in the debugger and it was immediately clear how primitive these conversions are

Code: Select all

byte_number := strtoint('a');           // 49
byte_number := lo(strtoint('b'));       // 50
byte_number := lo(strtoint('aa'));      // 27
byte_number := lo(strtoint('bb'));      // 38
word_number := strtoword('a');          // 49
word_number := strtoword('b');          // 50
word_number := strtoword('aa');         // 539
word_number := strtoword('bb');         // 550

so the 'conversion' simply takes the characters from the string, without any verifucation subtracts 48 and then assumes this to be a numerical value.
following digits are multiplying the current value by 10 to add the next 'digit'.

I am suprised to see such primitive and dangerous conversion exist over so many years of compilers without ever beeing noticed/repaired

[janni]

I am suprised to see such primitive and dangerous conversion exist over so many years of compilers without ever beeing noticed/repaired

Oh, it has been noticed and similar (or worse) problems may be found in other libraries. I've had strong motivation to rewrite all libraries that I use and wasn't alone in such efforts.

[filip]

Hi,

@jpc,
Doers the library from the attachment solves your issue ?
It returns 0xFF if any string character is not a number.

Regards,
Filip.

[jpc]

Filip,

what is in this library? and for which compiler? this is a general issue present in all compilerversions i use and very likely in all others as well. (all PIC Pascal compilers, ARM compiler tested, everywhere the same results)
I secured my code by filtering before using these conversions but i expect a fix of this library in all compilers, i consider this a rather stupid but dangerous bug that is probably very simple to fix .

[filip]

Hi,

I thought you wanted the library for mikroPascal PRO for PIC32, so I modified the StrToXXX functions according to your first post.

I will suggest our developers to implement the similar mechanism to other compilers as well.

Regards,
Filip.

[jpc]

Filip,

ok, i tested the lib you provided but it is not good enough, strtoint and strtoword both return 0xff as value, the proposal as made by Janni would we be to return 0xFFFF as these results are 16 bit so please change that in ALL compilers that have these functions( as far as i could check all Pascal and Basic compilers).

[Dany]

Please do not change these functions. I think it is a bad idea.
Their precondition says

The string is assumed to be a correct representation of a number.

(from the help). The programmer using the function should make sure the precondition is met.
Additionally, after the proposed change, one can e.g. not know any more if the inputstring was '65535' or contains an error (example for strtoint).

Perhaps it is wiser to create extra functions with a built in check on validity of the string characters.

[janni]

Additionally, after the proposed change, one can e.g. not know any more if the inputstring was '65535' or contains an error (example for strtoint).

Isn't it exactly so now ? (only with any resulting number - see jpc's results above)

[jpc]

in a way i think Dany has a point where he refers to the help, i must admit i never took the "The string is assumed to be a correct representation of a number." in account but it seems to me rather simple to have the validity check part of these functions, verification in front of these functions creates a lot more overhead and i imagine we could live with the sacrifice of one well identifiable returnvalue instead so i still would like this to be changed in the standard library.

[Dany]

in a way i think Dany has a point where he refers to the help, i must admit i never took the "The string is assumed to be a correct representation of a number." in account but it seems to me rather simple to have the validity check part of these functions, verification in front of these functions creates a lot more overhead and i imagine we could live with the sacrifice of one well identifiable returnvalue instead so i still would like this to be changed in the standard library.

Why not keep the original routines and make extra ones (with the validity check)? Doing so the user can choose himself.