Type RW_toKEN = Record token_str :String[9]; token_cod :toKEN_CODE; end; RW_Type = Array[0..9] of RW_toKEN; RWT_PTR = ^RW_Type; Const NULL = ''; Rw_2 :RW_Type = ((token_str : 'do'; token_cod : tdo), (token_str : 'if'; token_cod : tif), (token_str : 'in'; token_cod : tin), (token_str : 'of'; token_cod : tof), (token_str : 'or'; token_cod : tor), (token_str : 'to'; token_cod : tto), (token_str : NULL; token_cod : NO_toKEN), (token_str : NULL; token_cod : NO_toKEN), (token_str : NULL; token_cod : NO_toKEN), (token_str : NULL; token_cod : NO_toKEN) ); ...the difference being the explicit declaration of the Constant Record fields. (I'm used to Array Constants, not Record Constants - I was unaware of the requirement) PARSinG NUMBERS Now we'll concentrate on parsing Integer and Real numbers. The Pascal definition of a number begins With an UNSIGNED Integer. An unsigned Integer consists of one or more consecutive DIGITS. The simplest Form of a number token is an unsigned Integer: 1 9 120 12654 A number token can also be an unsigned Integer (the whole part) followed by a fraction part. A fraction part consists of a decimal point followed by an unsigned Integer, such as: 123.45 0.9987564 These numbers have whole parts 123 and 0 respectively, and fraction parts .45 and .9987564 respectively. A number token can also be a whole part followed by an EXPONENT part. An exponent part consists of an "E" (or "e") followed by an unsigned Integer. An optional exponent sign, + or -, can appear between the letter and the first exponent digit. Examples: 134e2 2E99 123e-45 73623E+4 Finally, a number token can be a whole part followed by a fraction part and an exponent part, in that order: 2.3498E7 0.00034e-66 I arbitrarily limit the number of digits to 20, and the exponent value from -37 to +37 - the exact value necessary to limit this value is dependant on how Real values are represented on the Computer. The "get_number" Function is likely to be the biggest Function in your scanner, but it should be relatively straighForward to code...in light of what has already been done With the scanner/ tokenizer module, and the definition of a number. EXERCISE #1 Write the get_number Function to parse Integers and Real numbers. You will need to add the following Types and Variables to your global data segment: Type { add "Real"s to list... } LITERAL_Type = (Integer_LIT, Real_LIT, String_LIT); LITERAL_REC = Record Case lType:LITERAL_Type of Integer_LIT: (ivalue :Integer); Real_LIT : (rvalue :Real ); String_LIT : (svalue :String ); end; Var digit_count :Word; count_error :Boolean; -------------- PART 2 --------------------------------------- The rest of this post will cover two simple topics - parsing Strings inside quotes, and parsing comments. PARSinG COMMENTS {} The Compiler should ignore the input between two curly braces ({}), and the curly braces themselves. My scanner is written so the entire comment is replace by a Single blank (" "), although you could possibly Write the scanner so that comments are _totally_ ignored. EXERCISE #2: Integrate COMMENT detection into the get_Char routine, so that when your Character fetching routine will ignore comments and pass a blank when a comment is encountered, skipping the comment entirely For the next fetch. Make sure that the routine keeps reading Until the right curly brace is detected, even past the end-of-line. if the end-of-File is encountered beFore the right curly brace is found, an "unexpected end" error should be generated. PARSinG StringS (QUOTES) '' The quote Character delimits Strings, any Character between the Strings is ignored by the Compiler, except to stored as a String LITERAL. if you wish a ' (quote) to be included in the literal, and extra ' must precede it. One possible tricky area is the {} (comment) Character. You must be careful not to inadvertently trigger the comment routine within the quote routine While reading a String, otherwise you will have a BUG. EXERCISE #3: Add a quote routine to the get_token routine within your module, to fetch Strings, as a LITERAL IDENTifIER when the QUOTE Character is detected. The following mods to your Types are required: Eof_Char = #$7F; Type Char_CODE = (LETTER, DIGIT, QUOTE, SPECIAL, Eof_CODE); { The following code init's the Character maping table: } Var ch :Byte; begin For ch := 0 to 255 do Char_table[ch] := SPECIAL; For ch := ord('0') to ord('9') do Char_table[ch] := DIGIT; For ch := ord('A') to ord('Z') do Char_table[ch] := LETTER; For ch := ord('a') to ord('z') do Char_table[ch] := LETTER; Char_table[ord(Eof_Char)] := Eof_CODE; Char_table[39] := QUOTE; end; ---------------------------------------------------------------- PLEASE, please let me know what you think about these posts, even if they're negative - I want to have some feedback on the difficulties, and whether or not people are having trouble following the material - I _can_ be more concise at the cost of being more verbose - if it's needed! if you are having problems With your source code, and want me to do a detailed examination of your code, expecially if it's written in a language other than Pascal, send me email via the Internet - to avoid "carpet bombing" the conference with undesired material. NEXT POST: Error codes, and putting your code to the test - our first utility (other than the lister) : a source Program Compactor (not cruncher). FUTURE POSTS: - Review and (hopefully) a status report from "students" - Symbol table - YA utility (cross - referencer) - YA utility (source Program CRUNCHer) - YA utility (source Program UNcruncher) - Parsing simple expressions - Utility : CALC, using infix-to-postfix conversions and stack ops. - Parsing statements - Utility: Pascal syntax checker part I - Parsing declarations (Var, Type, etc) incl's: much improved (and much more Complex) symbol table - Utility: Declarations analyzer. - Syntax Checker part II - Parsing Program, Procedure, and Function declarations (routines). - Syntax checker Part III - Review and discussion?