(closes issue #14022)

Reported by: wetwired
Tested by: murf

I checked, and I added a mod to the trunk version
of Asterisk that would make it 8-bit transparent
on 27 Nov 2007, but I made no such updates to
1.4. My best guess is that 1.4 was released, and
it was not appropriate to commit an enhancement.

But I'm going to add the same fixes to 1.4 now,
for the following reasons:
1. wetwired is correct; 1.4 is **mostly** 8-bit
   transparent now. This is because the lexical
   token forming rules use . in most 'word'
   state continuances.  It's just the beginning
   of a 'word' that is picky.
2. Accepting 8-bit chars in some places and
   not others leads to bug reports like this.




git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/1.4@162671 65c4cc65-6c06-0410-ace0-fbb531ad65f3

pbx/ael/ael.flex

@@ -211,6 +211,8 @@ NOARGG		([^(),\{\}\[\]]|\\[,()\[\]\{\}])*
 
 NOSEMIC		([^;()\{\}\[\]]|\\[;()\[\]\{\}])*
 
+HIBIT		[\x80-\xff]
+
 %%
 
 \{		{ STORE_POS; return LC;}
@@ -265,7 +267,7 @@ includes	{ STORE_POS; return KW_INCLUDES;}
 [ ]+		{ my_col += yyleng; }
 [\t]+		{ my_col += (yyleng*8)-(my_col%8); }
 
-({KEYWORD}?[-a-zA-Z0-9'"_/.\<\>\*\+!$#\[\]]|(\\.)|(\$\{)|(\$\[)) {
+({KEYWORD}?[-a-zA-Z0-9'"_/.\<\>\*\+!$#\[\]]|{HIBIT}|(\\.)|(\$\{)|(\$\[)) {
 		/* boy did I open a can of worms when I changed the lexical token "word".
 		all the above keywords can be used as a beginning to a "word".-
 		before, a "word" would match a longer sequence than the above
@@ -292,6 +294,7 @@ includes	{ STORE_POS; return KW_INCLUDES;}
 	} 
 
 <wordstate>[-a-zA-Z0-9'"_/.\<\>\*\+!$#\[\]] { yymore(); /* Keep going */ }
+<wordstate>{HIBIT} { yymore(); /* Keep going */ }
 <wordstate>(\\.)  { yymore(); /* Keep Going */ }
 <wordstate>(\$\{)  { /* the beginning of a ${} construct. prepare and pop into curlystate */
 		parencount2 = 0;