Home | Contact Us | FAQ | Search & Site Map | Link to Us
Sign In | Join | Other 45 Sites in Network
HomeAnnouncementsWhite Papers
Discussion GroupsFirst AidDatabasesJavaBeansGUIJava 3DVirtual MachineCORBASecurityToolsGeneral
Java DirectoryOpen Source ProjectsSample Book ChaptersUser GroupsWeb Resources
Related Topics
Databases.NETMore Topics ...

Java Forum / General / September 2006

Tip: Looking for answers? Try searching our database.

Unescaping Unicode code points in a Java string

Thread view: 
Greg - 31 Aug 2006 07:51 GMT
My Java program reads in (from an external source) text that contains
the same sort of unicode character escape sequences as java source
code. For example, one such string might be:

    "En Espa\u00f1ol"

Naturally, I would like to convert the five characters subsequence,
"\u00f1", into the single character codepoint (hex 00F1) that those
characters actually represent:

     "En Español"

I've been browsing the J2SE 1.5 docs hoping to find a convenient method
to perform this kind of conversion, but so far have not found one. Does
anyone have any suggestions?

Thanks,

Greg
Thomas Fritsch - 31 Aug 2006 12:46 GMT
> My Java program reads in (from an external source) text that contains
> the same sort of unicode character escape sequences as java source
[quoted text clipped - 11 lines]
> to perform this kind of conversion, but so far have not found one. Does
> anyone have any suggestions?

Long time ago I searched the Java API and sources for a method doing
that kind of String decoding, but to no avail. The only thing I found
was method
    private String loadConvert(String)
in class java.util.Properties. But because it is private, it is not
reusable outside Properties.

(You find the source in src.zip of JDK installation directory)

Signature

Thomas

Oliver Wong - 31 Aug 2006 14:47 GMT
> My Java program reads in (from an external source) text that contains
> the same sort of unicode character escape sequences as java source
[quoted text clipped - 11 lines]
> to perform this kind of conversion, but so far have not found one. Does
> anyone have any suggestions?

   Iterate through each character of the String, looking for the sequence
"\u". If you find it, delete those two chars, and read in the next 4 chars.
Parse that sequence of 4 characters into a integer assuming hexadecimal
notation. Take that integer and cast it to a char, and insert the resulting
char back into the String.

   - Oliver
Dale King - 01 Sep 2006 06:09 GMT
>> My Java program reads in (from an external source) text that contains
>> the same sort of unicode character escape sequences as java source
[quoted text clipped - 17 lines]
> assuming hexadecimal notation. Take that integer and cast it to a char,
> and insert the resulting char back into the String.

It's a bit more complicated than that because you will also need to
support things like \\ to actually insert a backslash and perhaps
support things like \n.

Signature

 Dale King

David Lee Lambert - 01 Sep 2006 14:16 GMT
>>> My Java program reads in (from an external source) text that contains
>>> the same sort of unicode character escape sequences as java source
[quoted text clipped - 11 lines]
> support things like \\ to actually insert a backslash and perhaps
> support things like \n.

If he is defining a new specification for escaped input,  this would be
nice but not necessary.  "\" can be escaped as "\u005C", and a newline
as "\u000A".  In Java source code,  "\u005C" results in a malformed string
literal (which means one needs to use "\n" instead),  but that escape
sequence is permitted in properties files.  On the other hand,  the Java
compiler and Properties.load() do not recognize the C escape-sequences
"\v" and "\a" for VT and BEL.

I think Arne's response (that used a regular expression) was too
complicated,  and the response to which you are responding was
poorly-thought-out (because strings are immutable in Java).  Here's a
possible solution:

  String unescape(String s) {
     int i=0,len=s.length(); char c; StringBuffer sb = new StringBuffer(len);
     while (i<len) {
    c = s.charAt(i++);
    if (c=='\\') {
       if (i<len) {
          c = s.charAt(i++);
          if (c=='u') {
         c = (char) Integer.parseInt(s.substring(i,i+4),16);
         i += 4;
          } // add other cases here as desired...
       }} // fall through: \ escapes itself, quotes any character but u
    sb.append(c);
     }
     return sb.toString();
  }

Unlike Arne's solution,  it examines each character in the string only
once,  and it doesn't require the java.util.regex package (which was not
introduced until Java 1.4).  I also think it's more readable,  to one who
is trying to verify that it does exactly what's expected and no more.

(What would Arne's solution do to "\u005Cu0020\u0020"?  Is that the
correct result?)

Signature

PGP key posted on website ... http://www.lmert.com/people/davidl/

Dale King - 01 Sep 2006 15:28 GMT
>>>> My Java program reads in (from an external source) text that contains
>>>> the same sort of unicode character escape sequences as java source
[quoted text clipped - 16 lines]
> literal (which means one needs to use "\n" instead),  but that escape
> sequence is permitted in properties files.

It's up to him what he wants to specify, but personally I would prefer
the \\ and \n.

> On the other hand,  the Java
> compiler and Properties.load() do not recognize the C escape-sequences
> "\v" and "\a" for VT and BEL.

Which is understandable. BEL is specific to consoles and Java has no
real support for consoles because they are too platform specific and VT
is rarely used.

> I think Arne's response (that used a regular expression) was too
> complicated,  and the response to which you are responding was
> poorly-thought-out (because strings are immutable in Java).  Here's a
> possible solution:
>
>    String unescape(String s) {

The proper time to do the conversion is when the text is being read from
the "external source" using some form of FilterReader subclass. I
remember now that I wrote one of those once, but after a long search I
have figured out that I left that code at my previous employer and did
not keep a copy of it (which is a shame because that was part of
something that was some really good work).

Signature

 Dale King

Arne Vajhøj - 01 Sep 2006 01:33 GMT
> My Java program reads in (from an external source) text that contains
> the same sort of unicode character escape sequences as java source
[quoted text clipped - 11 lines]
> to perform this kind of conversion, but so far have not found one. Does
> anyone have any suggestions?

One of many possible solutions:

    private static final Pattern p = Pattern.compile("\\\\u([0-9A-F]{4})");
    public static String U2U(String s) {
        String res = s;
        Matcher m = p.matcher(res);
        while(m.find()) {
            res = res.replaceAll("\\" + m.group(0),
Character.toString((char)Integer.parseInt(m.group(1), 16)));
        }
        return res;
    }

Arne


Free Magazines

Get these publications absolutely FREE for up to 12 months. There are no hidden fees and no obligation. Simply choose a title, complete the application form and submit it. Read more ...

Oracle MagazineNetwork ComputingComputer WorldBio-IT WorldeWeekInformation WeekInfosecurity
 
Sign In
Join
My Latest Posts
My Monitored Threads
My Blog
My Photo Gallery
My Profile
My Homepage

Start New Thread
Enable EMail Alerts
Rate this Thread



©2008 Advenet LLC   Privacy Policy - Terms of Use
This website includes both content owned or controlled by Advenet as well as content owned or controlled by third parties.