Unescape a String that contains standard Java escape sequences

I came across a problem that I needed to parse a String (coming from a database, file or web service) that contains standard Java escape sequences and had to convert the escape sequences to the proper characters.

Java escape sequences can be placed in a string literal in three ways:

  1. Standard escapes with \b \f \n \r \t \” \’ : These represent the standard control characters BS, FF, NL, CR, TAB and the double and single quote.
  2. Octal escapes with \0 to \377 : These represent a single character (0-255 decimal, 0x00-0xff in hexadecimal) in octal notation.
  3. Hexadecimal Unicode character with \uXXXX : A hexadecimal representation of a Unicode character.
  4. The \ character itself with \\ : If you want to type the \ character itself.

Though this an integral part of the Java compiler when compiling code, there is no function in the standard Java runtime, that will convert such a String notation into an unescaped target String.

Apache Commons has a StringUtils class, that can do this, but this requires a lot of overhead for just a small task. You need a rather large JAR to bind and have a license attached. Depending on the version, the Apache Commons implementation also lacks some functionality and has several implementation flaws.

Here’s a quick solution in one method, also available as a Gist on GitHub:

    /**
     * Unescapes a string that contains standard Java escape sequences.
     * <ul>
     * <li><strong>\b \f \n \r \t \" \'</strong> :
     * BS, FF, NL, CR, TAB, double and single quote.</li>
     * <li><strong>\X \XX \XXX</strong> : Octal character
     * specification (0 - 377, 0x00 - 0xFF).</li>
     * <li><strong>\uXXXX</strong> : Hexadecimal based Unicode character.</li>
     * </ul>
     * 
     * @param st
     *            A string optionally containing standard java escape sequences.
     * @return The translated string.
     */
    public String unescapeJavaString(String st) {

        StringBuilder sb = new StringBuilder(st.length());

        for (int i = 0; i < st.length(); i++) {
            char ch = st.charAt(i);
            if (ch == '\\') {
                char nextChar = (i == st.length() - 1) ? '\\' : st
                        .charAt(i + 1);
                // Octal escape?
                if (nextChar >= '0' && nextChar <= '7') {
                    String code = "" + nextChar;
                    i++;
                    if ((i < st.length() - 1) && st.charAt(i + 1) >= '0'
                            && st.charAt(i + 1) <= '7') {
                        code += st.charAt(i + 1);
                        i++;
                        if ((i < st.length() - 1) && st.charAt(i + 1) >= '0'
                                && st.charAt(i + 1) <= '7') {
                            code += st.charAt(i + 1);
                            i++;
                        }
                    }
                    sb.append((char) Integer.parseInt(code, 8));
                    continue;
                }
                switch (nextChar) {
                case '\\':
                    ch = '\\';
                    break;
                case 'b':
                    ch = '\b';
                    break;
                case 'f':
                    ch = '\f';
                    break;
                case 'n':
                    ch = '\n';
                    break;
                case 'r':
                    ch = '\r';
                    break;
                case 't':
                    ch = '\t';
                    break;
                case '\"':
                    ch = '\"';
                    break;
                case '\'':
                    ch = '\'';
                    break;
                // Hex Unicode: u????
                case 'u':
                    if (i >= st.length() - 5) {
                        ch = 'u';
                        break;
                    }
                    int code = Integer.parseInt(
                            "" + st.charAt(i + 2) + st.charAt(i + 3)
                                    + st.charAt(i + 4) + st.charAt(i + 5), 16);
                    sb.append(Character.toChars(code));
                    i += 5;
                    continue;
                }
                i++;
            }
            sb.append(ch);
        }
        return sb.toString();
    }

Does Java automatically switch daylight saving time?

Short answer: Yes, it does.

Long answer: The time in Java is just a simple long value (milliseconds since 1970) without any information about the time zone. The java.util.Date and java.sql.Date also store the date/time internally as milliseconds since 1970, but with an UTC time zone attached.

The time zone comes into play, when you format a date/time for output or when you parse a date/time from a string. The default time zone can be set through the -Duser.timezone system property during startup, or alternatively by calling the TimeZone.setDefault() method.

You can test it using a small test program. In most European countries, the switch to DST will happen on March 31st 2013 at 2am.
So here is a small program that loops 5 times from March 30th 11pm in one hour steps:

public static void main(String[] args) {
    Calendar c = Calendar.getInstance();
    c.set(2013, 2, 30, 23, 0, 0);
    long start = c.getTimeInMillis();
    long oneHour = 1000 * 60 * 60;
    long t = start;
    for (int i = 0; i < 5; i++) {
        System.out.println(new Date(t));
        t = t + oneHour;
    }
}

Starting it with -Duser.country=DE -Duser.timezone=GMT it will print the following:

Sat Mar 30 23:00:00 GMT 2013
Sun Mar 31 00:00:00 GMT 2013
Sun Mar 31 01:00:00 GMT 2013
Sun Mar 31 02:00:00 GMT 2013
Sun Mar 31 03:00:00 GMT 2013

Note that there is no switch.

Starting it with -Duser.country=DE -Duser.timezone=CET it will print the following:

Sat Mar 30 23:00:00 CET 2013
Sun Mar 31 00:00:00 CET 2013
Sun Mar 31 01:00:00 CET 2013
Sun Mar 31 03:00:00 CEST 2013
Sun Mar 31 04:00:00 CEST 2013

Note the Central European Time switch between 1am and 3am.

Starting it with -Duser.country=DE -Duser.timezone=EET it will print the following:

Sat Mar 30 23:00:00 EET 2013
Sun Mar 31 00:00:00 EET 2013
Sun Mar 31 01:00:00 EET 2013
Sun Mar 31 02:00:00 EET 2013
Sun Mar 31 04:00:00 EEST 2013

Note the Eastern European Time switch between 2am and 4am, because EET is CET plus one hour.

Java Expression Parser & Evaluator

There are quite a lot of free and commercial expression evaluators for Java out there.
From simple to complicated, you have a lot of choices.

But when I needed one the other day, it was quickly clear that none of the existing solutions fitted exactly my needs.

Download free Java Expression Parser & Evaluator at GitHub

First of all, it had to understand decimal and also negative values. You would be surprised to see how many of the existing solutions do not support that. The reason for this lack of support, by the way, is IMHO the lack of public available general tokenizing algorithms.
Solving expression like 1+(6-4)*5 is easy, when you have access to Google and/or Wikipedia, because it is a standard task in learning algorithms and data structures. When it comes to expressions like 1+(6 - -4.3)*-5.21, things get a bit more tricky and require some homemade logic in advance.

Second feature, the solution had to be simple and compact. Ideally it would be only a single class without the need of additional external libraries. Small footprint and based on standard 1.6 Java runtime.

Third and most important feature was the need for Java BigDecimal support. Nearly all of the existing simple solutions use Java’s double as a base data type for calculations. I had no need for high precision in trigonometric or logarithmic functions, but a strong need for exact precision in normal arithmetic. You think normal Java double based arithmetic is enough for everything that is below rocket science? You are utterly wrong. Take a look at this code snippet that adds 0.1 tens times and prints each result:

public static void main(String[] args) {
  double x = 0;
  for (int i = 0; i < 10; i ++) {
    x = x + 0.1;
    System.out.println("Value of x: " + x);
  }
}

It will print out the following:

Value of x: 0.1
Value of x: 0.2
Value of x: 0.30000000000000004
Value of x: 0.4
Value of x: 0.5
Value of x: 0.6
Value of x: 0.7
Value of x: 0.7999999999999999
Value of x: 0.8999999999999999
Value of x: 0.9999999999999999

This is not ‘wrong’, because of the limited possibilities to store fractional numbers in general.
Now, using Java double based arithmetic in a financial application should be considered harmful, from my point of view. At least if you are not extremely careful with rounding in every step. Therefore the need for BigDecimal support.

Fourth feature was the support for Boolean logic, ideally mixable with standard arithmetic. I wanted to be able to not only evaluate Boolean expressions like x < 4 && y > 5, but also expressions like 2 * x^2 > sqrt(y - 3.2) && max(x, 100) > 100.

And last but not least, of course support for functions and variables, as seen in the examples above.

So I sat down and implemented one, that had all what I wanted, plus some more features I caught on the way:

  • Uses BigDecimal for calculation and result
  • Single class implementation, very compact
  • No dependencies to external libraries
  • Precision and rounding mode can be set
  • Supports variables
  • Standard Boolean and mathematical operators
  • Standard basic mathematical and Boolean functions
  • Custom functions and operators can be added at runtime

The complete source can be found on GitHub, which is in fact only a single Java file, if you remove all the JUnit test cases and administrative files like license and readme.

The class com.udojava.evalex.Expression is all you need to have this handy expression parser and evaluator up and running.

To work with it, simply create an expression by passing the expression string:

Expression e = new Expression("1+1/3");

Then you can evaluate the expression by calling the eval() method on the expression:

BigDecimal result = e.eval();

Once you have an expression, you can re-evaluate it again and again, possibly with adjusting parameters like variables, precision, rounding mode and even modified functions and operators:

e.setPrecision(2);
e.setRoundingmode(RoundingMode.up);
BigDecimal result2 = e.eval();

A way of combining those actions, called chaining, is becoming more and more en vogue in Java and other programming languages. Java’s BigDecimal supports that pattern and my EvalExer also does:

BigDecimal result = new Expression("1+1/3").setPrecision(3).setRoundingMode(RoundingMode.UP).eval();

Variables can be set on an expression in different ways and then the expression can be re-evaluated without creating a new expression every time:

Expression e = new Expression("SQRT(a^2 + b^2)");
e.setVariable("a","3.42");
e.setVariable("b", new BigDecimal(9.6));
e.eval();
e.setVariable("a","9.51");
e.setVariable("b", new BigDecimal(8.4));
e.eval();

There are also some convenient methods for chaining expressions with variable setters:

BigDecimal result = new Expressions("SQRT(a^2 + b^2)").with("a",new BigDecimal (3.42)).and("b2,"9,6").setPrecision(2).eval();

And, of course, with BigDecimal chaining combined, you can have more chaining fun:

BigDecimal result = new Expressions("SQRT(a^2 + b^2)").with("a",new BigDecimal (3.42)).and("b","9,6").setPrecision(2).eval().add(14.2).divide(3);

Initially, the EvalEx parser/evaluator comes with a predefined set of operators and functions. (Plus a 100 digits exact predefined variable for PI, for free).

This set of operators, functions and variables can be extended at runtime without modifying the existing class.
Adding variables is easy, we have seen that before. Simply add them using the different chainable set()/with()/and() functions to an expression.

Custom operators can be added easily, simply create an instance of Expression.Operator and add it to the expression. Parameters are the operator name, its precedence and if it is left associative. The operators eval() method will be called with the BigDecimal values of the operands. All existing operators can also be overridden.
For example, add an operator x >> n, that moves the decimal point of x n digits to the right:

Expression e = new Expression("2.1234 &gt;&gt; 2");

e.addOperator(e.new Operator("&&", 30, true) {
  @Override
  public BigDecimal eval(BigDecimal v1, BigDecimal v2) {
    return v1.movePointRight(v2.toBigInteger().intValue());
  }
});

e.eval(); // returns 212.34

Adding custom functions is as easy as adding custom operators. Create an instance of Expression.Function and add it to the expression. Parameters are the function name and the count of required parameters. The functions eval() method will be called with a list of the BigDecimal parameters. All existing functions can also be overridden.
For example, add a function average(a,b,c), that will calculate the average value of a, b and c:

Expression e = new Expression("2 * average(12,4,8)");

e.addFunction(e.new Function("average", 3) {
  @Override
  public BigDecimal eval(List parameters) {
    BigDecimal sum = parameters.get(0).add(parameters.get(1)).add(parameters.get(2));
    return sum.divide(new BigDecimal(3));
  }
});

e.eval(); // returns 16

Supported Operators

Mathematical Operators
Operator Description
+ Additive operator
Subtraction operator
* Multiplication operator
/ Division operator
% Remainder operator (Modulo)
^ Power operator
Boolean Operators*
Operator Description
= Equals
== Equals
!= Not equals
<> Not equals
< Less than
<= Less than or equal to
> Greater than
>= Greater than or equal to
&& Boolean and
|| Boolean or

*Boolean operators result always in a BigDecimal value of 1 or 0 (zero). Any non-zero value is treated as a _true_ value. Boolean _not_ is implemented by a function.

Supported Functions

Function* Description
NOT(expression) Boolean negation, 1 (means true) if the expression is not zero
RANDOM() Produces a random number between 0 and 1
MIN(e1,e2) Returns the smaller of both expressions
MAX(e1,e2) Returns the bigger of both expressions
ABS(expression) Returns the absolute (non-negative) value of the expression
ROUND(expression,precision) Rounds a value to a certain number of digits, uses the current rounding mode
LOG(expression) Returns the natural logarithm (base e) of an expression
SQRT(expression) Returns the square root of an expression
SIN(expression) Returns the trigonometric sine of an angle (in degrees)
COS(expression) Returns the trigonometric cosine of an angle (in degrees)
TAN(expression) Returns the trigonometric tangens of an angle (in degrees)
SINH(expression) Returns the hyperbolic sine of a value
COSH(expression) Returns the hyperbolic cosine of a value
TANH(expression) Returns the hyperbolic tangens of a value
RAD(expression) Converts an angle measured in degrees to an approximately equivalent angle measured in radians
DEG(expression) Converts an angle measured in radians to an approximately equivalent angle measured in degrees

*Functions names are case insensitive.

Supported Constants

Constant Description
PI The value of PI, exact to 100 digits

Adding descriptions to JMX beans using an annotation wrapper

When working with standard JMX beans, a constant source of disappointment is the lack of descriptive information about the beans, attributes and operations.

Have a look again at how JConsole displays a bean, from my previous JMX introductory example:

JConsole view of a standard MBean

Now wouldn’t it be nice if it would look more like that:

JConsole view of a fully described MBean

Notice that attribute and operation names are now more meaningful than just some more or less cryptically Java identifiers. Also, the operation parameter to Shutdown now is correctly named, not the absolute meaningless p1 that makes any administrator scratch his head, if he doesn’t have your source code at hand.

And last but not least, there are real descriptions to all the elements in the info table!

What a pity that, though this is part of the JMX standard, it is not possible to achieve it using standard MBeans.

The only possibility is to create a so called Dynamic MBean. But this means implementing a rather complex and abstract interface and a lot of code to write for every bean.

Wouldn’t it be nice to define our example bean in a way like this?

 @JMXBean(description = "My first JMX bean test")
 public class MyBean {
    int level = 0;
    @JMXBeanAttribute(name = "Floor Level", description = "The current floor level")
    public int getLevel() {
        return level;
    }
    @JMXBeanAttribute
    public void setLevel(int newLevel) {
        level = newLevel;
    }
    @JMXBeanOperation(name = "Echo Test", description = "Echoes the parameter back to you")
    public String myMethod(
            @JMXBeanParameter(name = "Input", description = "String of what to echo") String param) {
        return "You said " + param;
    }
 }

Luckily, this is possible with a small wrapper utility that I created, named JMXWrapper.

With it, you can directly define a JMX bean through Java annotations, give descriptive names and add textual descriptions to all elements of a bean.

Notice that there is no interface definition needed anymore, any POJO (Plain Old Java Object) will do, as long as it follows the MXBean constraints, it can be turned into a dynamic JMX bean by simply adding some annotations.

You can map normal methods to attributes or operations. Read-only, write-only and read-write and boolean attributes are supported. Internal setter and getter method names do not have to follow the Java bean standard but can be any method (with one parameter for a setter and no parameter, but a return type for a getter).

And that’s not all; you can even externalize the names and descriptions and use the standard Java ResourceBundle features to localize the names and descriptions.

So it is possible to present the administration interface in the same language your application or server is running with.

Simply specify a standard Java ResourceBundle name in the JMXBean annotation and then specify name and description keys on the elements:

@JMXBean(resourceBundleName="com.example.my.package.BundleName",
    descriptionKey="myBeanDescription")
        public class MyBean {
           int level = 0;
           @JMXBeanAttribute(nameKey="level", descriptionKey="levelDescription")
           public int getLevel() {
               return level;
           }

The wrapper will automatically load locale specific translations from the resource bundle during bean creation time. If no translation can be found, the standard names and descriptions will be used.
And how do you use the wrapper? Simply drop an instance of your annotated normal Java class to the constructor of JMXBeanWrapper and register it with the JMX server:

MyBean bean = new MyBean();
JMXBeanWrapper wrappedBean = new JMXBeanWrapper(bean);
MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();
mbs.registerMBean(wrappedBean, new Objectname("com.example.my.package:type=TestBean,name=My Bean"));

That’s all!

I have placed the code of the wrapper on GitHub. Feel free to use it, all feedbacks are welcome. I published it under the MIT Open Source License.

Rotate image in memory by an arbitrary angle

Today I stumbled over a question which answer seemed to be easy: How to rotate an image in memory by an arbitrary angle? I had to find out that most examples out there are only for square images, rotations in 90 degree steps or are simply not working at all. It looks like most code snippets were copied from one to another without ever testing it!

This bothered me so much, that I sat down and worked out an example that really works.

It rotates an image by an arbitrary angle in degrees, with positive values to rotate clockwise and negative values to rotate counterclockwise.

The tricky part was to calculate the new image size, that fits exactly (well, as exactly as Java’s double can) the rotated image.

So, here it is:

	/**
	 * Rotates an image by an arbitrary amount of degrees around its center. A
	 * positive value for <code>degrees</code> will rotate the image clockwise,
	 * a negative value counterclockwise. An image will be returned with a size
	 * that exactly fits the rotated image.
	 * 
	 * @param src
	 *            The source image to rotate.
	 * @param degrees
	 *            The amount in degrees to rotate, a positive value rotates
	 *            clockwise, a negative counterclockwise.
	 * @return A new <code>BufferdImage</code> with a new size that exactly fits
	 *         the rotated image.
	 */
	public static BufferedImage rotateImage(BufferedImage src, double degrees) {
		double radians = Math.toRadians(degrees);

		int srcWidth = src.getWidth();
		int srcHeight = src.getHeight();

		double sin = Math.abs(Math.sin(radians));
		double cos = Math.abs(Math.cos(radians));
		int newWidth = (int) Math.floor(srcWidth * cos + srcHeight * sin);
		int newHeight = (int) Math.floor(srcHeight * cos + srcWidth * sin);

		BufferedImage result = new BufferedImage(newWidth, newHeight,
				src.getType());
		Graphics2D g = result.createGraphics();
		g.translate((newWidth - srcWidth) / 2, (newHeight - srcHeight) / 2);
		g.rotate(radians, srcWidth / 2, srcHeight / 2);
		g.drawRenderedImage(src, null);

		return result;
	}

The Unparseable Date Pitfall

Here is another pitfall with the SimpleDateFormat class.
Unlike my previous blog entry, this pitfall is not a multithreading issue; the following code is thread-safe. However, it can fail in certain situations.

package com.udojava.blog;

import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;

public class ParseDateDemo {

	public static void main(String[] args) throws ParseException {
		String dateString = "Thu Oct 25 6:12 pm PDT 2012";

		SimpleDateFormat dateFormat = new SimpleDateFormat(
				"EEE MMM d h:mm a z yyyy");

		Date theDate = dateFormat.parse(dateString);

		System.out.println(dateFormat.format(theDate));
	}

}

The code will run on a lot of systems without ever failing, but on certain system setups, it will fail.

For example, on my computer it won’t run and instead produce the following exception:

Exception in thread "main" java.text.ParseException: Unparseable date: "Thu Oct 25 6:12 pm PST 2012"
at java.text.DateFormat.parse(DateFormat.java:357)
at com.udojava.blog.ParseDateDemo.main(ParseDateDemo.java:15)

This is simply because my system is set up with a German locale setting, which does not understand the English names for the day (Thu) and month (Oct). The code will fail on all systems, which do not run under an English locale setup.

You can test it, by giving the Java arguments –Duser.country=DE –Duser.language=de when starting the program. This will set the Java default locale to German/Germany.

Now how to fix it? Starting the program with –Duser.language=en and forcing the users to use your locale is certainly not a good way to go.

Luckily, we can take care that the SimpleDateFormat class works with the correct locale, simply by adding a locale parameter to the constructor:

		SimpleDateFormat dateFormat = new SimpleDateFormat(
				"EEE MMM d h:mm a z yyyy", Locale.ENGLISH);

Now the program will run correctly, no matter what default locale Java is using.

The “write once, run anywhere” slogan from Java needs most of the time more care by the programmer than just using the Java programming language.

Java SimpleDateFormat gives wrong results

Would you expect the following lines of code to ever print something to the standard output?

public static SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd");
...
Date date = dateFormat.parse("2012-10-26");
String reformat = dateFormat.format(date);
if (!"2012-10-26".equals(reformat)) {
  System.out.println("Wrong date: " + reformat);
  }

Indeed, there are situations when such code will occasionally fail.
The SimpleDateFormat class is not thread safe, that means whenever two or more separate threads access the same class instance, the results from the parse() and format() methods can produce wrong results.
To reproduce this effect, I show the full test code:

package com.udojava.blog;

import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.Date;

public class DateFormatDemo {

	public static SimpleDateFormat dateFormat = new SimpleDateFormat(
			"yyyy-MM-dd");

	public class DemoThread extends Thread {

		@Override
		public void run() {
			for (int i = 0; i < 100; i++) {
				try {
					Date date = dateFormat.parse("2012-10-26");
					String reformat = dateFormat.format(date);
					if (!"2012-10-26".equals(reformat)) {
						System.out.println("Wrong date: " + reformat);
					}
				} catch (ParseException e) {
					e.printStackTrace();
				}
			}
		}

	}

	public void testRun() {
		DemoThread t1 = new DemoThread();
		DemoThread t2 = new DemoThread();
		t1.start();
		t2.start();
	}

	public static void main(String[] args) {
		new DateFormatDemo().testRun();
	}

}

If you run this, you will get differing outputs from one run to the next run, something like this:

Wrong date: 2201-10-26
Wrong date: 2201-10-26
Wrong date: 2012-12-26
Exception in thread "Thread-1" java.lang.NumberFormatException: empty String
at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1011)
at java.lang.Double.parseDouble(Double.java:540)
at java.text.DigitList.getDouble(DigitList.java:168)
at java.text.DecimalFormat.parse(DecimalFormat.java:1321)
at java.text.SimpleDateFormat.subParse(SimpleDateFormat.java:2088)
at java.text.SimpleDateFormat.parse(SimpleDateFormat.java:1455)
at java.text.DateFormat.parse(DateFormat.java:355)
at com.udojava.blog.DateFormatDemo$DemoThread.run(DateFormatDemo.java:18)

So, sometimes it can be dangerous to try to over-optimize by using static members that are shared between threads. Even simple things like a SimpleDateFormat can fail in multithreading.
And they usually then fail only in 1 out of 1000 times, making it a real horror to find, debug and fix those issues.

%d bloggers like this: