Unidev – Software Development

Archive for September, 2007

We have a project which needs to show foreign language, like Chinese, on a web page. And we are using RICO 2.0, PHP, MySQL & ASP.NET in the project. To achieve this, we decide to use UTF-8 character set and the following changes need to be done:

• MySQL:

To store UTF-8 in a table field, you need to set column’s charset => utf8 and collate => utf8_unicode_ci

• Web Page:

To show foreign language in a web page, the page file has to be saved as encoding UTF-8

• PHP:

The default encoding for PHP 5 is UTF-8, so it supports foreign language. But if you use PHP to retrieve UTF-8 characters from MySQL, you must add mysql_query(‘SET NAMES utf8′); after connecting to MySQL database for setting up the connection encoding.

• RICO:

As for RICO Live gird, Ajax response must start with <?xml version=”1.0″ encoding=”utf-8″?>

• ASP.NET:

First make sure you have following line in you web.config
<globalization requestEncoding=”utf-8″ responseEncoding=”utf-8″ fileEncoding=”utf-8″/>
Second, keep both .aspx and code behind file save as the same encoding UTF-8
If you need to access the UTF-8 characters in MySQL database, be sure to use MySQL Connector/Net 5.0 or up with undocumented setting: charset=utf8 in connection string. We tried ODBC driver for MySQL, it doesn’t work properly with UTF-8.

We recently pushed a couple of new Java programs to our client’s application server (HPUX) and soon thereafter they started to receive alerts about the swap running out of space. Of course, being the “new kid on the block” our programs were suspect. While these programs were not the only ones taking up a good deal of swap space, we were a bit perplexed why they were taking much at all.

These programs were fairly small in terms of functionality, but the amount of swap space it was using was around 600-700MB. Here is a summary of our findings (note that some of these details might be different on a Windows system due to differences in memory management):

• A common theme is that most java processes have a big difference between the residual memory and the total virtual memory.
• Java will allocate at a minimum about 200 MB of swap even for the simplest of programs. We tested using a simple program on both a HPUX and a linux server. The best and only documentation I could find about this was at the URL below. The bottom line is that this is probably the behavior of the JVM and out of our control so we just have to deal with it.

  http://groups.google.com/group/comp.lang.java.programmer/browse_thread/thread/778089cbe0c9814c/5b0dfcea715d9055?lnk=st&q=java+uses+virtual+memory&rnum=20&hl=en#5b0dfcea715d9055It

  It is reserving space for example for threads that you may or may not create. It is awkward to rearrange memory on the fly, so the JVM parcels out the virtual ram for various tables when it first loads.

• The -Xms and -Xmx parameters do affect the java heap but it does not control the total memory footprint. It is only a piece of the total size.
• -Xmx alone can affect the total virtual memory size. Currently the processes have -Xms24m and -Xmx512m. Immediately after starting these programs, we saw the swap allocation jump to 600-700MB. When we changed -Xmx to 24m, it dropped to a little over 200MB.
• In relation to the above, the normal behavior of the garbage collector is to garbage collect only when it needs to free up memory. Having a too high of a -Xmx value could mean that it will never garbage collect, which means it won’t reuse memory. We see this in the Weblogic environment too but in that environment all of the memory usually gets used at some point.
• There is a new tool built into Java 5 and up called jmap that will give you heap information about a java process. This tool would make tuning the heap parms really simple. Unfortunately the process also has to be running using Java 5 so we cannot use it at this time. There is a version for 1.4 but it only works on Solaris.