All About Unidev

Archive for 2007

Keeping a medium sized contract programming company up to speed on the latest technologies is a daily challenge. With a wide variety of technologies and skillsets in use at any given time and with the future technology needs of the company being very customer driven, it can be very difficult to put together individualized training plans. Our solution was to develop a more ‘university’ environment where we constantly have optional classes on a wide variety of new and current technologies. In this way, employees are able to choose which directions they feel works best for them and they are able to investigate a variety of choices without having to commit to a particular path. In our experience, this has worked quite well with a number of surprises about which directions people choose. With an array of topics such as J2EE, C# .NET, Web Marketing methods, HTML, Project Management, RAD and Network Architecture, employees can explore many possibilities. Unidev has a number of types of work including software developers who work on customer projects at the development center, technology consultants who work both at the customer site and in-house, Web marketing consultants and IT staffing personnel who augment customer development teams. This variety of work along with the various technologies employed makes for a very interesting learning environment.

We have a project which needs to show foreign language, like Chinese, on a web page. And we are using RICO 2.0, PHP, MySQL & ASP.NET in the project. To achieve this, we decide to use UTF-8 character set and the following changes need to be done:

• MySQL:

To store UTF-8 in a table field, you need to set column’s charset => utf8 and collate => utf8_unicode_ci

• Web Page:

To show foreign language in a web page, the page file has to be saved as encoding UTF-8

• PHP:

The default encoding for PHP 5 is UTF-8, so it supports foreign language. But if you use PHP to retrieve UTF-8 characters from MySQL, you must add mysql_query(‘SET NAMES utf8′); after connecting to MySQL database for setting up the connection encoding.

• RICO:

As for RICO Live gird, Ajax response must start with <?xml version=”1.0″ encoding=”utf-8″?>

• ASP.NET:

First make sure you have following line in you web.config
<globalization requestEncoding=”utf-8″ responseEncoding=”utf-8″ fileEncoding=”utf-8″/>
Second, keep both .aspx and code behind file save as the same encoding UTF-8
If you need to access the UTF-8 characters in MySQL database, be sure to use MySQL Connector/Net 5.0 or up with undocumented setting: charset=utf8 in connection string. We tried ODBC driver for MySQL, it doesn’t work properly with UTF-8.

We recently pushed a couple of new Java programs to our client’s application server (HPUX) and soon thereafter they started to receive alerts about the swap running out of space. Of course, being the “new kid on the block” our programs were suspect. While these programs were not the only ones taking up a good deal of swap space, we were a bit perplexed why they were taking much at all.

These programs were fairly small in terms of functionality, but the amount of swap space it was using was around 600-700MB. Here is a summary of our findings (note that some of these details might be different on a Windows system due to differences in memory management):

• A common theme is that most java processes have a big difference between the residual memory and the total virtual memory.
• Java will allocate at a minimum about 200 MB of swap even for the simplest of programs. We tested using a simple program on both a HPUX and a linux server. The best and only documentation I could find about this was at the URL below. The bottom line is that this is probably the behavior of the JVM and out of our control so we just have to deal with it.

  http://groups.google.com/group/comp.lang.java.programmer/browse_thread/thread/778089cbe0c9814c/5b0dfcea715d9055?lnk=st&q=java+uses+virtual+memory&rnum=20&hl=en#5b0dfcea715d9055It

  It is reserving space for example for threads that you may or may not create. It is awkward to rearrange memory on the fly, so the JVM parcels out the virtual ram for various tables when it first loads.

• The -Xms and -Xmx parameters do affect the java heap but it does not control the total memory footprint. It is only a piece of the total size.
• -Xmx alone can affect the total virtual memory size. Currently the processes have -Xms24m and -Xmx512m. Immediately after starting these programs, we saw the swap allocation jump to 600-700MB. When we changed -Xmx to 24m, it dropped to a little over 200MB.
• In relation to the above, the normal behavior of the garbage collector is to garbage collect only when it needs to free up memory. Having a too high of a -Xmx value could mean that it will never garbage collect, which means it won’t reuse memory. We see this in the Weblogic environment too but in that environment all of the memory usually gets used at some point.
• There is a new tool built into Java 5 and up called jmap that will give you heap information about a java process. This tool would make tuning the heap parms really simple. Unfortunately the process also has to be running using Java 5 so we cannot use it at this time. There is a version for 1.4 but it only works on Solaris.

One of our applications was determined to be leaking memory. The leak was traced back to some stateful EJB’s that were extending Spring’s AbstractStatefulSession bean class. While I could not trace to the exact source of the leak, it seems to be related to the unloading of the BeanFactory.

As per the documentation, Spring’s default behavior will initialize a new ApplicationContext when the EJB object is created. It also states that the BeanFactory (which is the ApplicationContext) is unloaded when the EJB is removed.

One of the first things I checked was ensuring the EJB’s were in fact being removed. They were. At some point I remembered that in a more recent project, we changed the beanFactoryLocator to the ContextSingletonBeanFactoryLocator so that all EJBs use the same context (it also happens we setup the web application to use the same context too). The newer application did not have the memory leak so I decided to implement it here. I also figured that even if it didn’t fix the leak, this method is more efficient since there is no reason to have the context loaded everytime. To implement, I just overloaded the setSessionContext method:

Once implemented the leak went away. I never took the time to dig in and see what the true offender was, but it was found that some of the hibernate SingleTableEntityPersister objects were still present after leak. This could mean that it is Spring not properly cleaning things up or Hibernate itself has the leak.

As info this method was already implemented with our stateless EJB’s, so I could not confirm if the issue existed for AbstractStatelessSessionBean.  However since both abstract classes use the unloadBeanFactory method from AbstractSessionBean, I would figure the issue would present itself — although due to Weblogic pooling the stateless EJB’s, it is probably not as noticable.

Specifics:

• Weblogic 8.1 sp4
• JRockit 1.4.2_13
• Spring 1.2.9
• Hibernate 3.2.5

We’ve had the opportunity to work with a lot of new technologies lately. Sometimes with friendly customer sites ;> but mostly within our own sites. Unidev strongly encourages and supports everyone within the company experimenting with new technologies as they come around. We do this by teaching internal classes, providing test web sites, providing a variety of real live high volume web sites for production testing as well as having a test lab with different operating systems, languages and browsers, Linux and Windows servers and pretty much any commercial software you could imagine.

The results of this policy are manyfold. The primary benefit of this goes to our customers as it ensures that we are using the very best available methods and tools for developing their software and websites. Recently a large customer with SQL Server and DB2 databases needed several dynamic data driven web pages quickly. We were able to deploy an Ajax tool we had created as an experiment for one of our internal web sites that worked very well and we were able to get the site developed very quickly.

Today we are continuing to experiment in the Windows .Net, Java/J2EE and Linux PHP areas. We continue working with new Ajax tools as well as Ruby on Rails, Microsoft Silverlight and databases, BLOGS, social networking, Hibernate, XML data messaging, RSS feeds and RSS feed readers (used to display news on the unidev home page).

Good Stuff

Interactive maps has become an inevitable part of modern web applications and introduction of Google Map API has made it even easier to integrate an interactive map to your website. Areas on which interactive maps have proved itself are tourism related websites and service oriented websites. In this blog we will see how we can add an interactive map to your website using Google Map API and AJAX to process the response from the server.

The first step in integrating a map in your website is to request  a Google map API key. You can request  a Google Maps API key here. If you have URL for your website, you need to specify the URL while requesting key. If you are playing around with the API, you can very well request for http://localhost. If you are using a port number , you need to specify the port number also. For example http://localhost:8080. Once you get the key you can use the key for all subfolders of http://localhost.

This is Source of an HTML page that displays map of 1415 Elbridge Payne Rd,Chesterfield,MO 63017. You need to know the latitude and longitude of the address to display map in your web page. If you don’t know the latitude and longitude , there are many online services that provide services to get latitude and longitude of an address . Another way of implementing this is to use geoCoder provided by Google Map API to get the latitude and longitude by providing an address. We will come to this in a while.

<!DOCTYPE html PUBLIC “-//W3C//DTD XHTML 1.0 Strict//EN”
  “http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd“>
<html xmlns=”http://www.w3.org/1999/xhtml“>
  <head>
    <meta http-equiv=”content-type” content=”text/html;
                                     charset=utf-8″/>
    <title>Google Maps JavaScript API Example</title>
    <script  src=”http://maps.google.com/maps?file=api&v=2&key=
         ABQIAAAAR7yXyXxWnmYNsvbV6JWv5hT2yXp_ZAY8_ufC3CFXhHIE1NvwkxTvGi-   GhomVbgkbC6tv2C8JocyNEg”
      type=”text/javascript”></script>
    <script type=”text/javascript”>
    //<![CDATA[
    function load() {
      if (GBrowserIsCompatible()) {
     var map = new GMap2(document.getElementById("map"));
     map.setCenter(new GLatLng(38.65, -90.56), 8);
     map.addControl(new GSmallMapControl());
     map.addControl(new GOverviewMapControl());
     var point = new GLatLng(38.65, -90.56);
     marker = new GMarker(point);
     map.addOverlay(marker)
     GEvent.addListener(marker, "click", function() {
     marker.openInfoWindowHtml("<p>Unidev: Custom Software and Applicaton Development</p>”});
  }

    }
    //]]>
    </script>
  </head>
  <body onload=”load()”>
    <div id=”map” style=”width: 800px; height: 600px”></div>
  </body>
</html>

Class GMap2 represents a map instance and you can create as many instances as you want. addcontrol() method can be used to add controls to your map.GSmallMapControl() enables Zoom functionality. Event handling is implemented using GEvent.addListener() method. GMarker class marks a point of interest in your map.OpenInfoWindowHtml() method opens an informational window on ‘OnClick’ event. Place holder for map in this example is a <div> tag with id ‘map’.
Adding an Interactive Map to your web application:
This example shows how to add a user driven map in your web application. First step in creating an interactive map is to have a form in your web page where users can enter the address and description of their point of interest.
Given below is the bean and DAO implementation class.
public class GMarker
{
    private Long id;
    private String address;
    private String description;
// Need to have setters , getters and a costructor that takes all the varaibales)
}

We have another Location object that stores data entered by users. We read data from the Location table(which is populated by the data submitted by users) and create Gmarker object. This DAO implementation is not Table  driven , we read hardcoded data from a Map. In realworld DAO Implmentation would be Table driven.

DAO implementation:
public class PickDAOImpl implements PickDAO
{
private static final Map myPicks = new HashMap();
    static {
    myPicks.put(“1″,new GMarker(new Long(1),”1415 Elbridge Payne  Rd,Chesterfield,MO,63017″,”Unidev: Custom Software and Application Development”));
    myPicks.put(“2″,new GMarker(new Long(2),”1400 Douglas Street,Omaha,NE”,”Union Pacific Railroad”));
    }
 public List findAllAsMarkers() {
        List results = new ArrayList();
        Collection allLocations = myPicks.values();
        Iterator locIt = allLocations.iterator();
        GMarker location = null;
        while(locIt.hasNext()){
            location = (GMarker)locIt.next();
            results.add(new GMarker(location.getId(),
                    location.getAddress(),
                    location.getDescription()));
        }
   return results;
    }
}

Heart of Ajax is XMLHttpRequest object and however in this example we are using Google’s version of XMLHttpRequest object, that is GXmlHttp class. Ajax makes the request to the servlet, which would be processed by the servlet  and Servlet returned XML Document object can be processed by Javascript DOM functions. We are using Xsteam implementation provided by http://xstream.codehaus.org/ to generate XML document from Java object. Other open source libraries like Apache and Castor can also be used to generate XML document from Java objects.

XML document generated by the servlet for the request to get all user submitted addresses in the database would look like this.
<list>
  <marker>
    <id>2</id>
    <address>1400 Douglas Street,Omaha,NE</address>
    <description>Union Pacific Railroad</description>
  </marker>
  <marker>
    <id>1</id>
    <address>1415 Elbridge Payne Rd,Chesterfield,MO,63017</address>
    <description>Unidev: Custom Software and Application Development</description>
  </marker>
</list>

Given below is the Servlet that generated the above XML document.

public class MyPickServlet extends javax.servlet.http.HttpServlet implements javax.servlet.Servlet
{
    private PickDAO pickDAO;
    public PickDAO getPickDAO()
    {
        if (pickDAO == null) {pickDAO = new PickDAOImpl();
        }
       return pickDAO;
    }

    public void setPickDAO(PickDAO pickDAO)
    {
        this.pickDAO = pickDAO;
    }
    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws  ServletException, IOException {
        String method=request.getParameter(“method”);
        XStream xstream = new XStream();
        xstream.alias(“marker”, GMarker.class);
        response.setContentType(“application/xml”);
 if (method.equals(“findAll”)) {
            List locations = getPickDAO().findAllAsMarkers();
            String xml = xstream.toXML(locations);
            response.getWriter().write(xml);
        }
}

The response generated by the Servlet is processed by javascript DOM functions. Given below is the HML page, that makes AJAX request to Server and process the response from the Server. The Javascript code has comments added to explain the important points.

<html>
 <head>
 <meta http-equiv=”content-type” content=”text/html; charset=utf-8″ />
  <title>Google Maps JavaScript API Example</title>
  <script
       src=”http://maps.google.com/maps?file=api&v=2&key=
                     ABQIAAAAR7yXyXxWnmYNsvbV6JWv5hT2yXp_ZAY8_ufC3CFXhHIE1NvwkxTvGi-GhomVbgkbC6tv2C8JocyNEg”
       type=”text/javascript”></script>
  <script type=”text/javascript”>
     var map = null;
         var gmarkers = [];
         var i =0;
         var address = null;
         var geocoder = null;
        var id = null;
      // method called on onLoad event
    function loadUserPicks() {
   map = new GMap2(document.getElementById(“map”));
         var request = GXmlHttp.create();
         request.open(“GET”, “myPick?method=findAll”, true);
         request.onreadystatechange = getCallbackFunction(request, processGmapData);
         request.send(null);
     }
    function getCallbackFunction(req, processData) {
          // Return an function that listens to the
          // XMLHttpRequest instance
              return function () {
         // If the request status is “complete”
           if (req.readyState == 4) {
             if (req.status == 200) {
                  // Got a response
                  // Pass the XML payload of the response to the
                  // handler function
          processData(req.responseXML);
 } else {
          // An HTTP problem has occurred
          alert(“HTTP error: “+req.status);
        }
      }
    }
  }
  var locMarkers = null;

function processGmapData(xmlDoc){
             // obtain the array of markers and loop through it
             locMarkers = xmlDoc.documentElement.getElementsByTagName(“marker”);
             displayPicks();
     }

    function displayPicks() {
      map.clearOverlays();
            for (var i = 0; i < locMarkers.length; i++) {
              // obtain the attribues of each marker
              var label = locMarkers[i].getElementsByTagName(“address”)[0].firstChild.nodeValue;
              address = label;
              map = new GMap2(document.getElementById(“map”));

            // Create new geocoding client object
          geocoder = new GClientGeocoder();

           // Retrieve location information for the address, pass it to addToMap()
           geocoder.getLocations(address, addToMap); }
     }
    function addToMap(response)
       {
            // Retrieve the object
           var place = response.Placemark[0];
           // Retrieve the latitude and longitude
           var point = new GLatLng(place.Point.coordinates[1],
             place.Point.coordinates[0]);
          // Center the map on this point
          map.setCenter(new GLatLng(place.Point.coordinates[1],place.Point.coordinates[0]),4);
          map.addControl(new GSmallMapControl());
          map.addControl(new GOverviewMapControl());
         var marker = new GMarker(point);
        GEvent.addListener(marker,”mouseover”, function() {
              marker.openInfoWindowHtml(place.address + ‘<br>’+'<b>Country:</b>’
              +place.AddressDetails.Country.CountryNameCode);
          });

           gmarkers[i] = marker;
           i++;
           map.addOverlay(marker);
  }
 </script>
  <form name=”mypickform” action=”/myPick” method=”get”>
  <body onload=”loadUserPicks()”>
  <div id=”map” style=”width: 800px; height: 600px”></div>
  </body>
  </form>
 </html>

Enjoy……

As we know, Silverlight is a portable client side technology and requires no runtime on the server. So moving a Silverlight app to a new server requires no server side configuration. But one point is important. If you don’t have .NET 3.0 framework installed on your new server, you may have to add a new MIME type for the website which hosting Silverlight. Otherwise the Silverlight will just refuse to work and show no clue what’s wrong.

Here are the steps on an IIS server:

Start -> Run -> type inetmgr to open iis manager.

Then rigth click on your site and click properties. After that choose tab Http Headers and there you will see button MIME Types … click it and add xaml in textbox.

This blog  shows how to save CLOB to a Relational Database using Java, Spring and Hibernate.This example is implemented using two classes from Spring Framework

1)OracleLobHandler

2)NativeJdbcExtractor (This example has been tested in Weblogic J2ee Container and this example uses WebLogicNativeJdbcExtractor.)

Spring Framework applicationContext.xml configuration:

<bean id=”mySessionFactory class=”org.springframework.orm.hibernate3.LocalSessionFactoryBean”>

<property name=”dataSource” ref=”myDataSource” />

<property name=”lobHandler” ref=”lobHandler” />

<property name=”mappingResources”>

<list> <value>com/clob/hibernate/lohHandler.hbm.xml</value>

</list></property> <property name=”hibernateProperties”><props><prop key=”hibernate.dialect”>org.hibernate.dialect.OracleDialect </prop>

<prop key=”hibernate.show_sql”>true</prop>

<prop key=”hibernate.connection.autocommit”>false</prop>

<prop key=”hibernate.connection.release_mode”>auto </prop>

<prop key=”hibernate.jdbc.use_streams_for_binary”>true </prop>

<prop key=”SetBigStringTryClob”>true</prop>

</props>

</property>

</bean>

<bean id=”lobHandler” class=”org.springframework.jdbc.support.lob.OracleLobHandler”

lazy-init=”true”>

<property name=”nativeJdbcExtractor”>

<beanclass=”org.springframework.jdbc.support.nativejdbc.WebLogicNativeJdbcExtractor” />

</property>

</bean>

Code block from lobHandler.hbm.xml:

<property name=”extdXml” type=”java.sql.Clob”>

<column name=”DTL_XML” /> </property>

Java bean class:

public class lohHandler implements java.io.Serializable {

 private Clob extdXml;

public void setExtdXml(Clob extdXml) { this.extdXml = extdXml;} public Clob getExtdXml() { return this.extdXml;}

public void setClobContent( String sourceStream )throws IOException{

setExtdXml( Hibernate.createClob( sourceStream ) );

}} Spring Framework Controller class:Upload the xml file and store contents of xml file in a String.Instantiate the bean.

lohHandler lHandler = new lohHandler ();

lHandler.setClobContent(xmlString);

setClobContent() method inturn calls Hibernate.createClob() method, which creates a CLOB out of a String. Now we are ready to save the lHandler bean object in a Hibernate Session.

This series of posts on Unidev is for potential customers or employees to learn a little bit more about this company, it’s history and it’s people.

Unidev has three major service lines, custom software development, consulting and staffing. Over the years the relative size of each area as varied as the market has evolved. In the early 1990′s staffing was the main component and that area grew consistently throughout the 90′s peaking with the dotcom era and Y2k. After those bubbles burst, the staffing area became much more challenging and difficult, at least for us.

With the advent and rapid growth of the Internet, the software development area, especially web development, became a much stronger growth area. The initial web sites for companies tended to be pretty simple affairs, primarily just brouchures. At the time, we didn’t have a lot of creative types so we didn’t get too much into this. As companies learned more of the capabilities of the Internet, they started developing their second generation sites which required much more programming and often real time interfaces with their internal systems.

 These new requirements fit much better into our background as application developers and this became a strong growth area for us.

More to come….

Unidev proudly announces they are partnering with Inquiry Intelligence Systems (IQ Systems). IQ Systems has provided valuable marketing services to their clients for over 25 years. Their services include sales lead distribution and product literature fulfillment. To learn more about IQ Systems go to www.iqsalespro.com

Currently Unidev is assisting IQ Systems transition their internal and external systems to the Microsoft .NET platform. Microsoft describes .NET as “..the next generation of software that connects our world of information, devices and people in a unified, personalized way.”

Unidev is a company that provides software development and consulting services including application and web software design and development, IT project management services, technology and systems consulting to business and government. If you have any questions regarding your IT needs, please direct your questions to sales@unidev.com.