Of Intelligence and Rationality

Most people will think I'm suffering from a serious bout of 'sour grapes' if I say that the legendary investor, Warren Buffet, isn't that smart after all. Most likely, you'll sneer in disgust and retort, 'if he isn't smart, how did he become one of the richest men in America?'

Before I answer that, let me quote what Buffet thinks about intelligence himself:

"My partner Charlie Munger and Tony Nicely at Geico are always rational. 160 IQs can say stupid things that sound good. People do silly things, whether they have 120 IQ or 160. You can always improve your rational thought. Rationality is the only thing that helps you. One thing that could help would be to write down the reason you are buying a stock before your purchase. Write down 'I am buying Microsoft for $300 billion because you'd force yourself to write this down. It clarifies your mind and discipline. This exercise makes you more rational"

So he's successful only because he is more uber-rational than you or I am, and that has nothing in relation to intelligence at all. So doesn't intelligence mean anything?

Sure it does, just that it doesn't mean you have to be smart in order to be wealthy as Jay Zagorsky from Ohio state University in Columbus found out. From the study, it affirms the fact that while there are research indicating that intelligence is correlated with earning power, they generally end up spending more of their earnings as compared with people of average intelligence.

Contrary to common expectation, intelligence does not always predict financial wellbeing. Even though smart people earn more, on average, it does not protect them from financial difficulty.

New research has found that people who score higher on intelligence tests end up with the same net worth as others when lifestyle factors are taken into account. And the study confirms that you don't have to be smart to be wealthy.

The work reveals that while exceptionally smart individuals typically earn more, they are also more likely to spend to their credit card limit, compared with people of average intelligence.

Jay Zagorsky at Ohio State University in Columbus, US, analysed personal financial information collected from 7500 people between the ages of 33 to 41. Subjects provided details about their cash flow including wages, welfare payments, alimony, and stock dividends and their overall net worth. They also answered questions about whether they had "maxed out" any of their credit cards, missed bill payments or filed for bankruptcy.

All the participants had taken an intelligence exam known as the Armed Forces Qualification Test (AFQT) in 1980, an intelligence test used by the US military to assess recruits. The AFQT can be translated into IQ scores.

Squirreled wealth

On the surface, Zagorsky's analysis confirms the findings of previous studies linking higher intelligence with higher income. "Each point increase in IQ test scores is associated with $202 to $616 more income per year," he says.

For example, a person with a score of 130 (in the top 2%, in terms of IQ) might earn about $12,000 more per year than someone with an average IQ score of about 100.

On the surface, people with higher intelligence scores also had greater wealth. The median net worth for people with an IQ of 120 was almost $128,000 compared with $58,000 for those with an IQ of 100.

But when Zagorsky controlled for other factors such as divorce, years spent in school, type of work and inheritance found no link between IQ and net worth. In fact, people with a slightly above-average IQ of 105 , had an average net worth higher than those who were just a bit smarter, with a score of 110.

Cultural trump

People who had divorced once had about $9600 less wealth on average than their never-divorced counterparts. And those who smoked heavily had an $11,000 reduction in net worth. These external factors rather than IQ could explain the differences in wealth, Zagorsky suggests.

"IQ is clearly overwhelmed or trumped by the cultural imperative to consume," says economist Richard Wolff at the University of Massachusetts-Amherst, US. "People with higher IQs are acutely aware of all the goods and services that they can consume," he says.

Wolff believes that smart people often have high expectations for what they deserve. "It's a notion of 'That's what I'm entitled to as an American & that's what I get for working hard.'" He notes that wages of American workers increased steadily, in real terms, from the 1820s to the 1970s, and people in the US expect their standard of living to constantly improve. However, the buying power of US wages has recently declined.

Reference: Intelligence (DOI: 10.1016/j.intell.2007.02.003)

It seems like the best way to get rich is probably to stop smoking, drinking and partying instead. It also means we shouldn't get too jealous just because someone else appears to be better off than you just because they can afford more 'visible', conspicuous spending. I don't think happiness and satiety has much to do with how much you can write out of your cheque-book, don't you agree?

---

Additional Links
[1] Smarter People are no Better Off: http://www.newscientist.com/article/dn11711-smarter-people-are-no-better-off-.html

Two Americans do not make one good English

"We are two great nations, separated by a common language"

- Winston Churchill, on America and Britain

I love the Americans, they're great folks, and have an absolutely great sense of humour. It is just unfortunate that they do not have a good command of English though. I can hear the jeers at me now, well if you're unconvinced, allow this 'non-native speaker of English' to teach a thing or two on how bad American English is, and take a rib at them with a few examples:

1) They refuse to understand "mobile phone" as "cell phone"
There was this time when I was at a hotel in San Mateo talking to the concierge, when we found out that one of the members in our group went missing. So I told the lady at the counter to hold for a second, while I call my colleague on my mobile phone, and she went 'Excuse me?!', in her loud, American fashion.

An English colleague of mine kindly stepped in and reiterated what I've said, replacing it with a 'cell phone', and turned back to me and said semi-jokingly, 'mobiles only mean those things that they hang from the ceilings here'.

From a Singaporean perspective, we could be worse, since we proudly call them 'handphones', which is literal transmogrification of the word from Chinese to English! In this case, we're one up the Americans already!

2) They do not understand what a "fortnight" is
This one's a real gem, because it came from a well educated and rather senior banker. To cut the story short, when someone in my office used the word 'fortnight', the guy had to do a double-take, and only after a while later did he overcome his embarrassment and asked what that meant, not knowing that it's just another way of saying 'half a month'!

3) Takeaways are "to go's"
This happened to me on my first trip to MacDonalds. The girl at the counter asked, 'having it here, or to go?'

Innocently I replied, 'to go where?'

4) They call "full stop" a "period"
The Americans are embarrassingly wrong in this case! A "period" is one of those sexist thing that only a woman can have. It comes once a month, and they get pretty twitchy whenever it happens. But remind me not to be this explicit when explaining to my bosses' 9-year old, who pointed it out to me!

5) Bills and checks
They don't call a bill "a bill". Instead they call it "a check". Well usually if I ask anybody for "a check", I'd expect to bring it to the bank and exchange it for cash. Who says Americans aren't generous, when they are quite happy to give you a check instead of making you pay for a meal?

6) They don't "take a turn", but instead they "make one"
This is one of the weirdest usage of English I've ever encountered. Whenever Americans give directions, they won't tell you to take a left turn down the block, but instead you'll have to demolish a few buildings and pave your own pathways by 'making a left'. I'm still holding off on my temptation to reply, "but I haven't got the tools!"

7) They don't apologise when they bump into you
I'm not saying that Americans are offensive or rude. They are just more accustomed to using "excuse me", when they want you to move aside, rather than "sorry" when there isn't a need to. The Aussies and the Brits are "more polite" in this respect, and more prone to using "sorry" than "excuse me".

8) What's a biscuit? And a boot?
This is one of the instances that I'm more in agreement with my American friends. I'm quite fine with swapping the usage of a biscuit and a cookie, they don't mean anything materially different to me, but in the case of a car, the trunk (as in a suitcase) makes more sense than a smelly old boot (that people wear on their feet).

9) They can't spell people's names
They spell 'Bill' as 'William' , 'Bob' as 'Robert' and 'Chuck' as 'Charles'. No wonder TV shows like 'Are you Smarter than a 5th Grader' are so popular in the US. These shows will probably just have very limited appeal everywhere else in the world!

10) They are bad at pretending French
I was out on those weekend markets one day, looking at some sweet potatoes, when the lady serving me said, "these goes well with 'erbs". That got me confused for a while, like what are 'erbs'?

Only after a while did it strike me that she meant 'herbs'!

Gosh, even the English know better than to have a silent pronunciation at the first consonant of the word! So will all you pseudo-French-Americans out there, learn this:

'In Hartford, Hereford and Hampshire, Hurricanes Hardly ever Happen. HaHa, HaHa!' [1]

---

[1] Quoted from Professor Higgins, in the movie 'My Fair Lady'. Ok, I added the Haha part. But hey, you don't laugh without your 'H's do you?

Proxy forwarding on Apache

If you're hosting on a web server that requires Apache to be the front-end, like using it as a virtual domain host for multiple domains, chances are it'll be difficult for you to swap Apache with any other web server, especially if the other domains are happily hosted without a problem. But what if you are required to use some other app servers without taking away Apache instead?

I'm not sure about most distributions, but mod_proxy comes default with the Apache distribution, but it's just not enabled in RedHat. So in order to make use of mod_proxy, you'll need to enable it first. Stick this somewhere in your Apache configuration file:

LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule proxy_http_module modules/mod_proxy_http.so

After which you can start your other app servers at unoccupied ports on your server, eg. 8080, and setup Apache to perform the forwarding. An example setup may look something like this:

<virtualhost *.80>
ServerName yourdomain.com
ServerAlias www.yourdomain.com
ServerAdmin admin@yourdomain.com

ErrorLog /var/log/[wherever.your.error.file.is]
CustomLog /var/log/[wherever.your.log.file.is] custom

ProxyPreserveHost On

# Example to serve the entire domainname, from the root directory (/)
ProxyPass        / http://127.0.0.1:8080/
ProxyPassReverse / http://127.0.0.1:8080/

</virtualhost>

The configuration assumes that your alternate app server is hosted on the same server on port 8080 as indicated, given that port 80 is already occupied by Apache. Otherwise this should allow all connections to be transparently forwarded to your new app server without losing functionality of your existing hosts.

Circumventing Java's Initialization Process

This is really not a circumvention per se, but rather an understanding of how the Java initialization behaviour works, and highlight the violation of the JVM specification, even in Sun's version, so that Java will be able to function pragmatically. The JVM specification (§2.17.4) indicates that:

A class or interface type T will be initialized immediately before one of the following occurs: * T is a class and an instance of T is created. [...]

This means that if object T is being created, then the class of T will have to be initialized. But is it always being upheld? For a fact, it's never upheld in a cyclic self-referencing case:

public class T {
int value;
T my_object;

static {
 my_object = new T();
 my_object.print();
 value = 1;
}

public void print() {
 System.out.println("The value is " + value);
}
}

If you ran the application, even after the object for T is being created, the value returned is '0' rather than '1' which violates the definition as described above. If this rule is not violated, if I am correct, on older JVMs like 1.0, this will translate into a cyclic situation, as the object creation will trigger it's object constructor, which will trigger the class constructor, which it is already in, and so on.

So what's the problem with that? The reason why initialization is important, is so that you'll expect a reliable value to be set by the time you use the class/object, which may not be necessarily be the case if that constraint is relaxed. Consider a fringe example like this:

// ---- DualWait1.java -----

public class DualWait1 {
 public static int value;
 static DualWait2 field;

 static {
     System.out.println("DualWait1.clinit() invoked");
     try { Thread.sleep(1000); }
     catch (Exception e) { throw new AssertionError(e); }    

     System.out.println("DualWait1.clinit(): creating a new DualWait2() object");
     field = new DualWait2(2);
     System.out.println("DualWait1.clinit(): setting value to 1");
     value = 1;
     System.out.println("DualWait1.clinit(): DualWait2.value="+DualWait2.value);
 }

 public DualWait1(int id) {
     System.out.println("DualWait1: constructor called id["+id+"]");
 }
}

// ---- DualWait2.java -----

public class DualWait2 {
 public static int value;
 static DualWait1 field;

 static {
     System.out.println("DualWait2.clinit() invoked");
     try { Thread.sleep(1000); }
     catch (Exception e) { throw new AssertionError(e); }     

     System.out.println("DualWait2.clinit(): creating a new DualWait1() object");
     field = new DualWait1(1);
     System.out.println("DualWait2.clinit(): done creating a DualWait1 object");
     System.out.println("DualWait2.clinit(): setting value to 1");
     value = 1;
     System.out.println("DualWait2.clinit(): DualWait1.value="+DualWait1.value);
 }

 /** Only for a placeholder. */
 public static void main(String args[]) {
     System.out.println("main(): DualWait1.field = " + DualWait1.field);
     System.out.println("main(): DualWait2.field = " + DualWait2.field);
 }

 public DualWait2(int id) {
     System.out.println("DualWait2: constructor called, id["+id+"]");
 }

}

Compile the two classfiles separately and once you've done so, you'll get the following output after execution:

DualWait2.clinit() invoked
DualWait2.clinit(): creating a new DualWait1() object
DualWait1.clinit() invoked
DualWait1.clinit(): creating a new DualWait2() object
DualWait2: constructor called, id[2]
DualWait1.clinit(): setting value to 1
DualWait1.clinit(): DualWait2.value=0
DualWait1: constructor called id[1]
DualWait2.clinit(): done creating a DualWait1 object
DualWait2.clinit(): setting value to 1
DualWait2.clinit(): DualWait1.value=1
main(): DualWait1.field = DualWait2@15ff48b
main(): DualWait2.field = DualWait1@affc70

The line in red is where the problem lies. At DualWait1's static constructor, we see that the value for DualWait2's value field is still not initialized, even after a DualWait2 object has already been constructed.

Normally this shouldn't be a problem for normal functioning cases, given that no coder in his right mind would write code like this to trigger the anomaly, so it's more of a curiosity than anything really useful. It's just useful to note, that specifications, while good-intentioned, may sometimes turn out to be impossible to fulfill anyway.

Difference between a Register-based and a Stack-based CPU

This is just one small little aspect of how a stack and a register based CPU differs. It is not by any means a good comparison of the intricate differences, but rather just a little trivia that I've remembered in an older CPU era. Back in those days, where people would try to make use of every single CPU cycle that's available, the 'optimal' way of clearing a register on for example, old 8086 machines, a programmer may code their application in C to make use of XOR negation to clear a register. In code, it looks like this:

int a = 8;      // just assigning a random value to clear to 0
a = 0;          // setting a variable to 0 normally

Versus the 'efficient' way:

int a = 8;
a ^= a;         // faster on register CPU, slower on stack CPU

If you translate that to x86 assembly code, it becomes:

MOV AX, 8
MOV AX, 0

Versus:

MOV AX, 8
XOR AX, AX      ; same number of instructions, but faster

The arithmetic operation of XOR vs MOV is just faster by 2-3 bus cycles. I might be wrong on the number but the key point is that the savings are really trivial.

Here's a snippet of code generated for a stack based architecture, for example Java:

BIPUSH 8;
ISTORE_0;
ICONST_0;
ISTORE_0;       // 4 instructions

Versus:

BIPUSH 8;
ISTORE_0;
ILOAD_0;
ILOAD_0;
IXOR;
ISTORE_0;       // 6 instructions!

And you've just added an unnecessary overhead increase of 50%. That's sometimes why people do mumble about stack based machine architectures like Java and C#, which probably have some kernels of truth. But with trivial savings like that, it's probably relevant for old machine architectures, but for modern day RISC machine architectures, the stack slots are mapped onto registers directly, and with compiler optimizations, the differences are probably immaterial.

As a comparison, if the programmer fires up his profiler and optimize on even just the trivial-est of loops, he'll probably end up having a faster application just by doing that, than having to manually tweak every single instance of the given example, with negligible gain (or even a loss of) performance. So if you're ever looking for a good example of an unnecessary and counterproductive 'premature optimization', this is it!

Compiling VLC for Streaming Music

Compiling media players from source are perhaps the most problematic and most confusing among most linux packages, given the vast number of dependencies required, and the different choices of codecs to choose from, with some of them just competing sounds and video libraries that provides similar features, adding to the confusion, making it an exercise not for the faint of heart.

I wouldn't have tried doing it myself, if not for the fact that I felt ransomed by pandora's music service:

1) It's only available within the United States, which luckily doesn't affect me for now while I'm here;

2) The flash player is excruciatingly slow, but more importantly that it causes music to skip uncontrollably when the CPU is under high load.

The second problem is a big issue, given that the player does not return to normalcy even after the CPU load subsides. If I were to do some compilation, or simply just tax X a little bit with some graphical applications, the flash player will start skipping, forcing me to stop and reload it.

I'm not sure what to blame in this case, as it could be the flash player's shoddy programming (which is doubtful, because it doesn't seem to happen to a Mac machine I use from time to time), or it's the flash or other associated libraries provided for linux, or maybe it could be an underlying hardware issue.

Since pandora's flash player high load requirement is the trigger for the problem, and the fact that they do not provide a native player version of it, I had to dump it and look for other alternatives. Fortunately there are still a number of good free internet radio stations around, and it happens that my favourite station, KKSF, does broadcast online, which makes it a plausible alternative to just listen to the music streams direct instead.

Unfortunately, of all the media player software I've installed, xine didn't seem know how to deal with the stream, and mplayer decided to conk out after spewing an error message saying that it required a windows library called 'avisynth.dll'. And after much Googling and trials, they all led to dead ends - none of the touted solutions worked.

Frustrated with my attempts, I've decided to install VLC, given that I had good success with it when playing most media on the Mac. But it turned out not to be as straightforward as I wanted: first of all, the mpeg player library, 'ffmpeg', refused to emerge, and keeps giving me some dubious compile errors.

But it was needed as a VLC dependency, to which is failing to compile because of some missing symbols that weren't available in my older version of 'ffmpeg' I had installed. In the end, I had to keep emerging older versions of 'ffmpeg' until emerge is finally happy to compile. And fortunately, it's a version that is compatible with VLC.

Then comes the tweaking of USE flags. The instructions are available here, and in addition to gentoo's install guide. The instructions given on by VLC's website is still lacking an essential USE flag, at least for me, that left me scratching my head for a while why I haven't got any sound despite the lack of error messages.

Given my experience with compiling mplayer in the past, the problem was easily rectified by the addition of the sound output management library/platform ALSA, so if you aren't having any sound as well, make sure you add the "alsa" USE flag.

Once compiled, I must say that out of all the available players I've used on linux, VLC is probably the most versatile one compared to xine and mplayer. But I'll give my overall preference to mplayer, given that it is strictly command line based, and looks better than the GUI versions. I do confess that GUIs looked bad because I don't use either Gnome or KDE, and hence the players I compile usually just uses the minimalistic gtk widgets, thus making them uglier than usual.

At least for now, music streaming works, and I'm restored to normal sanity again!

Tab Completion for Vim (Updated)

As I've said before, I wasn't really satisfied with the original tab completion script, which didn't perform all the possible search completion combinations vim is capable of. After accidentally overwriting my existing .vimrc file, I just thought it was high time I remedied the incomplete implementation of tab completion for vim to work properly with Java.

As with before, the script has to utilize existing auto-completion script, and the additional changes on my script now makes tab 'intelligently' to perform completion on incomplete methods and fields, rather than having only to be able to do so only at the start of the dot ('.'). Also, if tab occurs at locations where it doesn't fit the profile of a method or field (i.e, it's not in the pattern of 'package.class.methodname_or_fieldname', where package is optional), it will try to use vim's built-in keyword completion (<C-X><C-P>) instead.

Here's the script that you'll need to copy and paste into your .vimrc:

  " Modified tab completion. It works fine now.
function! My_TabComplete()
let line = getline('.')                         " curline
let substr = strpart(line, -1, col('.')+1)      " from start to cursor
let substr = matchstr(substr, "[^ \t]*$")       " word till cursor
if (strlen(substr)==0)                          " nothing to match on empty string
 return "\<tab>"
endif
let bool = match(substr, '\.')                  " position of period, if any  
if (bool==-1)
 return "\<C-X>\<C-P>"                         " existing text matching
else
 return "\<C-X>\<C-U>"                         " plugin matching 
endif
endfunction
autocmd BufNew,BufRead *.java inoremap <tab> <C-R>=My_TabComplete()<CR>

If you like reading this, you may also enjoy:

• Vim Tips for Java #1: Build Java files with Ant Automatically
  
• Vim Tips for Java #2: Using exuberant-ctags
  
• Vim Tips for Java #3: Use Omni-Completion (or Intellisense) for Automatic Syntax Completion
  
• Vim Tips for Java #5: Folding Code Blocks to Prevent Visual Blindness
  
• Vim Tips for Java #6: Auto Bracketing for Java
  

Writing your own custom loader for Java

One of the interesting things I've learnt about the Java is that, however much under the illusion that 'java' (or 'java.exe' for windows) is perceived as the JVM itself, the actual fact, is that it is actually not, but rather a very thin front-end for the JVM. The actual code that provides the functioning core of the JVM actually resides in the library (like 'libjvm.so' or 'jvm.dll'), and that the 'java' executable is just a thin veneer on top of the virtual machine.

To demonstrate that this is the case, I'll write a custom loader that invokes the JVM to load a simple Java class file. The code for the simple class file is as follows:

/**  Hello world app. */
public class HelloWorld {
 public static void main(String args[]) {
     System.out.println("Hello World");
 }

 public static void execute() {
     System.out.println("Executed from launcher!");
 }
}

The details to the custom loader are documented in Java's Invocation API, which is provided at the end of this article. The code for the loader is down to the bare minimum just for the example to work:

#include <stdlib.h>
#include <stdio.h>
#include <jni.h>

/* This is the program's "main" routine. */
int main (int argc, char *argv[]) {

 JavaVM *jvm;                    /* denotes a Java VM */
 JNIEnv *env;                    /* pointer to native method interface */
 JavaVMInitArgs vm_args;
 JavaVMOption options[1];

 jint res;
 jclass cls;
 jmethodID mid;

 /* IMPORTANT: need to specify vm_args version especially if you are not using JDK1.1.
     *            Otherwise, will the compiler will revert to using the 'JDK1_1InitArgs' struct. 
     */
 vm_args.version = JNI_VERSION_1_4;

 /* This option doesn't do anything, just to illustrate how to pass args to JVM. */
 options[0].optionString = "-verbose:none";
 vm_args.nOptions = 1;
 vm_args.options = options;
 vm_args.ignoreUnrecognized = JNI_FALSE;

 /* load and initialize a Java VM, return a JNI interface pointer in env */
 res = JNI_CreateJavaVM(&jvm,(void**)&env,&vm_args);
 if (res < 0) {
     fprintf(stderr, "Can't create Java VM\n");
     exit(1);
 }

 jclass ver;
 jmethodID print;

 ver = (*env)->FindClass(env, "sun/misc/Version");
 if (ver == 0) {
     fprintf(stderr, "Can't find Version");
 }
 print = (*env)->GetStaticMethodID(env, ver, "print", "()V");
 (*env)->CallStaticVoidMethod(env, ver, print);

 /* invoke the Main.test method using the JNI */
 cls = (*env)->FindClass(env, "HelloWorld");
 if (cls == 0) {
     fprintf(stderr, "Can't find HelloWorld.class\n");
     exit(1);
 }

 mid = (*env)->GetStaticMethodID(env, cls, "execute", "()V");
 if (mid==0) {
     fprintf(stderr, "No such method!\n");
     exit(1);
 }
 // otherwise execute this method
 (*env)->CallStaticVoidMethod(env, cls, mid);

 /* We are done. */
 (*jvm)->DestroyJavaVM(jvm);

 return 0;
}

What remains is just compiling and executing it. I'm very rusty on using 'make', and have really little experience with any of the gnu build tools (auto{make,conf} and family), but since the compilation is rather straightforward, you can pass it something like:

gcc -g -Wall -I/opt/jdk1.6.0_02/include/ -I/opt/jdk1.6.0_02/include/linux/ -L./jre/lib/i386/client/ -ljvm -o invoker invoker.c

Just change /opt/jdk1.6.0_02/include/{,linux} to where ever your java header files reside. One of the funny things I've found with gcc, was that no matter how I force the linker to link it with the java library I've provided, it always seems to link with the original libraries that came installed on my computer. So at the first execution the output comes out like this:

% invoker
java version "1.4.2-03"
Java(TM) 2 Runtime Environment, Standard Edition (build Blackdown-1.4.2-03)
Java HotSpot(TM) Server VM (build Blackdown-1.4.2-03, mixed mode)
Can't find HelloWorld.class

Not only it could not find HelloWorld.class, which simply resides in the same directory as 'invoker', it's telling me that it's running the original Blackdown 1.4 JVM that came default on my linux distro as well! Unfortunately that's not what I wanted. The way to remedy that is either to dynamically link to 'libjvm.so' (see [1] for details) or just cheat by modifying the LD_LIBRARY_PATH variable in linux:

% LD_LIBRARY_PATH=./jre/lib/i386/client/ ./invoker
java version "1.5.0_03"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_03-b07)
Java HotSpot(TM) Client VM (build 1.5.0_03-b07, mixed mode)
Executed from launcher!

By mangling LD_LIBRARY_PATH, I've just swapped out the JVM without even recompiling my invoker application, which surprised me just how trivial the 'java' executable actually is, even that's what everybody uses all the time.

Links
[1] The Invocation Interface: http://java.sun.com/docs/books/jni/html/invoke.html
[2] The Invocation API: http://java.sun.com/j2se/1.5.0/docs/guide/jni/spec/invocation.html

Bundling a minimal 'bare bones' JVM with your Application

One of the gripes why people dislike Java as a desktop application is probably because of the additional requisite of having the presence of the JVM, which is generally either not bundled with any of the major operating systems in the market today, or that it can be the wrong version of the JVM that is needed. This makes it really annoying for end users if they just wanted something that 'works out of the box'.

I got interested with the idea that, what if your application can create a seamless experience, as if it's like a binary application running on the native platform, by having the JVM embedded in as part of the package, so that most of the core application besides the loader can be written in Java?

The impetus for this is because that the entire JVM is usually a huge piece of bloat, and mostly, you don't really need to use every single feature that is present. So what about just packing only the actual components that really have to present for an application to work?

With respect to a simple application that prints out just 'Hello World', just how many files are there that the JVM really needs in order to function? So that's when I started experimenting by taking my Java installation and picking out each individual file apart, leaving only the files that would otherwise cause the JVM to fail. By trial and error, here's what I found to be required in order to have the minimal JVM functioning:

70k     jre/bin/java
103k    jre/bin
29k     jre/lib/i386/native_threads/libhpi.so
33k     jre/lib/i386/native_threads
4.3M    jre/lib/i386/client/libjvm.so
4.4M    jre/lib/i386/client
91k     jre/lib/i386/libzip.so
50k     jre/lib/i386/libverify.so
148k    jre/lib/i386/libjava.so
4.1k    jre/lib/i386/jvm.cfg
4.7M    jre/lib/i386
467k    jre/lib/rt.jar*

What you're seeing here is the output from a linux version of the JVM, hence the point to note from this is, that the file prefix and what follows after will be different for different operating systems, although the names should largely remain the same. What I mean by that is, that for example in Windows, 'java' will be 'java.exe', and for libraries like libjvm.so' will be corresponding to 'jvm.dll' instead.

The sum of the size of the files are roughly a little past 5Mb. But in order to get to this figure, there are a number of additional steps that I have to take, so in some ways this isn't actually a fully functioning JVM, for various reasons:

1) I've left out most of the 'core' dynamic libraries, like awt, sound, network, io, nio, awt and various libraries that resides in the /lib/i386/ directory. If the application attempts to utilize those corresponding java classes, the JVM will fail given the underlying libraries which provides the actual implementation aren't present.

2) Also because that actual binary libraries aren't around, there really isn't any need in having those corresponding class files as well, so I've taken it out from the core rt.jar jar package, which is where the Java system classes reside. The original rt.jar is probably 20Mb worth, so in this case, it is worth the trouble in doing that.

Caveat is, this is an error-prone process, and the files used by the JVM is guesstimated by passing the java launcher with the '-verbose:class' flag and capturing the resulting classes loaded, and subsequently removing all the remaining untouched files from 'rt.jar'. The list of classes loaded looks something like this:

% java -class:verbose HelloWorld
[Opened /opt/sun-jdk-1.5.0.08/jre/lib/rt.jar]
[Opened /opt/sun-jdk-1.5.0.08/jre/lib/jsse.jar]
[Opened /opt/sun-jdk-1.5.0.08/jre/lib/jce.jar]
[Opened /opt/sun-jdk-1.5.0.08/jre/lib/charsets.jar]
[Loaded java.lang.Object from /opt/sun-jdk-1.5.0.08/jre/lib/rt.jar]
[Loaded java.io.Serializable from /opt/sun-jdk-1.5.0.08/jre/lib/rt.jar]
[Loaded java.lang.Comparable from /opt/sun-jdk-1.5.0.08/jre/lib/rt.jar]

  ... [ lines truncated for brevity ] ...

[Loaded java.security.Principal from /opt/sun-jdk-1.5.0.08/jre/lib/rt.jar]
[Loaded java.security.cert.Certificate from /opt/sun-jdk-1.5.0.08/jre/lib/rt.jar]
[Loaded HelloWorld from file:/home/vincent/code/]
Hello World
[Loaded java.lang.Shutdown from /opt/sun-jdk-1.5.0.08/jre/lib/rt.jar]
[Loaded java.lang.Shutdown$Lock from /opt/sun-jdk-1.5.0.08/jre/lib/rt.jar]

Many people may be surprised that for a simple 'HelloWorld' application needs to load so many class files prior to execution. Because of that, and the series of auxiliary operations that the JVM performs before the application is start, this accounts for the 'slow start' phenomenon that we normally encounter with Java apps. In order to use this information, you'll probably need to pipe the result into a file, and perform some manipulation before you can pass the list to 'jar' for repackaging.

The error proneness that I mentioned about, lies in the fact that applications that can dynamically load other classes at any given point in time, which means you can have missing class files that are required which is not captured by the profile of a single instance of execution. This shouldn't be a problem as a developer, since it should be a relatively straightforward exercise to find out what system packages your application use.

After all that trimming, the entire embedded system adds up to about 5Mb worth, which is a sensible size for embedding into your application. But when compared to the offline java installer, which only comes about 20Mb compressed, the savings isn't really that substantial, given that downloads are getting cheaper and faster by the day. I'm sure your YouTube bandwidth use will easily have exceeded that in any given day, so it might just be as well that not worth the effort to have the JVM integrated into your application, and instead have an installation script that will seamlessly install the JVM to work with it.

Blackhat SEO Abuses

My original purpose for taking off the rel='nofollow' meta tag was to encourage readers to post constructive comments, and be rewarded for their efforts by allowing Google to index and increase the link count of their sites, but I don't take kindly to nonsensical 'exploits' by people trying to game my goodwill.

Since I've found my blog listed as a target for Blackhat SEO purposes, I've re-enabled the 'nofollow' tag. Thanks a few unscrupulous people, now nobody gets to benefit from their selfish actions.

So learn how to play nice next time, people.

Opera 9.50 vs Firefox 3.0

While Opera had been a fast browser, somehow it has been growing increasingly slow on my machine. Nowadays, at regular intervals and for no apparent reason, it has been causing my harddisk churn, even while it's idling. After regular usage, the problem worsens if I do not try to clear up the cache, as the automatic lookups of visited sites while typing urls into the address bar slows it to a crawl. And there doesn't seem to be any way of disabling this feature.

But it is still Opera's arbitrary scans that got me most antsy, given the possibility that I might have caught some malware with it, even after the fact that I had ran a few diagnostics (tops, ps, lsof), and not turning up with anything alarming. I mean, there just isn't any reason for the disk accesses, and so it shouldn't, even if it's doing something seemingly harmless.

After reading up on a review on Firefox 3.0 beta's glowing review and potential speed increases, I decided to give it a try, from the Firefox 1.5 and Opera 9.50 combo that I'm currently using. I've skipped Firefox 2.x totally, given it didn't seem to confer any potential speed advantages, besides being largely similar in terms of featureset compared to 1.x.

What surprised me is that Firefox 3.0 beta actually loads faster than Firefox 1.5. By my casual observation, where I normally start both my newsreader (Liferea) and Firefox simultaneously, what used to be always my newsreader window popping up first all the time, it is now replaced with Firefox 3.0's browser window opening ahead instead.

In terms of page rendition, 3.0 also feels much speedier than its 1.x cousins, which is a pleasant surprise, given that the norm in the software industry is to create incremental versions that offers more bloat and less speed. That is a credit I have to give to the Mozilla developers. Rendering speed is probably similar to Opera's, or at least in terms of perception, I can't really judge the difference between one to another. From a recent Opera convert, I've now reverted back to a Firefox convert again. Well, so much for brand loyalty!

Anyway, Firefox 3.0 does have the problem of having incorrect presentations for certain web pages, which as it stands, is probably still not ready for prime time yet. But the quirks are minor, and being me, I'm biased towards sacrificing presentation over speed, especially when the presentation problems are trivial.

The only major site I've encountered that wouldn't work with Firefox 3.0 is the 'enhanced' mode for Hotmail, which seems to just hang at the startup page. It isn't really much of a big deal, given that you're still be able to access it normally via the selecting the un-AJAX'ed mode from the startup page. Besides, rendition problems exist within Opera 9.50's beta too. If you try reading articles on Yahoo Finance Australia, you'll see what I mean.

The only other downside about 3.x, is that existing Firefox 2.x's plugins are not compatible with it yet, something that's largely expected to be remedied once 3.0 goes past beta. Not that it's a show-stopper anyway: besides the ad blocker plugin that I miss, otherwise there's really nothing that I'm would complain about the current incarnation of Firefox.

How I've learnt programming

The first time I heard of the C language was through my brother's schoolwork he had been given for one of his programming classes. Before which, all I ever knew was BASIC, which was I had a bit of a resentment for, because it could only work in computers that had the interpreter, rather than a fully independent executable program that could run autonomously in an operating environment.

What I wanted something that I could 'write once, and work everywhere' executable, i.e. in compiled binary form, which in those days, meant anything that could execute on DOS natively. Java would have failed my criteria terribly!

You could hardly blame me for having an awfully backwards view about computers in my younger years: I had never even seen an Apple Macintosh, or knew what was a PDP-11, UNIX or any other computer systems. The word 'computer' was unanimously equivalent to the 8086 architecture, and anything outside of it was regarded as a bizarre mistake. That notion got stuck for a long time, even after learning a bit more of UNIX and using more advanced processors like the Motorola 68000 series loaded with HP-UX for my CAD/CAM lessons.

BASIC was fine and dandy itself, but just not a language that an elite 'hacker' would use. I was rather enamored by the idea of becoming one, after getting pwned by a virus for the first time in my life. It was one of those prank 'keypress' viruses, which traps the keyboard interrupt to generate false keyevents of the keys you pressed at random intervals, causing you to believe that a key on your keyboard got stuck.

It was only after a long as 2 years later, after I junked the old Wang IBM-compatible, that I had learnt enough to realise that. My brother and mom never did though, and had always unfairly blamed the guy who sold us the salvaged equipment left over from Wang Corporation's bankruptcy.

Anyway, a big 'Damn You' to the author of 'keypress'! I'm sure with the pervasiveness of the Internet, you'll get to read this message and know that the prank you wrote is not funny!

Computer viruses were the 'in' or 'l33t' (hackerspeak for elite) things to do, and they were mostly written for gaining fame (or notoriety), unlike today where it is more driven towards commercial gain. Either way, viruses are destructive, the only difference being it was just senselessly destructive in the past. The most famous of those era, was probably the 'Stone' virus, which resided on the boot sector of the floppy disk. It had a pretty nasty payload, often overwriting the boot sector with junk past the trigger date. Barring a reformat of the disk, or a restoration of the boot sector (if someone had the knowledge to do so), the computer is rendered useless whenever the virus struck.

By the time my brother got into a polytechnic, he had 'required' a 'real' computer to work on, which I've always maintained that it was just an excuse, because of my observations, whether fairly or not, he's spent most of his time on just playing computer games. Anyway, it was to our good fortune that we've managed to get our second computer for free, from a friend of mom's, whose son had just recently upgraded to a new 16Mhz 386SX from his old 4Mhz 286 machine.

While old, it was a pretty functional machine. And the good news was, the machine had a CGA display, and hence colour capable, which made it well qualified for game playing. So most of the time, I would see the screen of 'Romance of the 3 Kingdoms' whenever my brother was at the computer desk.

The computer also had a copy of Borland's Turbo-C compiler in it, which was the original intent for the gift. But my brother wasn't terribly interested in programming, so it was hardly common to see him fire that bluish-green coloured IDE environment until he was pressed to, like when there the deadline for his assignments were closing. But because he had the 'right excuse' for legitimate access to the computer, I was relegated to only using the it only whenever he was not around.

So between the multiple fights we had over the use for the computer, it was fortunate since secondary school sessions were only half a day, I would get my chance to mess around with it while he was out during the mornings before my classes start.

That was when I got my first chance of getting my hands on a C programming book as well. Problem was, the book belonged to my brother, and he didn't like to share anything with me when we were young, so touching his books were way out of the question. But I kept doing it anyway, in secret. However, because I had really sweaty palms, my 'stealthy' accesses to his book were easily discovered: at the end of the year, his book was just in a rather rotten state, after soaking it with my palm sweat! (I'll take it that he did have a good reason for not letting me touch his books!)

Anyway, I remembered the excitement for compiling my very first C application, which I started out from without knowing any syntax, to peering over my brother's shoulders to learn enough about '#include <stdio.h>', 'if', 'else', 'printf()' and 'gets()'. At around the same time while doing that, I started having those delusions of grandeur, of the idea of writing an 'zork'-like RPG clone of the game, although it soon became obvious that my design wasn't going anywhere, given that writing countless 'if-else' statements for all the possible branches got tiring after a while. I also remembered how unimpressed I was about C, that for a supposedly advanced language, it didn't even have constructs like the 'GOTO statement'! (Edsger Dijkstra, eat your heart out!)

By the time I attended polytechnic myself, I was self-taught enough that I was already proficient at all those rudimentary programming concepts, so those programming assignments and exams were a breeze. Those days, I was able to maintain an illusion of scoring well without making any apparent effort at all, so some friends of mine thought it was just a matter of genius. But of course, the truth was far from it. It was all a matter of interest at play.

So that is why I largely do not believe in innate geniuses, but more in persistence and hard work. Because if you become good at whatever you're doing, you're automatically a 'talent', and that's irregardless of you're a fast or a slow learner, or whether if whatever you're doing is sexy/fun/cool or not, and it's irrespective of whether if others recognize you for whatever you're good at. What matters, is that you enjoy doing it.

Hard work is often less hard when you are sufficiently interested, and it was probably extremely lucky for me that I had found what I've enjoyed much earlier than most people. Knowing myself, I probably would have never been able to keep my motivation up for doing anything that I'm not interested in. So in all sense of the word, I should be considered lazy more than anything else.

Personally, I have met and is impressed by a number of people who have a remarkable ability to persevere, and to be 'successful' at studying or whatever drastically boring things they are doing out of necessity rather than interest, but I'm could have never been one of them. If I had to be forced to study to become a doctor or a lawyer, like all Asian parents aspire their kids to (which is dreadful advice, unless your it's of your own consented, informed choice), I would probably have fared pretty badly, and probably ended up hating my life.

Programming became a profession driven by interest, and because it is a passion, one's more willing to spend 14 hours a day at it, because he/she feels that it's fun, rather than because it is a means of paying your bills. It is just a good bonus if people are willing to pay for what you like to do.

But before anyone start seeing things through rosy-tinted lenses, I should qualify that, even with interest, in reality, most programming jobs will involved in writing code or solving problems that you are inherently not interested in. Which is why I'm always seeking to have financial self-sufficiency, so that you'll ultimately have enough to never have to worry about going hungry while doing something you love.

So what about you? Are you out there doing something you love as well?

It's Just Family

I have a troubled family, I kid you not. It had always been dysfunctional. I grew up in the lowest social-economic group during my younger years, but even as today, while we may have moved past from the hand-to-mouth existence we used to live in, but in terms of mindset, I think we may not have moved very far from that tiny, closed mentality inherent of my poverty-stricken past.

I knew the vacation was going to be running amock when it had a bad start. On the first night my family arrived, everybody was fighting on the smallest of things, such as issues with misplaced towels and such. Like little kids, everybody seem intent on bickering on these trivial things, between my mom, brother and sister-in-law. And after living physically away for so many years already, it was quite a bit of a shock having to re-acclimatise myself to such a dystopian familial feuding.

But the worst has yet to come, triggered from just an incident when mom had taken some extra bathroom paper towels from a public restroom, to which sis-in-law had perceived such action as being cheap, not understanding the background behind my mother's actions from her past. With that, and subsequent series of further miscommunication and confusion, who could have expected that it could blow into something of unimaginable proportions?

That finally erupted into a fight one night during supper in Vegas, after we returned from a performance one night. What should have been an uneventful meal, ended up being a series of arguments between both of them, with the night ending in tears for my sis-in-law, my brother hollering in frustration, and my mother fuming and returning to her room in the middle of the meal.

There wasn't much I could do at that moment. I was just in absolute shock of what had just transpired.

Even as we were nicely dressed that evening, it wasn't difficult picking out the unsophistication behind the facade of our fine attires, that this is a family who didn't know how to communicate, or one that has any social graces. I would be understating myself if I were to say that it was only just embarrassing, when everybody in the fully packed restaurant looked on while the spectacle was happening.

Only I remained on the table, and later joined by my brother after he returned from placating sister-in-law who went away crying. We had a sobering conversation on what was the best course of action to take, but the conversation somehow ended up going sideways: my brother ending our conversation with a speculation that whatever had happened, must be caused by the terrible burials plot that our ancestors have had!

Yes, my family is just that superstitious, but I didn't feel the need to exacerbate the situation in pointing out that the problem doesn't lie with the dead, but with the living. The hard truth is still that, we are the causes of our own actions. But after witnessing a 'Mike Tyson' moment for the night, a 'Round Two' is the last thing I would want to initiate!

Later that night, I went back, pulled my mother aside and chided her. Yes, a son chiding his mother! Nicely, of course. She did not understand that by yielding to her own emotional whims and causing a ruckus, there is no benefit to be gained from her actions, but only made things difficult for everybody: my brother sandwiched between her and sis-in-law, upsetting me, wasted my effort for planning this trip, the money paid for the entire vacation, and the only vacation time I had for the entire year.

Yet still, it was unimaginable to think that the only other thing she had in mind was wanting me to change her flight so that she can return home immediately. I was fuming mad, having flushed $6500 of my money down the drain, just because they couldn't keep their petty little mindsets out of the way, but even so, I had decided to keep my composure, and put what I was going to preach for the next 5 hours to practice, and that was: 'words have consequences'.

In choosing my words badly, getting mad and yelling at her, I would have only succeeded in upsetting her further and caused the cancellation of all the other shows and trips that I've already paid for, the money that I'll not be able to recover. So what ended up happening, was a counselling session lasting till 4am in the morning, explaining to her that even that she had been slighted and mistaken, her outbursts do not result in anything meaningful, especially after the fact that I had already made sis-in-law apologise. There isn't any reason for her to be fuming any more. But even after that whole night of 'peace talks', the point just didn't seem to register inside her head. She was like North Korea gone wild.

This is the second year I've instituted a annual vacation for mom already, and twice the vacations had ended up with fights like these. It wasn't surprising that I got pretty tired of all these firefighting, while she kept defending her actions and harping on her illogical arguments repeatedly. The night ended with me telling her that there wasn't any point in hearing her out anymore, and she should examine her own actions and see for herself if she has accomplished anything besides making everybody miserable, including herself.

While I had disavowed superstitions a long time ago, preferring reason to religion as a means of mediation, it is just impossible to reason when emotions are running high. Understanding my family, I decided that using religion would be most the most effective way in dealing with the situation, even if it felt like hitting below the belt. Subsequently, I told mother that what she had done was not good karma, and one day she may beget the fruits she's sowed.

That probably did the trick, and for the remaining of the vacation, things went on largely without any more major hitches, barring the initial awkwardness. Gradually the awkwardness subsided the next few days and everybody went back to talking terms again. By the end of the vacation, everybody have had a good time, and mom was particularly thankful for the wonderful trip I had for her. It was as if that incident had never happened at all!

For good or bad, and whatever troubles my family may have, in the end, they are still family. As people say, you can choose your friends, but you cannot choose your family. In all honesty, they are not bad people, it is just that they can be incredibly tough to deal with at times.

But unfortunately for me, even with the threat I've made to mother that she should not be expecting any more vacations in the future, she seemed to have promptly forgotten all about it at the end of it all, and is now looking forward that I take her to Taiwan for the next vacation!

Little does she know, I do plan to make good of my threat, well, at least for this year. As you all already know, there is such thing called 'karma', isn't there?

Much Ado About Resolutions

I've never had the habit of making new year resolutions, and I'll not make an exception by starting one now. The last time when I was told to make a resolution was by my primary school teacher when I was 12 years old. Obviously, she didn't understand the futility of having one, but I did: a week later, I had totally forgotten about it.

There is nothing remarkable about my inability to keep resolutions. Most people never do, because that's just how humans are. We all 'live in hope', making wishes year after year and yet without fail, almost every single time, the wishes fall flat on our faces. To me, it is more remarkable that only a few has picked up on the futility of this exercise.

A wish generally isn't sufficient in attaining what you want, it requires planning and concerted effort to accomplish. Most self-help books will have harped on this ad nauseam, so go to the nearest library and pick up a book; There won't be any merit for me in further expanding this.

'Live in hope', while positive sounding, is just insidious optimism, especially in the case of blind hope. Hoping for a positive outcome that has not a sliver of a chance does not empower, but only enfeebles the mind: it stops us from taking action, but only to leave things to chance and wish that someone, somehow will appear and make things right.

Hope is probably one of the compelling reasons why so many people adopt a faith. Not that it is necessarily a bad thing, by the way, especially when one has to emotionally rely on faith as a moral guidance. It is just sad when we have to act based on the tenets set by dead ancient history, which sometimes becomes irrelevant in modern context, than to act with better sensibilities and knowledge that's available to our own living, thinking senses today.

As my first post of the year, I wish a Happy New Year to all my friends, and to all of you who read my blog. But instead of just wishing for a good year ahead, I hope that you'll make more enlightened, informed decisions, and may you make the best of all possible choices for 2008!


PS: For those who doesn't know, or hasn't subscribed to my feed yet, please do so, as it usually has additional links anything that I find interesting or relevant to you. Thanks for the support!