Another way to find out where a class was loaded from is by using the getCodeSource method, like so:

public static void which(Class aClass) throws Exception {
    System.out.println(aClass.getProtectionDomain().getCodeSource().getLocation());
}

I used to use this so much that Ravi finally decided to check it into cvs. Thanks buddy! Lot of people wondered why I always called it which, it is in memory of the Unix command which, which would tell you the directory in the PATH variable from which the shell was picking up the exectuable from.

And if you are coming for an interview and give this as the answer to my question, you better be able to explain what a protection domain is and when/by-whom is the codeSource set.

Update: I was inspired to blog about this after I spoke with Deepak about this. I didn’t know he had a blog and he had blogged about the same topic too!

For the past six months or so I have been having serious problems with blogger. It was not able to publish my posts or the comments I was receiving on them to the server. All attempts to get in touch with the folks at blogger were in vain and finally I decided to move out.

The blog is now hosted at http://www.0xcafefeed.com. The old url redirects to this site. I am running wordpress. Let me know if you have any comments on the design or issues with commenting or any other suggestions.

Day before:

Too many issues resizing photos in Java. Takes too much memory, takes too long, runs out of mem on Mac. A quick comparison with ImageResizer PowerToy shows that the Java code is taking too much memory and time. The difference is an order of magnitude higher on V’s machine which is slightly older and has lesser memory.

Yesterday (and day before):

Implement a photo resize server using GDI+. Go to bed happy that mem consumption is low, speed is very good and all kinks in passing around UTF-8 filenames between Java and VC++ have been ironed out.

Noon:

J integrates the new resizer into the code base.

2pm:

J verifies perf is 4 times better

6pm:

Code is checked in … waiting for integration build

8pm:

Leave for home

10pm:

Receive an e-mail, integration build ready .. start downloading with the hope of gloating in the perf gains

10.30pm:

Installation started … baby crying … unplug the laptop move closer to baby … play with the baby and complete the installation … start the test case in background

10.35pm:

No sign of test completing … should have finished 2mins ago

10.40pm:

Suspect some “first time” bug, run the test case again …

10.46pm:

No luck … still pathetic perf … does not match what I saw in the morning

10.50pm:

Pissed … shoot of a mail to J … “dood, did you checkin the integration … i see no diff in perf …

10.55pm:

No reply … shall I wait till morn …

11:00pm:

Let me try to recreate morning’s perf numbers … run the main method on java wrapper for the resizer … the same run which took 2mins in the morning is taking 6mins … as bad as the java resizer …damn ..

11:10pm:

WTF?!!

11:11pm:

Maybe I messed up the java wrapper … run the main method on exe resizer …

11:18pm:

Just as bad … three times slower than what it was in the morn …

11:20pm to 12:05am:

Make many runs trying to change some code in the exe … nothing seems to improve perf

12:05am:

Low battery alert … curse the laptop … can’t run even a few hrs without power

12:07am:

Make some more changes to c++ code … build … run …

12:09am:

By Joe! Done in 2mins instead of 6!… damn thing … why is this small change causing the whole thing to be 3 times slower … do some googling …

12:10am:

Undo the changes … recompile and run … see if it becomes slow again …

12:09am:

Thighs are burning … prop up the laptop

12:12am:

Hain?! Done in 2mins … WTF … it was taking 6mins just a few mins ago …

12:15am:

Rollback all the changes … recompile … run …

12:17am:

Done in two mins … WTF … How can this be … Check the logs for the timining … it was taking 6mins till a few mins ago … and the whole thing is back to 2mins now … no changes done … desperate to blame it on something … maybe anti virus was running earlier (should I check the anti virus logs?)… maybe the windows kernel has cached the files now (should I restart the laptop and see?) … maybe it became slow when i was running on battery …

12:18am:

Is that really possible … will my CPU run slow if it is running out of power …

12:18am:

Or will it be slow if it is running on battery … to save power

12:19am:

Hain … what a joke … laptop can’t be so smart … it was just a coincidence … I plugged in the power and it became faster …

12:19am:

There are no coincidences in this business … unplug the power … run the test …

12:22am:

No sign of ending it really is slow

12:25am:

Back to 6mins for the test

12:25am:

… can’t be … really i mean … can the laptop be so smart … what was that thing that showed clock speed … cpu-z … google: cpu-z … download … run … wow! It is true … clock speed has been reduced to 598MHz instead of 1.6GHz

12:25am:

OK … now connect the power and see … there goes my theory … still 598MHz

12:25am:

Hmmm … maybe it realized though it is powered, I am not running any CPU intensive task …. could it really really that smart … nyaaah …

12:26am:

Start the test with power still connected … run cpu-z … whoa! 1.6GHz …

More googling reveals, this is Intel SpeedStep^TM technology at work. It can be enabled/disabled from BIOS. A quick intro at Bay Wolf. Implementation details at Intel.

Now I have to locate some java API to detect if the CPU is running at a lower clock speed so I can warn the users to power up!

Update June 05, 2008: I was wrong in suspecting my dying battery. Replacing it with a brand new one didn’t help.

Long long ago I was bugged by the fact that the screen would go blank when I was reading some content heavy websites. I opened up the power management dialog, noticed that the default power scheme used to switch off the monitor too soon when running on battery. So I decided to change the power scheme to Presentation. There is NO indication that this setting does anything other than affect: Monitor, Hard Disc and System Standby.

However, following Alexander’s suggestion when I changed the power scheme from “Presentation” to “Home/Office desktop”. I notice that the clock speed behaves the same when on battery or power: When there is no CPU intensive task, it runs on low clock speed and when there is a CPU intensive task it runs at the rated clock speed.

I wish XP warned me or atleast hinted in some fashion that the power scheme would affect the clock speed. *sigh*

If you enable message archiving, YMessenger saves all the conversations with your friends in C:\Program Files\Yahoo!\Messenger\Profiles\${userid}\Archive\Messages\ directory in files with the extension .dat. The menu option Contacts -> Message Archive shows all the archived conversations grouped by user.

Figuring out the format of this file was thrilling … but probably not as challenging as the codes faced by codebreakers in the real world. (Simon Sigh’s The code book did leave a lasting impression on me … as did his other books!)

If you consider the equation encoded-message = code(original-message), the real codebreakers have access to only the encoded message. They have to figure out the code and in the process figure out the original message. The biggest leverage I had was that I had to figure out only the codem given encoded-message and original-message.

For example, I could send myself the message “a” and look at the contents of the .dat file. Then I could send myself the message “aa” and look at the contents of the .dat file. Then send the message “b”, followed by “bb”, followed by “ab”. Looking at the changes to the .dat file after every step.

Each message (from your to your buddy or vice-versa) in the .dat file is represented by a Record. Every Record has a timestamp of when it was sent, whether it was from you or from your buddy and the formatted (bold, italic etc) message. Your profile name and the the name of your buddy can be derived from the name of the .dat file and the name of its parent directory respectively.

The format for a Record is:

• The first int (four bytes) represents the number of seconds from Java epoch (Jan 1, 1970)
• The second int … i don’t know what it is
• The third int indicates that the message is from you to your buddy if it is zero or from your buddy to you if it is non-zero
• The fourth int (msgLen) represents the length of the encoded formatted-message. There is no encoding/encryption till this point.
• Next msgLen bytes represent the encoded formatted-message

The formatting of the message is indicated by some special tokens in the .dat file. These font-attribute tokens always start with 0×1B5B. So if you type the message “this is NOT acceptable”, the formatted message would be “this is {[0x1B5B][0x31]}NOT{[0x1B5B]x[0x31]} acceptable” (where {} indicate one token and [] indicates bytes shown in hex value instead of ASCII values). {[0x1B5B][0x31]} is a token that indicates begin bold and {[0x1B5B]x[0x31]} indicated end bold.

There are tokens for (begin and end of) bold, italic, underline. Then there are tokens that mark the begining of custom and standard(/palette) colors. The ending of colors is indicated by a token that indicates begining of standard color: black! The only peculiarity I could notice was that they use HTML like tags (instead of tokens starting with 0×1B5B) when you send messages with color gradient.

The formatted-message is then encoded before saving into the record. One of the challenges with a web-desktop application is that while storing the encrypted data on the desktop, what key do we use for encryption? The key used for encryption has to be different for each user, should not be guessable by other users and should not be stored on the PC. One option could be to use the password of the user as the key. But, whenever the user changes the password the archive has to be decrypted with the old password and encrypted with the new one.

The alternate solution is to have a autogenerated key stored per user on the website. An authenticated user can download his key and decrypt the archive. Changing the password does not change this key.

YMessenger uses the simple XOR cipher to encrypt the messages. The key used for the cipher is highly guessable: your user-id! Every byte of the formatted-message is XOR’ed with a byte from the user-id. For example if you message was “Hello World!” and your user-id was “doofy“, then the encrypted bytes would be:
“[H^d][e^o][l^o][l^f][o^y][ ^d][W^o][o^o][r^f][l^y][d^d][!^o]”

The beauty of XOR cipher is that if encoded-message = xor-cipher(original-message, key) then original-message = xor-cipher(encoded-message, key)

Parser.java documents the file format in more detail. Main.java demonstrates how one can use the Parser to convert the .dat files to HTML or plain text format.

How did you spend your new year’s eve?!

Searching through mails also worked pretty well when was using Evolution on Linux. But search in Outlook sucks, especially if you are using IMAP. The amazing Lookout plugin for Outlook was my saviour. Unfortunately, it had its own problems. It used to crash my Outlook 2K often; and once Microsoft bought them, there was no hope of getting things fixed. Microsoft has been pushing its own Windows Desktop Search instead of Lookout. Though not as fast as Lookout … it is the compromise solution I have been using for the sake of stability.

I know there are other desktop search products out there, including the one from google. But the thing I like about windows desktop search is that I can do Outlook operations on the search results (like forward the mail or move them into folders). Now I am so addicted to search that I move mails to folders only when it has huge attachment. Otherwise, it is pretty much the gmail model. I only use Inbox.yyyy and sent-mail.yyyy folders actively (yyyy being the year). I considering setting up a rule in Outlook to save sent-mail in Inbox, that way I would have only Inbox.yyyy folders and I can do a threaded view of the conversations (i.e. the gmail’s “All Mail” label)!

Apart from email, our other significant communication medium at work is Yahoo! messenger. I archive all my conversations and refer to them very often. The unfortunate side effect of this is that some conversations that start on email are concluded on chat and six months later when I search mails, I do not find the mail with the conclusion. Over the years I have wizened up to search conversations on email and followup the search in ymessenger. Unfortunately, search in ymessenger also sucks! You have to do a manual search based on the timestamp of email conversation and people involved.

YMessenger saves conversations in C:\Program Files\Yahoo!\Messenger\Profiles\${userid}\Archive\Messages\ directory in files with the extension .dat. It would be nice to have Windows Desktop Search (WDS) index these files and show my conversation results when I search for communications. I can think of couple of approaches to achieve this:

• Convert ymessenger archives to Outlook mailbox format (.pst) and let WDS index it
• Convert ymessenger archives to RSS and import the RSS into Outlook using RSS Popper and once the messages are in Outlook, WDS will index it
• Convert the ymessenger archive files (.dat) to html format and have WDS index these. Probably the easiest integration, but the limitation would only issue will not be able to run searches of the type “customer requirements from:myYahooBuddy date:last month”
• WDS supports plugging-in IFilters to search new file types. I could implement IFilter to index the ymessenger archive files (.dat).

All of these presume there is some API to decode the content in the ymessenger archive files (.dat). The search is on!

Map<String,List<Thing>> map = new HashMap<String,List<Thing>>();

looks like:

Map<String,List<Thing>> map = new HashMap<>();

Though there is an alternate proposal to deduce the LHS type instead, like:

map := new HashMap<String,List<Thing>>();

I find Neal’s proposal more appealing and more natural to java. It feels so Java, that I have already used it thrice since morning only to be surprised by IDEA’s warnings! So, if anyone is counting votes, +1 from me.

It is generally known that, in Java, the order of the catch clauses is important. The more specific exceptions have to be handled first followed by the less specific exceptions. So, the following snippet of code causes a compilation error, as FileNotFoundException is more specific than IOException (FileNotFoundException extends IOException).

 try{
  //Some File I/O operations here
 }catch(IOException e){
  //handle the I/O error
 }catch(FileNotFoundException fnfe){
  //handle the case when the file is not found
 }

To fix it, you need to change the order in which the exceptions are handled by moving the more specific exception (FileNotFoundException) before the less specific exception (IOException), like so:

 try{
  //Some File I/O operations here
 }catch(FileNotFoundException fnfe){
  //handle the case when the file is not found
 }catch(IOException e){
  //handle the I/O error
 }

This change is so straight forward that, any smart IDE can do it for you

So Vinod’s question really was: Why didn’t the designers of the Java language make the compiler smart enough to sort the catch clauses automatically (instead of pushing the burden on to the IDEs/developers)?. To quote him verbatim (including the typos) from my messenger archive: “.. i mean you are asking the programmer to think like compiler than compiler think like a programmer … from a programmer perspective… i want to catch FNFE if the exception is of that type other wise cathc IO”. Interesting point .. I never thought of it before.

Spoiler: I don’t know the answer, what follows are my thoughts or possibly my stream of consciousness, like my article on Why is finalize method protected?.

Having found no clues in The Java language specification I thought the answer probably lies in the history of Java. A Brief History of the Green Project is a good place to start. This page gives history of Java (it was originally called Oak) and has a copy of the version 0.2 of the Oak language specification [PDF]. The spec gives an interesting perspective on how the Java language evolved.
Having found no clues in The Java language specification I thought the answer probably lies in the history of Java. A Brief History of the Green Project is a good place to start. This page gives history of Java (it was originally called Oak) and has a copy of the version 0.2 of the Oak language specification [PDF]. The spec gives an interesting perspective on how the Java language evolved.

Side tracking: Interesting tidbits from the Oak specification

• Throwable was earlier called GenericException
• Asynchronous Exceptions: one thread can throw an exception (using Thread’s postException() instance method) to another thread
• The protect/unprotect keywords
• You could use //* javadoc here notation to write java docs apart from /** javadoc here
• print and println were operators. System.out was possibly a refactoring
• Interfaces declared constants using const instead of public static final. Like const int aConstant = 42;
• Supported assertions, preconditions and postconditions
• Has no details on threads, serialization nor does it have a BNF

Side tracking again: Catching multiple exceptions in one catch clause

Many a times people ask me: “Why can’t I catch multiple exceptions in one catch clause, I generally end up pasting same error recovery code in all the catch clauses. Why isn’t a catch clause like a method signature, where I can have a comma separated list of all the exceptions to be handled?” What they want is some thing like this:

 try{
  //Some File I/O operations here
 }catch(FileNotFoundException fnfe, IOException e){
  //common error handling
 }

The question itself seems to have the answer. If all the exceptions were listed like method parameters, the above snippet of code would mean “Do the common error handling if BOTH FileNotFoundException and IOException are raised” instead of “Do the common error handling if EITHER FileNotFoundException or IOException is raised”. The solution probably would be to use the OR operator “||” instead of commas? Some thing like:

 try{
  //Some File I/O operations here
 }catch(FileNotFoundException||IOException||MyNewException e){
  //common error handling
 }

Incidentally, the Oak specification also compares catch clauses to method definitions. From section 9.4:

A catch clause is like a method definition with exactly one parameter and no return type. When an exception occurs, the runtime system searches the nested try/catch clauses. *snip*

If you have two overloaded methods called handle, of which one takes FileNotFoundException as a parameter and the other takes IOException as a parameter, java always knows which method to call. It automatically calls the most specific method based on the runtime type of the object.

 private void handle(FileNotFoundException fnfe){
 }

 private void handle(IOException e){
 }

Now, as suggested by the spec, each catch clause can be treated as an overloaded method which takes a subclass of Throwable as a method parameter and no return type. Now extending the method overloading analogy shouldn’t java be able to detect which catch clause to invoke? Unfortunately, the complete paragraph from section 9.4 reads:

A catch clause is like a method definition with exactly one parameter and no return type. When an exception occurs, the runtime system searches the nested try/catch clauses. The first one with a parameter type that is the same class or a superclass of the thrown object has its catch clause executed. After the catch clause executes, execution resumes after the try/catch statement. It is not possible for an exception handler to resume execution at the point that the exception occurred.

The question now is, instead of continuing the method definition analogy and supporting overloading semantics to the catch clauses, why does the spec say the first catch clause will be chosen?

One possible reason could be for ease of compiler development. This seems to be an unlikely motivation.

Other possible reason could be for code clarity. What if the java developers start to expect that all the exception handlers that match are invoked? The problem exists with or without auto-sorting of catch clauses. A switch like construct would have been more appropriate then:

 try{
  //Some File I/O operations here
 }catch(Throwable t){
  switchOnClass(t){ //using a hypothetical keyword switchOnClass
   case FileNotFoundException:
    //handle file not found error
    break;
   case FileNotFoundException:
   case MyNewException:
    //some processing for both FileNotFoundException and MyNewException
    break;
  }
 }

Or the other possible reason is because Java’s exception handling was based on C++’s (as mentioned in the foot notes of the Oak spec page 26). C++ allows multiple inheritance. So my class MusicStreamingException could extend both MusicPlayerException and IOException. Now assume the compiler see’s this piece of code:

 try{
  if(someCheckHere())
   throw new MusicStreamingException();
 }catch(IOException e){

 }catch(MusicPlayerException e){

 }

Both the catch clauses match equally and the compiler has no way of determining which one to invoke. Hence the best policy would be to choose the first catch clause. However, this would never happen in Java as it does not allow multiple inheritance, else the same problem would exist in overloaded methods. Is it possible that this requirement in the spec is only a legacy from C++? And can it be done away with without impacting the existing code?

Raymond Chen has written a lot of anecdotes on compatibility issues in windows. (See Compatibility Constraints and Handling Compatibility Hacks.) He has also authored a book with the same title as that of his blog: The old new thing, which I am sure will have even more interesting tidbits.

Even application developers have to worry about the API they expose and its backward compatibility. The fine people at viewvc believed the URL scheme used by viewvc is part of their “API” and had a thorough discussion on what changes could be made to the URL scheme before and after the 1.0 release!

Being in the middle tier, we seem to face more than our fair share of compatibility issues. First, the product is based on the specifications which are written in English. Each vendor is free to interpret it the way they want. Eventually this leads to customer queries like: “But this works in [plug the customer's current vendor name here]!” This results in new flags in the product configuration files. Over the years we have brought up such ambiguities in the expert groups and the newer versions of the specs clarify the interpretation.

The other category of issues arise from ensuring compatibility with the innumerable combinations of databases, drivers, operating systems and JVMs. JV talks about an issue we recently faced wherein upgrading the VM from 1.4.2_10 to 1.4.2_11 breaks our product. JDK 1.4 introduced a new feature of disconnected sockets. The new implementation was supposed to be backward compatible. However, the implementation had a bug (2126509 ). In the process of fixing this bug in 1.4.2_11, (I believe) they introduced a new bug which breaks backward compatibility.

Such explicit problems are easier to manage than the unwritten contracts. While debugging I rely heavily on the exceptions raised. For example when you cast an object to a type, if there is a ClassCastException, the message generally has the class name of the object. So, in your code, if you are casting to java.util.List and the exception message is java.util.String, you know you are casting a String to a List. Also if you are casting to foo.bar.Klass and the message name is also foo.bar.Klass, then it is a class loading issue. The object you have loaded is of the same class but loaded from a different classloader. Looking at the exception message I could figure out whether it was a genuinely wrong cast or a ClassLoader issue. Unfortunately in JDK 1.4 ClassCastExceptions do not have a message. For a developer this is an annoyance, but is it worth logging a bug?

Here is a sample output with different jdks:

D:\code\41>java -cp PS30\tp\classes ClassCastTest
1.5.0_06
java.lang.ClassCastException: java.lang.String
        at ClassCastTest.main(ClassCastTest.java:25)

D:\code\41>java -cp PS30\tp\classes ClassCastTest
1.4.2_10
java.lang.ClassCastException
        at ClassCastTest.main(ClassCastTest.java:25)

D:\code\41>java -cp PS30\tp\classes ClassCastTest
1.3.1_16
java.lang.ClassCastException: java.lang.String
        at ClassCastTest.main(ClassCastTest.java:25)

Would you write unit tests for the output messages?

Thanks for the tips buddy … look forward to more of them.

While I was installing the plugin (firefox waits for a couple of seconds before enabling the install button … i wonder why?!), I see a message from deeps:

Deeps: http://www.geocities.com/junebug585

…and he logs out. I promptly clicked on the link which showed the page:

“Hmmm … password” I say. I ping deeps …”Dood … it’s asking for password”. No response. Being a stickler for online security (ask my wife on how i nag her into setting a different password for each site and make sure she does not note down her passwords anywhere!) I think to myself… “Hey! This could be a phishing site!” (what with me looking at Google SafeBrowsing site just a few mins ago). “Very well”, I tell myself, “… the url is Yahoo! Geocities, the logos and the layout looks ok … Why would deep try to phish my yahoo account details … what the hell .. let’s try to login”. Key in the user id pass nothing happens. Back to Yahoo home page. “D’oh! Have I been phished?!” with this nagging doubt go have a shower … come back… notice deeps has replied to my message:

Deeps: what login?
Deeps: did u a get any message from me?
Deeps: i did not send it..

“Oh God! I have been phished!! Change the password quickly.” Chirpy wifey: “Breakfast time!” “Oh God! Imagine after all my taunts … I’ve been phished … how am I going to tell her this?! Have I really been phished?!” So I decide to go back to the site and verify. LiveHTTPHeaders shows:

http://www2.fiberbit.net/form/mailto.cgi
POST /form/mailto.cgi HTTP/1.1
Host: www2.fiberbit.net
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.6) Gecko/20060728 Firefox/1.5.0.6
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://www.geocities.com/junebug585/?200614
Content-Type: application/x-www-form-urlencoded
Content-Length: 138
Mail_From=GOD&Mail_To=jawsy1%40gmail.com&Mail_Subject=Gift&Next_Page=http%3A%2F%2Fwww.yahoo.com
&.pd=fpctx_ver%253d0&login=asdf&passwd=asdf

HTTP/1.x 302 Moved Temporarily
Date: Thu, 14 Sep 2006 04:01:30 GMT
Server: Apache/1.3.26 (Unix) mod_perl/1.26
Location: http://www.yahoo.com
Content-Type: text/html; charset=iso-8859-1
X-Cache: MISS from downloads.pramati.com
X-Cache-Lookup: MISS from downloads.pramati.com:3128
Connection: close

“Oh no! I really have been phished! Bugger has mailed my yahoo password to himself!! Change the password … change the password … change the password!”. Wifey, annoyed after waiting for me at the b’fast table: “Wot’chu doin buster?!” … where do I hide my face now?!

The phisher is not Deeps… some one phished his id and sent me the message. Maybe it is not even junebug585 whose geocities site was used to phish for my account details … maybe that id was also phished and misused. Maybe it is not even jawsy1@gmail.com where my userid/pass have been mailed … maybe that id/pass also have been phished?! I wonder who all will receive a message from my id now and be phished!

Imagine the coincidence, I am finicky abt internet security, I was just checking out Google’s SafeBrowsing and I got phished! “I took a chance typing my yahoo id, surely I wouldn’t have taken a chance if the site asked for my bank account-nos/user-ids” I rationalize. But still … what if the phisher downloaded all my password reminder mails from my yahoo mailbox??!!

All the best dad!

… now the painful part of reporting abuse to yahoo and gmail …

Update: Looks like I am not the first! … and looks like google pages are also being used for phishing. Google: yahoo geocities phishing

Update 1: Yahoo!’s soln for phishing?! (via deeps):

———- Forwarded message ———-
From: Kalyan K Kumar
Date: Sep 14, 2006 11:04 AM
Subject: phishing
To: sammelan

keep an eye on those yahoo login look alike geocities links. don’t enter
yahoo password anywhere
other than login.yahoo.com
you can setup a sign in seal to protect partly.

http://protect.login.yahoo.com/