Web Monstrosities

I was wondering today why my homebrew java apps didn't compile, the version range resolution somehow didn't work as expected

Failed to resolve artifact.



Couldn't find a version in [2.0.0.m1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.1.0,

1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.1.6, 2.0.0-m2, 2.0.0-m3, 2.0.0-m4,

2.0.0-release, 2.0.1, 2.0.2, 2.0.3, 2.0.4, 2.0.5, 2.1.0-m1, 2.1.0-m2,

2.1.0-m3, 2.1.0-release, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.2.0-m1, 2.2.0-m2,

2.2.0-m3, 2.2.0-release, 2.2.1, 2.2.2, 2.2.3, 3.0.0-m1, 3.0.0-m2, 3.0.0-m3]

to match range [3.0,)

  org.datanucleus:datanucleus-core:jar:null

Well, shouldn't 3.0.0.-m1 (m2,m3) at least mach 3.0. ? As it turns out, as soon as a version string "violates" the maven2 naming schemes, your whole version string is being interpreted as a qualifier.

This discussion @ stackoverflow demonstrates that problem

Well are version ranges a bad thing? IMHO Version ranges are fine in certain cases, if you require a certain amount of api compatibility for your code to work.

I took a quick glance at the 2009 sortbenchmark.org results, where Hadoop reached the first place.

The benchmark category is called GraySort
Metric: Sort rate (TBs / minute) achieved while sorting a very large amount of data (currently 100 TB minimum).

This is how it looked like


73 times more nodes!?
(the raw cpu power of the nodes are roughly comparable - both Dual Quadcore)

Even though the benchmarks themselves aren't 100% identical (but triton was also sorting 100 terabytes), I think its remarkable..

Sometimes you just have to look at the numbers.

A lot of people like to point out that code efficiency for webapps isn't relevant nowadays, code can be slow, even glacially slow given the fact that the webapp mostly waits for the database anyway.

This, quite frankly, is wrong almost all of the time.

Let's take a look at writes (since the reading problem can be solved by some kind of caching strategy - once you have almost everything cached you are measuring Dictionary/Map performance).

I invite you to turn on statement logging on your database and capture the SQL DML that's being emitted by your favourite webapp framework for a specific write use case (including the transaction boundaries). Turn off your logging again for max. performance. Normally your web framework will be capable of handling 5-20 non-cacheable user requests/sec that result in direct write requests to the database (measure with httperf or whatever). Now shut down your webapp, and run that queries directly against your database, using the correct transaction boundary. Then run queries in parallel. Get some solid numbers.

You may discover that the transaction and its queries complete in a few msecs, and you get a lot more requests/s to the DB right away. From a plain DB perspective on a developer machine. Now who's bottlenecking? Certainly not your DB.

If your queries really run slow, the first step is to ensure correct DB design and DB configuration. Your db design must be sane (2NF/3NF or BCNF) and indices must have been set correctly. Technical issues like full table locks with MyISAM matter. String operators and functions in general, LIKE queries etc. Yes, of course you will have to be able to optimize or circumvent really "hard" queries.. At the end of the day, with sane database design, you can get really decent performance.. on a totally normal, average box. And the database will prevail. A whole lot more is going to die before that CUSTOMERS table won't exist any more.

Of course, Disc IO is the upper limit for DB IO (if the db really fsyncs on every transaction), but even with std 7200RPM Discs on developer machines you can get fast performance with correct db design.

But don't tell me your webapp is only capable of handling just a few req/s per second for your average webapp because of your database.

It's highly probable that everything else is preventing you from achieving higher throughput...

...Your ORM that thinks its an object runtime
...Your favourite programming language
...Your beloved ultimate framework,
...Your own application code
...Missing caching strategy
...And a whole lot more

PS
I am not talking about ultra-high-traffic sites here, like facebook. I am talking about normal webapps with moderate traffic.. It makes me wonder why everybody tries to solve the problems facebook has to solve (millions of users and gazillions of internet traffic) ...

PPS
If in doubt, measure.

Development turn-around times for java development can be ridicously high, especially if you add things like grails to the soup.

Setting explicit JVM params such as -Xmx , Permgen size are somehow mandatory. Using 4GB RAM (for web programming, lol) is important, too.

Two other things I consider useful (for the "offiical" Sun Java SDK):

•Use the -client JVM param if you're restarting your app over and over again. This can speed up the startup times because -server is optimized for a JVM that must be running all the time and therefore does some very time consuming JIT operations.

•Tune kernel parameters. I set kernel.shmmax to the amount of available physicial memory (in bytes). I have a thorough understanding of shared memory and its usages cause I come from the C++ world, anyway I don't know all the implications /wrt Java. It made a difference

I am still not really happy with the end results - its far too slow to be funny.

Lets say you are about to evaluate Spring Roo. Lets start the with the Get Started Guide.
There's this very innocent looking line:

roo> gwt setup

Yeah. Lets say gwt doesn't work for whatever reasons with your configuration.
How do you uninstall/remove gwt?
What does the roo console say?

roo> hint gwt
It's easy to create a GWT client in your project.
Just type 'gwt setup' and press ENTER.
Note Roo's GWT support outputs GWT 2.1 applications.

. Nice but how do I uninstall?

Well, If anyone has found a simply way to do that, please just comment.

Man thats sooo enterprisey.

Hey, I am used to getting tons of stacktraces on initial plumbing/integration from any combination of OSS java components, and I can handle a lot of them. Some people would describe this job as "java enterprise developer."

But its a sunday, after all...

..Is the sentence I came up with while struggling with an Convention over Configuration Framework.

Because,as Tim Peters said in this document

Explicit is better than implicit.

Nuff said...

(P.S.: Nope, not Spring Roo)

If you are doing AJAX calls from javascript to a server in a different domain or even in the same domain but using a different port, the Same Origin Policy will strike you

JSONP is a solution to this problem. What is JSONP?

The whole idea behind that approach is that

 <javascript src="www.blabla.com"> 

works, therefore you can load javascript source from a different domain...

Therefore your Server on the different domain simply generates Javascript that "contains your data", it generates a method invocation where the parameters contain the required data (f.e. as JSON). The method is provided by you on invocation time.

If your normal data looked like this:

 { ['id':12, 'name' : 'Alfred e Neumann'] } 

and your callback is called myAjaxCallback, this is what the server would create:

 myAjaxCallback( { ['id':12, 'name' : 'Alfred e Neumann'] } ) 

Pretty simple.

The best thing is that JQuery supports this transparently

Thats just about it ...