Mostly Harmless

Geschafft.

Hi everybody,

I have almost completed by BSc Informatik Studies (BSc Computer Science), and I am now writing my BSc Thesis...

It is called

Parallelisierung von Genetischen Algorithmen für Anwendungen der Finanzwirtschaft

which can be roughly translated to

Parallelization of Genetic Algorithms for Applied Finance

More specifically, the financial application Domain is Modern Portfolio Theory, and it's extensions (integer constraints, Transaction costs). See Luenberger - Investment Science for an overview of the topic, and Maringer - Portfolio Management with Heuristic Optimization for the extensions.

The target platforms will be "traditional" shared memory processors (x86) using C/C++ and GPUs using CUDA.

I have three months to complete my thesis and I am starting right now..

That's UBER-fantastic.



After watching this you can probably easily figure out a way to parallelize this algo

And now you can measure quick sort in calories burnt.

Remember O(n log n) ? You better do...

Hi Folks,

Problem
I downladed the latest NVIDIA drivers for my 8800GT. Everything worked fine until I tried to start Borderlands.

I was pretty busy lately - as always - and wanted to relax for an hour or so.

It hung at the splash screen. Damn. It really hung. CPU 0%, NO IO being done. Nothing.

Argh...

I attached my VS.NET 2010 debugger to that process and took a glimpse.




Hmm.. nothing interesting. WaitForSingleObject means basically waiting for ownership of a Mutex or Win32 Kernel Event. A deadlock? Seems like some optimizations in the latest NVIDIA driver exposed a vulnerability of the game code towards deadlocks. It's probably not NVIDIAs fault ..

Anyway, the other threads didn't provide any useful info. Actually, I had no time. I just wanted to take one or two hours off and relax now.. So i decided to work-around it.

This is my quick hack
I decided to forcibly serialize / "single-thread" the execution..

I simply started Borderlands from Steam, as soon as the process started to appear in the process manager, I changed the affinity for that process to one physical CPU This won't make the execution single-threaded, but would reduce the probability of the dead lock because there is no hardware parallelism anymore wrt CPU cores.



You have to be quick If the splash screen appears, it's too late ... .



... Then the game started normally, it passed the splash screen ... I immediately activated all cores again.

If you have a SSD this may be difficult :-/ Anyway I could continue this way.

And off I went blasting off some psychos

REMEMBER: If it took more than one shot, you weren't using a Jakobs

A lot of people like to point out that code efficiency for webapps isn't relevant nowadays, code can be slow, even glacially slow given the fact that the webapp mostly waits for the database anyway.

This, quite frankly, is wrong almost all of the time.

Let's take a look at writes (since the reading problem can be solved by some kind of caching strategy - once you have almost everything cached you are measuring Dictionary/Map performance).

I invite you to turn on statement logging on your database and capture the SQL DML that's being emitted by your favourite webapp framework for a specific write use case (including the transaction boundaries). Turn off your logging again for max. performance. Normally your web framework will be capable of handling 5-20 non-cacheable user requests/sec that result in direct write requests to the database (measure with httperf or whatever). Now shut down your webapp, and run that queries directly against your database, using the correct transaction boundary. Then run queries in parallel. Get some solid numbers.

You may discover that the transaction and its queries complete in a few msecs, and you get a lot more requests/s to the DB right away. From a plain DB perspective on a developer machine. Now who's bottlenecking? Certainly not your DB.

If your queries really run slow, the first step is to ensure correct DB design and DB configuration. Your db design must be sane (2NF/3NF or BCNF) and indices must have been set correctly. Technical issues like full table locks with MyISAM matter. String operators and functions in general, LIKE queries etc. Yes, of course you will have to be able to optimize or circumvent really "hard" queries.. At the end of the day, with sane database design, you can get really decent performance.. on a totally normal, average box. And the database will prevail. A whole lot more is going to die before that CUSTOMERS table won't exist any more.

Of course, Disc IO is the upper limit for DB IO (if the db really fsyncs on every transaction), but even with std 7200RPM Discs on developer machines you can get fast performance with correct db design.

But don't tell me your webapp is only capable of handling just a few req/s per second for your average webapp because of your database.

It's highly probable that everything else is preventing you from achieving higher throughput...

...Your ORM that thinks its an object runtime
...Your favourite programming language
...Your beloved ultimate framework,
...Your own application code
...Missing caching strategy
...And a whole lot more

PS
I am not talking about ultra-high-traffic sites here, like facebook. I am talking about normal webapps with moderate traffic.. It makes me wonder why everybody tries to solve the problems facebook has to solve (millions of users and gazillions of internet traffic) ...

PPS
If in doubt, measure.

As found on the OpenMPI FAQ Page : OpenMPI Faq



Cheers

As a Software Dev, during the lifetime of a software project, you probably are writing some kind of docs, brainstorming etc, documentation...

At least you should

Since you are a developer, this (probably) is you wishlist

• everything should be checked into your VCS
• hence it shouldn't be a blob - a binary
• docs should be editable on multiple platforms
• by multiple users
• hence you would like to diff things
• open file formats.
• be able to use your beloved editor
• still all features of a word processor

Ideally - text files. Actually it's pretty funny. Developers stick to text files while all kind of interactive apps exist for the rest of the world. But we choose independence. Still we would like to have cross references, images, etc..

Zim hits this sweet spot. You get an explorer-like desktop user interface to a wiki that is saved as plain text files on your harddrive. You get the wysiwyg thing, wiki cross-references and export functionality.

Now there's no reason not to document features, requirements and code

Actually I am a huge OneNote Fan and User. I guess that's why I like Zim, too...

(Update: me happy ... DataNucleus Platform 3.0M4 released ...
"RDBMS : Support JDOQL String.equalsIgnoreCase()")

I have been hacking around with JDO Typesafe for a private litte project - it's pretty nice. Actually, JDO typesafe isn't part of the standard -yet.

I hit an issue with this query utilizing equalsIgnoreCase - I posted a message on the Datanucleus bulletin board

I guess it simply isn't implemented yet

I have to prepare for an exam therefore I didn't have time to take a closer look at the code.

But I did take a glance at the code. All in all, the code quality of the Data Nucleus platform seems to be lightyears ahead of hibernate. Not to mention one other word: bugs.

Another small problem
On the downside, it's pretty hard to get Spring 3.0X support for one particular feature- which is a convenience feature, not critical: org.springframework.orm.jdo.TransactionAwarePersistenceManagerFactoryProxy, mentioned on this page .

Basically, for JDO Typesafe you need a JDOPersistenceManager, not a PersistenceManager - it's a DataNucleus feature right now, not part of the standard yet. This sounds like a solveable problem, with pure OO you can use covariance - but here we are dealing with a proxy thats created with CGlib at runtime, and its built to mimic PersistenceManager (of course!). You can't get away with simply extending TransactionAwarePersistenceManagerFactoryProxy and overriding some methods. Therefore I don't use TransactionAwarePersistenceManagerFactoryProxy - Its convenience anyway.

Back to basic, a simple, plain and clean page.

Reason: Small is beautiful. And this blog is powerful enough for dynamic content.

Good bye, old page:

Just wo-n-d-er-ing



I also witnessed the classic symptoms of GDI resource exhaution: greyed-out Icons, Areas, Text etc.

I used RAII to solve that problem when I was doing Win32 C++ Programming.

Over and out

Why you should learn just a little bit of Awk

Over and out.

Video:




Via the good folks at netbooknews.de.

Its just funny because I did stuff like this myself a long,long time ago ... 2005 .. Dude time is moving on so fast.. Good old days

So,

I just moved my site to the Serendipity Weblog system.

All older entries can be found here.