My BSc Thesis Wed, Sep 11. 2013
Hi everybody,
I have almost completed by BSc Informatik Studies (BSc Computer Science), and I am now writing my BSc Thesis...
It is called
which can be roughly translated to
More specifically, the financial application Domain is Modern Portfolio Theory, and it's extensions (integer constraints, Transaction costs). See Luenberger - Investment Science for an overview of the topic, and Maringer - Portfolio Management with Heuristic Optimization for the extensions.
The target platforms will be "traditional" shared memory processors (x86) using C/C++ and GPUs using CUDA.
I have three months to complete my thesis and I am starting right now..
I have almost completed by BSc Informatik Studies (BSc Computer Science), and I am now writing my BSc Thesis...
It is called
Parallelisierung von Genetischen Algorithmen für Anwendungen der Finanzwirtschaft
which can be roughly translated to
Parallelization of Genetic Algorithms for Applied Finance
More specifically, the financial application Domain is Modern Portfolio Theory, and it's extensions (integer constraints, Transaction costs). See Luenberger - Investment Science for an overview of the topic, and Maringer - Portfolio Management with Heuristic Optimization for the extensions.
The target platforms will be "traditional" shared memory processors (x86) using C/C++ and GPUs using CUDA.
I have three months to complete my thesis and I am starting right now..
Quick-sort with Hungarian folk dance Tue, May 8. 2012
That's UBER-fantastic.
After watching this you can probably easily figure out a way to parallelize this algo
And now you can measure quick sort in calories burnt.
Remember O(n log n) ? You better do...
After watching this you can probably easily figure out a way to parallelize this algo
And now you can measure quick sort in calories burnt.
Remember O(n log n) ? You better do...
Latest NVIDIA 285.58 WHQL + Quadcore + Borderlands == Deadlock Sun, Nov 13. 2011
Hi Folks,
Problem
I downladed the latest NVIDIA drivers for my 8800GT. Everything worked fine until I tried to start Borderlands.
I was pretty busy lately - as always - and wanted to relax for an hour or so.
It hung at the splash screen. Damn. It really hung. CPU 0%, NO IO being done. Nothing.
Argh...
I attached my VS.NET 2010 debugger to that process and took a glimpse.

Hmm.. nothing interesting. WaitForSingleObject means basically waiting for ownership of a Mutex or Win32 Kernel Event. A deadlock? Seems like some optimizations in the latest NVIDIA driver exposed a vulnerability of the game code towards deadlocks. It's probably not NVIDIAs fault ..
Anyway, the other threads didn't provide any useful info. Actually, I had no time. I just wanted to take one or two hours off and relax now.. So i decided to work-around it.
This is my quick hack
I decided to forcibly serialize / "single-thread" the execution..
I simply started Borderlands from Steam, as soon as the process started to appear in the process manager, I changed the affinity for that process to one physical CPU
This won't make the execution single-threaded, but would reduce the probability of the dead lock because there is no hardware parallelism anymore wrt CPU cores.

You have to be quick
If the splash screen appears, it's too late ... .

... Then the game started normally, it passed the splash screen ... I immediately activated all cores again.
If you have a SSD this may be difficult :-/ Anyway I could continue this way.
And off I went blasting off some psychos
REMEMBER: If it took more than one shot, you weren't using a Jakobs
Problem
I downladed the latest NVIDIA drivers for my 8800GT. Everything worked fine until I tried to start Borderlands.
I was pretty busy lately - as always - and wanted to relax for an hour or so.
It hung at the splash screen. Damn. It really hung. CPU 0%, NO IO being done. Nothing.
Argh...
I attached my VS.NET 2010 debugger to that process and took a glimpse.
Hmm.. nothing interesting. WaitForSingleObject means basically waiting for ownership of a Mutex or Win32 Kernel Event. A deadlock? Seems like some optimizations in the latest NVIDIA driver exposed a vulnerability of the game code towards deadlocks. It's probably not NVIDIAs fault ..
Anyway, the other threads didn't provide any useful info. Actually, I had no time. I just wanted to take one or two hours off and relax now.. So i decided to work-around it.
This is my quick hack
I decided to forcibly serialize / "single-thread" the execution..

I simply started Borderlands from Steam, as soon as the process started to appear in the process manager, I changed the affinity for that process to one physical CPU

You have to be quick

... Then the game started normally, it passed the splash screen ... I immediately activated all cores again.
If you have a SSD this may be difficult :-/ Anyway I could continue this way.
And off I went blasting off some psychos

REMEMBER: If it took more than one shot, you weren't using a Jakobs
Posted by Amanjit Singh Gill
in C++, Mostly Harmless, Windows programming Comments: (0)
Trackbacks: (0)
Normally, the database isn't the bottleneck. Tue, Oct 18. 2011
A lot of people like to point out that code efficiency for webapps isn't relevant nowadays, code can be slow, even glacially slow given the fact that the webapp mostly waits for the database anyway.
This, quite frankly, is wrong almost all of the time.
Let's take a look at writes (since the reading problem can be solved by some kind of caching strategy - once you have almost everything cached you are measuring Dictionary/Map performance).
I invite you to turn on statement logging on your database and capture the SQL DML that's being emitted by your favourite webapp framework for a specific write use case (including the transaction boundaries). Turn off your logging again for max. performance. Normally your web framework will be capable of handling 5-20 non-cacheable user requests/sec that result in direct write requests to the database (measure with httperf or whatever). Now shut down your webapp, and run that queries directly against your database, using the correct transaction boundary. Then run queries in parallel. Get some solid numbers.
You may discover that the transaction and its queries complete in a few msecs, and you get a lot more requests/s to the DB right away. From a plain DB perspective on a developer machine. Now who's bottlenecking? Certainly not your DB.
If your queries really run slow, the first step is to ensure correct DB design and DB configuration. Your db design must be sane (2NF/3NF or BCNF) and indices must have been set correctly. Technical issues like full table locks with MyISAM matter. String operators and functions in general, LIKE queries etc. Yes, of course you will have to be able to optimize or circumvent really "hard" queries.. At the end of the day, with sane database design, you can get really decent performance.. on a totally normal, average box. And the database will prevail. A whole lot more is going to die before that CUSTOMERS table won't exist any more.
Of course, Disc IO is the upper limit for DB IO (if the db really fsyncs on every transaction), but even with std 7200RPM Discs on developer machines you can get fast performance with correct db design.
But don't tell me your webapp is only capable of handling just a few req/s per second for your average webapp because of your database.
It's highly probable that everything else is preventing you from achieving higher throughput...
...Your ORM that thinks its an object runtime
...Your favourite programming language
...Your beloved ultimate framework,
...Your own application code
...Missing caching strategy
...And a whole lot more
PS
I am not talking about ultra-high-traffic sites here, like facebook. I am talking about normal webapps with moderate traffic.. It makes me wonder why everybody tries to solve the problems facebook has to solve (millions of users and gazillions of internet traffic) ...
PPS
If in doubt, measure.
This, quite frankly, is wrong almost all of the time.

Let's take a look at writes (since the reading problem can be solved by some kind of caching strategy - once you have almost everything cached you are measuring Dictionary/Map performance).
I invite you to turn on statement logging on your database and capture the SQL DML that's being emitted by your favourite webapp framework for a specific write use case (including the transaction boundaries). Turn off your logging again for max. performance. Normally your web framework will be capable of handling 5-20 non-cacheable user requests/sec that result in direct write requests to the database (measure with httperf or whatever). Now shut down your webapp, and run that queries directly against your database, using the correct transaction boundary. Then run queries in parallel. Get some solid numbers.
You may discover that the transaction and its queries complete in a few msecs, and you get a lot more requests/s to the DB right away. From a plain DB perspective on a developer machine. Now who's bottlenecking? Certainly not your DB.
If your queries really run slow, the first step is to ensure correct DB design and DB configuration. Your db design must be sane (2NF/3NF or BCNF) and indices must have been set correctly. Technical issues like full table locks with MyISAM matter. String operators and functions in general, LIKE queries etc. Yes, of course you will have to be able to optimize or circumvent really "hard" queries.. At the end of the day, with sane database design, you can get really decent performance.. on a totally normal, average box. And the database will prevail. A whole lot more is going to die before that CUSTOMERS table won't exist any more.
Of course, Disc IO is the upper limit for DB IO (if the db really fsyncs on every transaction), but even with std 7200RPM Discs on developer machines you can get fast performance with correct db design.
But don't tell me your webapp is only capable of handling just a few req/s per second for your average webapp because of your database.
It's highly probable that everything else is preventing you from achieving higher throughput...
...Your ORM that thinks its an object runtime
...Your favourite programming language
...Your beloved ultimate framework,
...Your own application code
...Missing caching strategy
...And a whole lot more
PS
I am not talking about ultra-high-traffic sites here, like facebook. I am talking about normal webapps with moderate traffic.. It makes me wonder why everybody tries to solve the problems facebook has to solve (millions of users and gazillions of internet traffic) ...
PPS
If in doubt, measure.
Howto: Make a clear statement in a FAQ Mon, Oct 17. 2011
Zim - a really cool desktop wiki Tue, Jul 19. 2011
As a Software Dev, during the lifetime of a software project, you probably are writing some kind of docs, brainstorming etc, documentation...
At least you should
Since you are a developer, this (probably) is you wishlist
• everything should be checked into your VCS
• hence it shouldn't be a blob - a binary
• docs should be editable on multiple platforms
• by multiple users
• hence you would like to diff things
• open file formats.
• be able to use your beloved editor
• still all features of a word processor
Ideally - text files. Actually it's pretty funny. Developers stick to text files while all kind of interactive apps exist for the rest of the world. But we choose independence. Still we would like to have cross references, images, etc..
Zim hits this sweet spot. You get an explorer-like desktop user interface to a wiki that is saved as plain text files on your harddrive. You get the wysiwyg thing, wiki cross-references and export functionality.
Now there's no reason not to document features, requirements and code
Actually I am a huge OneNote Fan and User. I guess that's why I like Zim, too...
At least you should

Since you are a developer, this (probably) is you wishlist
• everything should be checked into your VCS
• hence it shouldn't be a blob - a binary
• docs should be editable on multiple platforms
• by multiple users
• hence you would like to diff things
• open file formats.
• be able to use your beloved editor
• still all features of a word processor
Ideally - text files. Actually it's pretty funny. Developers stick to text files while all kind of interactive apps exist for the rest of the world. But we choose independence. Still we would like to have cross references, images, etc..
Zim hits this sweet spot. You get an explorer-like desktop user interface to a wiki that is saved as plain text files on your harddrive. You get the wysiwyg thing, wiki cross-references and export functionality.
Now there's no reason not to document features, requirements and code

Actually I am a huge OneNote Fan and User. I guess that's why I like Zim, too...
Fun/Little Gotchas with JDO Typesafe queries.... Mon, Apr 25. 2011
(Update: me happy ... DataNucleus Platform 3.0M4 released ...
"RDBMS : Support JDOQL String.equalsIgnoreCase()")
I have been hacking around with JDO Typesafe for a private litte project - it's pretty nice. Actually, JDO typesafe isn't part of the standard -yet.
I hit an issue with this query utilizing equalsIgnoreCase - I posted a message on the Datanucleus bulletin board
I guess it simply isn't implemented yet
I have to prepare for an exam therefore I didn't have time to take a closer look at the code.
But I did take a glance at the code. All in all, the code quality of the Data Nucleus platform seems to be lightyears ahead of hibernate. Not to mention one other word: bugs.
Another small problem
On the downside, it's pretty hard to get Spring 3.0X support for one particular feature- which is a convenience feature, not critical: org.springframework.orm.jdo.TransactionAwarePersistenceManagerFactoryProxy, mentioned on this page .
Basically, for JDO Typesafe you need a JDOPersistenceManager, not a PersistenceManager - it's a DataNucleus feature right now, not part of the standard yet. This sounds like a solveable problem, with pure OO you can use covariance - but here we are dealing with a proxy thats created with CGlib at runtime, and its built to mimic PersistenceManager (of course!). You can't get away with simply extending TransactionAwarePersistenceManagerFactoryProxy and overriding some methods. Therefore I don't use TransactionAwarePersistenceManagerFactoryProxy - Its convenience anyway.
"RDBMS : Support JDOQL String.equalsIgnoreCase()")
I have been hacking around with JDO Typesafe for a private litte project - it's pretty nice. Actually, JDO typesafe isn't part of the standard -yet.
I hit an issue with this query utilizing equalsIgnoreCase - I posted a message on the Datanucleus bulletin board
I guess it simply isn't implemented yet
I have to prepare for an exam therefore I didn't have time to take a closer look at the code.
But I did take a glance at the code. All in all, the code quality of the Data Nucleus platform seems to be lightyears ahead of hibernate. Not to mention one other word: bugs.
Another small problem
On the downside, it's pretty hard to get Spring 3.0X support for one particular feature- which is a convenience feature, not critical: org.springframework.orm.jdo.TransactionAwarePersistenceManagerFactoryProxy, mentioned on this page .
Basically, for JDO Typesafe you need a JDOPersistenceManager, not a PersistenceManager - it's a DataNucleus feature right now, not part of the standard yet. This sounds like a solveable problem, with pure OO you can use covariance - but here we are dealing with a proxy thats created with CGlib at runtime, and its built to mimic PersistenceManager (of course!). You can't get away with simply extending TransactionAwarePersistenceManagerFactoryProxy and overriding some methods. Therefore I don't use TransactionAwarePersistenceManagerFactoryProxy - Its convenience anyway.
Cleaning up my homepage Sat, Apr 2. 2011
Back to basic, a simple, plain and clean page.
Reason: Small is beautiful. And this blog is powerful enough for dynamic content.
Good bye, old page:

Reason: Small is beautiful. And this blog is powerful enough for dynamic content.
Good bye, old page:

Is the latest Version of Acrobat Reader X leaking GDI Resources? Thu, Feb 24. 2011
Just wo-n-d-er-ing

I also witnessed the classic symptoms of GDI resource exhaution: greyed-out Icons, Areas, Text etc.
I used RAII to solve that problem when I was doing Win32 C++ Programming.
Over and out
I also witnessed the classic symptoms of GDI resource exhaution: greyed-out Icons, Areas, Text etc.
I used RAII to solve that problem when I was doing Win32 C++ Programming.
Over and out
Link: A useful introduction to Awk Thu, Sep 30. 2010
Cool - Wireless healthcare from Qualcomm Mon, Sep 13. 2010
Video:
Via the good folks at netbooknews.de.
Its just funny because I did stuff like this myself a long,long time ago ... 2005 .. Dude time is moving on so fast.. Good old days
Via the good folks at netbooknews.de.
Its just funny because I did stuff like this myself a long,long time ago ... 2005 .. Dude time is moving on so fast.. Good old days

Weird Stuff :-) Sun, Sep 12. 2010
Okay, a real blog Fri, Aug 27. 2010
« previous page
(Page 1 of 1, totaling 14 entries)
next page »