Wednesday, September 30, 2009

TestIndex on Android developer phone1(ADP1) with Berkeley DB

After a long day of testing I got some results on the ADP1. BerkeleyDB does seem to like heap space more than the other database managament systems(DBMS), but it seems to be able to run ok. I think I'll need to think about ways to shrink the memory usage of the different tests.

Well here are the results:




Perst-ADP1
SQLite-ADP1
DB4O-ADP1
BerkeleyDB-
ADP1
insert
21150
600824
88059
113668
search
16176
107514
470561
74417
iterate
11747
6777
90657
62190
delete
37704
562522
155778
215499

TestIndex on Android emulator now also with BerkeleyDB

It's taken a bit longer time than I had expected to get some TestIndex results for BerkeleyDB on Android. A while back I had a running version up and running, but managed somehow to messup the serialization part, so I had to create a new version from scratch that does not use serialization. Instead I used BerkeleyDB's custom TuppleBinding which gives somewhat better performance than serilization. I hope to soon be able to post results using serilization as well, to compare the difference in performance. I will also do some research into the other database management systems that I have tested to see if they have similar optimalizations that can be used.


Results from the test in milliseconds.


Perst-Emu
SQLite-Emu
DB4O-Emu
BerkeleyDB-Emu
insert
15158
72426
46648
55698
search
12480
100528
290541
37612
iterate
8365
5157
51335
29484
delete
25100
65498
93396
100455





Sunday, September 27, 2009

BerkeleyDB for Java on Android

After having tried a couple of databases on Android I thought it would be nice to add another one to the bunch. So after a little searching I found BerkeleyDB for Java, downloaded it and tried to run it on the Android emulator. To get it to work with Android there are some instruction in a HowToAndroid.hmtl file that's bundled with the documentation. It describes the required modifications that are needed to do to the source in order to get it to work. There are a couple of folders that need to be deleted and some extra java files that need to be created, but all-in-all it's quite manageable. After doing the required modification to the source of BerkeleyDB I tried to run a sample small sample that I had for BerkeleyDB and ended up with this exception: "com.sleepycat.je.log.LogException: (JE 3.3.87) java.io.FileNotFoundException: /data/je/je.lck". I concluded that this had to do with BerkeleyDB's need for a folder to store it's enviroment(database files, logs). BerkeleyDB requires you to create this folder, or else you will need end up with the lovely exception above. In the how to there is some instructions on how to create the environment( folders ) for BerkeleyDB, "adb shell mkdir /data/je" and removal of the folder by "adb shell rm /data/je/*". This is a bit quirky, and leaves you with the responsibility of cleaning after you application, as the folder is directly placed in the data folder and not in the "home" folder of your application. So I'd recommend using the Activity.getDir("folderName", 0) or something similar to create the directory for BerkeleyDB, so that it will be cleaned up when the application is removed. So sum it all up I got it up and running and in a couple of days I'll post some results from the benchmark.

Memory limit in Android

It has come to my attention that Android applications have a heap limit of 16MB. Normally this won't pose a problems to ordenary applications, but I have found that it isn't as hard as one would think to reach this limit, especially when you are doing resource intensive things like testing. When the heap size reaches above 16MB you will get an java.lang.OutOfMemoryError exeption.
It is posible to increase this limit, but it seem to require direct intervension into the the Android source. By defining a compile flag, CUSTOM_RUNTIME_HEAP_MAX one can set the parameter that defines the max heap size of dalvik. This flag will then be used by the AndroidRuntime::start method in the frameworks/base/core/jni/AndroidRuntime.cpp to define the -Xmx parmater that sets the max heap size of dalvik.

ifdef CUSTOM_RUNTIME_HEAP_MAX
#define __make_max_heap_opt(val) #val
#define _make_max_heap_opt(val) "-Xmx" __make_max_heap_opt(val)
opt.optionString = _make_max_heap_opt(CUSTOM_RUNTIME_HEAP_MAX);
#undef __make_max_heap_opt
#undef _make_max_heap_opt
#else
/* limit memory use to 16MB */
opt.optionString = "-Xmx16m";
#endif

So if you define CUSTOM_RUNTIME_HEAP_MAX with a value like "20m" you will have increased the maximum heap size to 20MB.

Saturday, September 26, 2009

Stopping Android from sleeping

Yesterday I published the source for the extended Android TestIndex application. The primary difference from that my extension and the original is the introduction of wakelocks. The reason for that is that when tests started to use longer and longer time, I felt a need to be able to do the tests without having to hover over the phone and push "menu" to give it input, and thus stopping it from going into sleep mode. Another thing that was starting to happen was that if the test took really long time and I just left the phone running it would restart the application, and thus removing any results that it may or may not have produced. So I started to digg into a way  to keep the CPU from going into sleepmode while the tests was running. After a short while I found the PowerManager sections in the Android documentation, which talks about wakelocks. The wakelocks are aquired from the PowerManager, and controls the different sleep stages that the phone goes into. I choose to use the partial wakelock wich just keeps CPU from sleeping, but allows the screen and keyboard to go into various sleep stages. Below I've pasted the table from the Android documentation that tells what the different wakelocks do.
Flag ValueCPUScreen Keyboard
PARTIAL_WAKE_LOCKOn*
OffOff
SCREEN_DIM_WAKE_LOCK
OnDimOff
SCREEN_BRIGHT_WAKE_LOCK
OnBrightOff
FULL_WAKE_LOCKOnBrightBright
 
So to solve the problem you just:
PowerManager pm = (PowerManager) getSystemService(Context.POWER_SERVICE);
 PowerManager.WakeLock wl = pm.newWakeLock(PowerManager.PARTIAL_WAKE_LOCK, "MyWakelock");
 wl.acquire();
  // test code 
 wl.release();
 

Friday, September 25, 2009

TestIndex with DB4O(part two) on Android developer phone(ADP1)

I finished my ADP1 testing early this morning and here are the results.

Here are the results:


Perst-ADP1
SQLite-ADP1
DB4O-ADP1
insert
20583
444980
83922
search
15982
103047
471148
iterate
11187
6578
88179
delete
28798
432789
156262



Here we also see that DB4O uses a long time on the search as it did on the Emulator, still don't have a good eplanation for this, I can only speculatate that it has something to do with the way it stores its records. But we allso see that SQLite uses a lot of time on the insert and delete part, which I think has to do with free memory, as my ADP1 still has quite a few application installed.

Android TestIndex extended with DB4O source

After a request I've uploaded the source for the Android TestIndex with DB4O here, so that anyone that wants can  have a look at it and perhaps come with sugestions for improvements.The original application can be found at McObjects site here To run the application you will need a copy of Perst, which can be found at McObjects site here. You will also need a copy of DB4O as well which can be found here and the Android development kit which can be downloaded here. .For those who doesn't want to go through the hazzle of setting up the eclipse project, and downloading all of the different components needed to compiling the test I've created an APK that can be found here.

Thursday, September 24, 2009

TestIndex with DB4O(part one) on Android Emulator

When I did my last benchmark on the Android I got a question if I had tried out DB4O on Android, something I had not. So I had to try it out, to see how it was to use on the Android platform and compare it to SQLite and Perst. Using object oriented databases makes it alot easyer to develop applications as you can think "objects" all though out the development process, where you don't have to think about converting the results back into obejcts again. Using DB4O on android worked completely painless, and after some searching I found a compatible version of the TestIndex application for DB4O that I could integrate in the android. Below are the results for the emulator, I hope that i can test it on the ADP1 later today. How did I test ? I ran the test index application several times until I got some stable / reliable results.

Here is a graph of the results on the emulator. 




Perst-Emulator
SQLite-Emulator
DB4O-Emulator
insert
11465
67420
36980
search
9601
90328
226624
iterate
6578
4504
39405
delete
18408
59998
71463


If we look at the results we see that the only thing that is really standing out is the search on the DB4O, I have no good explanation for this.

Tuesday, September 15, 2009

Android Perst Benchmark on ADP1(1.5) and emulator

I downloaded the new version of the Perst TestIndex bechmark application from McObject today, and it worked like a charm. When you start the test you can choose to do a benchmark of Perst or SQLite. The Perst part of the benchmark finnishes pretty quickly on the ADP1, but the SQLlite part takes 15 min to finish with(35MB memory free). I allso ran it on the Emulator to see how the results would differ.
The test application can be downloaded from McObject's site, and McObject have published a report containing their results and conclusion of running the Applicaiton on a G1 handset and on the emulator.
Bellow are my results from my execution of the test application on the Emulator and the ADP1:
The results of the Perst test on the emulator
The results of the SQLlite test on the emulator
 
Graph of results on emulator



Perst Emulator SQLite Emulator
insert 12455 94676
search 9836 98260
iterate 7240 4625
delete 30707 77907

The results of the Perst test on the ADP1 handset with around 7MB space free
The results of the SQLite test on the ADP1 phone with around 7MB space free, I just let it run and looked like it wasn't able to finnish even after more than one hour. So I tried again after some cleanup of various unused applications now with arond 14-20MB free space, still wouldn't finish after 15 minutes, so I shut it down again. Probably a good idea to have an application like "close everything" so the benchmark doen't run the background. So I tried another time now with 35MB free space. Now it finished after about 15 minutes or so, so you'll need some free space on the phone to be able to run the SQLite benchmark within a resonable amout of time.
The results of the Perst test on the ADP1 handset with around 35MB space free
The results of the SQLite test on the ADP1 handset with around 35MB space free

 
Graph of results on ADP with 35MB free space.


Perst ADP1 SQLite ADP1
insert 22399 471281
search 17528 103076
iterate 11394 6620
delete 56299 445178