JODB (Java Objects Database) for Android (3) with reduced TestIndex benchmark

While creating and running the TestIndex for JODB I've discovered a few things.

• Search operations are slow, and scale with the amount of records that is present in the database, so if it tok 2 seconds to get 10 distinct records with 100 records it will take 200 seconds with 10000, which are the number of records currently used in the TestIndex test. So large record-sets are not a good idea.
• It seems to use a very little amount of memory, but when searching goes as it goes, that doesn't really help that much when things take as much time as it does.
• As mentioned before it only supports primitives as indexes, so one really can't do a full TestIndex test.

As search operations was so costly I though I run a very reduced test with 100 records just to see how all of the other database management systems (DBMS).

As we see search operations are costly, my guess to why they are so costly is the way that JODB interacts with files on the system. I think that it do a lot of input/output (IO) operations when it does it's search operations. From what I gathered from looking in the source code it uses a FileChannel object to access its database files, which just reads or writes on a specific part of a file. So when doing a search operation it will read through the file until it finds the "part" (Record) that we are interested in. This would also explain why it use so little memory when it is running.

	Perst	SQLite	DB4O	BerkeleyDB	JODB
Insert	106	586	278	354	2183
Search	48	889	989	334	31021
Iterate	14	36	238	192	718
Delete	115	498	415	465	1011

TestIndex on Android developer phone1(ADP1) with Berkeley DB

After a long day of testing I got some results on the ADP1. BerkeleyDB does seem to like heap space more than the other database managament systems(DBMS), but it seems to be able to run ok. I think I'll need to think about ways to shrink the memory usage of the different tests.

Well here are the results:

	Perst-ADP1	SQLite-ADP1	DB4O-ADP1	BerkeleyDB- ADP1
insert	21150	600824	88059	113668
search	16176	107514	470561	74417
iterate	11747	6777	90657	62190
delete	37704	562522	155778	215499

TestIndex on Android emulator now also with BerkeleyDB

It's taken a bit longer time than I had expected to get some TestIndex results for BerkeleyDB on Android. A while back I had a running version up and running, but managed somehow to messup the serialization part, so I had to create a new version from scratch that does not use serialization. Instead I used BerkeleyDB's custom TuppleBinding which gives somewhat better performance than serilization. I hope to soon be able to post results using serilization as well, to compare the difference in performance. I will also do some research into the other database management systems that I have tested to see if they have similar optimalizations that can be used.

Results from the test in milliseconds.

	Perst-Emu	SQLite-Emu	DB4O-Emu	BerkeleyDB-Emu
insert	15158	72426	46648	55698
search	12480	100528	290541	37612
iterate	8365	5157	51335	29484
delete	25100	65498	93396	100455

TestIndex with DB4O(part two) on Android developer phone(ADP1)

I finished my ADP1 testing early this morning and here are the results.

Here are the results:

	Perst-ADP1	SQLite-ADP1	DB4O-ADP1
insert	20583	444980	83922
search	15982	103047	471148
iterate	11187	6578	88179
delete	28798	432789	156262

Here we also see that DB4O uses a long time on the search as it did on the Emulator, still don't have a good eplanation for this, I can only speculatate that it has something to do with the way it stores its records. But we allso see that SQLite uses a lot of time on the insert and delete part, which I think has to do with free memory, as my ADP1 still has quite a few application installed.

Android TestIndex extended with DB4O source

After a request I've uploaded the source for the Android TestIndex with DB4O here, so that anyone that wants can  have a look at it and perhaps come with sugestions for improvements.The original application can be found at McObjects site here To run the application you will need a copy of Perst, which can be found at McObjects site here. You will also need a copy of DB4O as well which can be found here and the Android development kit which can be downloaded here. .For those who doesn't want to go through the hazzle of setting up the eclipse project, and downloading all of the different components needed to compiling the test I've created an APK that can be found here.

TestIndex with DB4O(part one) on Android Emulator

When I did my last benchmark on the Android I got a question if I had tried out DB4O on Android, something I had not. So I had to try it out, to see how it was to use on the Android platform and compare it to SQLite and Perst. Using object oriented databases makes it alot easyer to develop applications as you can think "objects" all though out the development process, where you don't have to think about converting the results back into obejcts again. Using DB4O on android worked completely painless, and after some searching I found a compatible version of the TestIndex application for DB4O that I could integrate in the android. Below are the results for the emulator, I hope that i can test it on the ADP1 later today. How did I test ? I ran the test index application several times until I got some stable / reliable results.

Here is a graph of the results on the emulator. 

	Perst-Emulator	SQLite-Emulator	DB4O-Emulator
insert	11465	67420	36980
search	9601	90328	226624
iterate	6578	4504	39405
delete	18408	59998	71463

If we look at the results we see that the only thing that is really standing out is the search on the DB4O, I have no good explanation for this.

Android Perst Benchmark on ADP1(1.5) and emulator

I downloaded the new version of the Perst TestIndex bechmark application from McObject today, and it worked like a charm. When you start the test you can choose to do a benchmark of Perst or SQLite. The Perst part of the benchmark finnishes pretty quickly on the ADP1, but the SQLlite part takes 15 min to finish with(35MB memory free). I allso ran it on the Emulator to see how the results would differ.
The test application can be downloaded from McObject's site, and McObject have published a report containing their results and conclusion of running the Applicaiton on a G1 handset and on the emulator.
Bellow are my results from my execution of the test application on the Emulator and the ADP1:

The results of the Perst test on the emulator

The results of the SQLlite test on the emulator

 

Graph of results on emulator

	Perst Emulator	SQLite Emulator
insert	12455	94676
search	9836	98260
iterate	7240	4625
delete	30707	77907

The results of the Perst test on the ADP1 handset with around 7MB space free

The results of the SQLite test on the ADP1 phone with around 7MB space free, I just let it run and looked like it wasn't able to finnish even after more than one hour. So I tried again after some cleanup of various unused applications now with arond 14-20MB free space, still wouldn't finish after 15 minutes, so I shut it down again. Probably a good idea to have an application like "close everything" so the benchmark doen't run the background. So I tried another time now with 35MB free space. Now it finished after about 15 minutes or so, so you'll need some free space on the phone to be able to run the SQLite benchmark within a resonable amout of time.

The results of the Perst test on the ADP1 handset with around 35MB space free

The results of the SQLite test on the ADP1 handset with around 35MB space free

 

Graph of results on ADP with 35MB free space.

	Perst ADP1	SQLite ADP1
insert	22399	471281
search	17528	103076
iterate	11394	6620
delete	56299	445178