Starting Android applications from shell

Starting application from the shell can be helpful in many scenarios. On scenario is when you want to limit the resources that the application is going to use with ulimit.
1. Start the shell.

2. Start the ActivityManager(am) command-line tool with appropriate parameters.
The syntax for the am command-line tool is the following.
ssue am command. am command syntax is as follow :
am [start|instrument]
am start [-a <action>] [-d ] [-t <mime_type>]
[-c <category> [-c <category>] ...]
[-e <extra_key> <extra_value> [-e <extra_key> <extra_value> ...]
[-n <component>] [-D] [<uri>]
am instrument [-e <arg_name> <arg_value>] [-p <prof_file>]
[-w] <component>


3. Start the web browser with the following command.

am start -a android.intent.action.MAIN -n com.android.browser/.BrowserActivity

to start your own application just type:
am start -a android.intent.action.MAIN -n "Your Package"/."Your Activity"
If you hare unsure of what your package and main activity are look in your manifest file:

<manifest xmlns:android="http://schemas.android.com/apk/res/android"
 package="com.TordSoft.HelloWorld">
  <application android:icon="@drawable/icon">
   <android:name=".HelloWorld" android:label="@string/app_name">
    <intent-filter>
    <action android:value="android.intent.action.MAIN" />
    <category android:value="android.intent.category.LAUNCHER" /> </intent-filter>
  </activity></application>..</manifest>

NeoDatis Benchmark on Android

After doing a lot of different testing and trying a lot of different settings I finally decided to do some testing. I had some heap size memory when doing the complete TestIndex test with all of the 10000 records, NeoDatis like to store all of entries in it's internal cache, so I settled with 1000 for now, until I can figure out a way to get around the memory related issues.

Here are the result table:

	Perst	SQLite	DB4O	BerkeleyDB	NeoDatis
Insert	1159	6892	4168	4619	25022
Search	599	9481	18261	3405	18325
Iterate	120	290	4392	2889	11126
Delete	1659	6315	7084	5565	10156

It should be noted that the NeoDatis team releases new versions often, so if you have some problem, it might be fixed in a new version.

NeoDatis on Android intro

I've been playing around with NeoDatis a little while now so I thought I'd write a little intro post on it before I did the testing.
NeoDatis ODB like a lot of the other database management system (DBMS)'s that I've tested lately can be used as both an an embedded database engine or in client/server mode.
Size
The makers of NeoDatis say that ODB runtime is less than 550k, witch would make it quite suitable for use on android. In the some time soon I'll do some heap usage tests to see how much heap the different DBMS's use while running, just need a tool to do it, worst case I'll write it myself.
Features
It supports ACID transactions to insure data integrity,  it also has a automatic transaction recovery that runs in the startup sequence that will commit the records in case of a "fatal" shutdown due to an unexpected event such as a hardware failure.

Storage
NeoDatis ODB uses a single file to store all it's data:

• The Meta-model
• The objects
• The indexes

NB: How ever you should be aware of a few things before starting using the NeoDatis.

The file that you are going to use for the database, must not be a new empty file. If you give NeoDatis a empty file, it will crash and give you an exception looking something like this:

Version=1.9.17 , Build=632, Date=03-10-2009-05-38-18, Thread=Thread-8
NeoDatisError:276:End Of File reached - position = 0 : Length = 0
StackTrace: org.neodatis.odb.impl.core.layers.layer3.buffer.MultiBufferedIO.manageBufferForNewPosition(MultiBufferedIO.java:151)
org.neodatis.odb.impl.core.layers.layer3.buffer.MultiBufferedIO.setCurrentReadPosition(MultiBufferedIO.java:271)
org.neodatis.odb.core.layers.layer3.engine.FileSystemInterface.setReadPosition(FileSystemInterface.java:202)
org.neodatis.odb.impl.core.layers.layer3.engine.ObjectReader.readEncryptionFlag(ObjectReader.java:180)
org.neodatis.odb.impl.core.layers.layer3.engine.ObjectReader.readDatabaseHeader(ObjectReader.java:244)
org.neodatis.odb.core.layers.layer3.engine.AbstractStorageEngine.init(AbstractStorageEngine.java:206)
org.neodatis.odb.core.layers.layer3.engine.AbstractStorageEngine.(AbstractStorageEngine.java:171)
org.neodatis.odb.impl.core.layers.layer3.engine.LocalStorageEngine.(LocalStorageEngine.java:18)
org.neodatis.odb.impl.DefaultCoreProvider.getClientStorageEngine(DefaultCoreProvider.java:140)
org.neodatis.odb.impl.main.LocalODB.(LocalODB.java:49)
org.neodatis.odb.impl.main.LocalODB.getInstance(LocalODB.java:35)
org.neodatis.odb.ODBFactory.open(ODBFactory.java:72)

I've submitted it as a bug, and got a reply pretty fast so It will probably be fixed or documented soon. 

Thread safety
NeoDatis ODB supports usage in a multi-threaded environment.

Import/Export
ODB lets you export & import all data to XML.

Refactoring

An interesting feature that I haven't seen before. You can add and remove fields without problems, it allso support additional refactoring through through it's Object Explorer(Extra app) where you can rename classes and fields.

Licence
ODB is distributed under the LGPL license.

So this was a quick introduction, at the moment I'm working on a TestIndex for it which should be done soon. If you want to know more about NeoDatis I suggest their homepage which has pretty good documentation.

JODB (Java Objects Database) for Android (3) with reduced TestIndex benchmark

While creating and running the TestIndex for JODB I've discovered a few things.

• Search operations are slow, and scale with the amount of records that is present in the database, so if it tok 2 seconds to get 10 distinct records with 100 records it will take 200 seconds with 10000, which are the number of records currently used in the TestIndex test. So large record-sets are not a good idea.
• It seems to use a very little amount of memory, but when searching goes as it goes, that doesn't really help that much when things take as much time as it does.
• As mentioned before it only supports primitives as indexes, so one really can't do a full TestIndex test.

As search operations was so costly I though I run a very reduced test with 100 records just to see how all of the other database management systems (DBMS).

As we see search operations are costly, my guess to why they are so costly is the way that JODB interacts with files on the system. I think that it do a lot of input/output (IO) operations when it does it's search operations. From what I gathered from looking in the source code it uses a FileChannel object to access its database files, which just reads or writes on a specific part of a file. So when doing a search operation it will read through the file until it finds the "part" (Record) that we are interested in. This would also explain why it use so little memory when it is running.

	Perst	SQLite	DB4O	BerkeleyDB	JODB
Insert	106	586	278	354	2183
Search	48	889	989	334	31021
Iterate	14	36	238	192	718
Delete	115	498	415	465	1011

JODB (Java Objects Database) for Android (2)

After my previous post on JODB and after using it for a while I discovered a couple of things about JODB. JODB is picky of what it allows you to use as an Index. It will only allow you to use primitives as an index, so if you try to use something that is not a primitive, such as a string:

db.configureIndex(Record.class, "strKey", true); won't work.

You will get an exception looking something like this:

com.mobixess.jodb.core.JodbIOException: can't find description for primitive java.lang.String
com.mobixess.jodb.util.PrimitiveJavaTypesUtil.getEnumeratedType(PrimitiveJavaTypesUtil.java:141)
com.mobixess.jodb.core.index.JODBIndexingAgent.(JODBIndexingAgent.java:73)
com.mobixess.jodb.core.index.JODBIndexingRootAgent.addAgent(JODBIndexingRootAgent.java:148)
com.mobixess.jodb.core.index.JODBIndexingRootAgent.enableIndex(JODBIndexingRootAgent.java:72)
com.mobixess.jodb.core.JODBSessionContainer.configureIndex(JODBSessionContainer.java:130)
com.mobixess.jodb.core.JODBSessionContainer.configureIndex(JODBSessionContainer.java:102)

Another thing you must watch out for is when you are deleting your object from the database, you should(must at least when they contain other stuff than primitives) deactivate them, before, deleting them.

Record r = set.next();
db.deactivate(r, 1);
db.delete(r);

If you do not do this you will get an exception from deep within the Apache harmoney classes, in mmap method.

java.io.IOException
org.apache.harmony.luni.platform.OSMemory.mmap(OSMemory.java:617)
org.apache.harmony.luni.platform.PlatformAddressFactory.allocMapPlatformAddressFactory.java:107)
org.apache.harmony.nio.internal.FileChannelImpl.mapImpl(FileChannelImpl.java:201)
org.apache.harmony.nio.internal.ReadWriteFileChannel.map(ReadWriteFileChannel.java:53)
org.apache.harmony.nio.internal.FileChannelImpl.transferFrom(FileChannelImpl.java:422)
com.mobixess.jodb.core.io.buffers.RandomAccessFileTransactionBuffer.transferFrom(RandomAccessFileTransactionBuffer.java:156)
com.mobixess.jodb.core.io.JODBIOBase.applyTransaction0(JODBIOBase.java:501)
com.mobixess.jodb.core.io.JODBIOBase.applyTransaction0(JODBIOBase.java:433)
com.mobixess.jodb.core.io.JODBIOBase.applyTransaction(JODBIOBase.java:336)
com.mobixess.jodb.core.JODBSessionContainer.commit(JODBSessionContainer.java:202
com.mobixess.jodb.core.JODBSessionContainer.commit(JODBSessionContainer.java:191)

So now I got a working TestIndex for JODB, so soon I'll hopefully have some results on the emulator and on the ADP1. The biggest problem I found with using JODB is the lack of documentation, so if you're considering using it you should be aware that it's not that much help to get.

JODB (Java Objects Database) for Android

JODB (Java Objects Database) is an open source object-oriented database that is somewhat similar to db4objects.

License
JODB is completely free for personal and commercial use.

JODB

The developer of JODB sais that JODB is created with performance, size and scalability in mind. It uses soft reference based caches to effectively use the available ram but at the same time reduce the memory footprint as much as possible when memory is pricey, like it is on Android.

The developer say that the queries are optimized to the minimal of objects instantiations during the search so that garbage collector will have more time to 'rest' which is especially important for busy server applications (GC as is expensive and this is unlikely to change in near future).

Things you should watch out for

The Android version of JODB does not have the Client/Server functionality and can thus only be used as an embedded database. This really doesn't matter that much to me, as I was going to use it as an embedded database anyway, but it is something to take not of. The current version of JODB mini doesn't support running of native queries. The developer(s) say this will be corrected in future versions, but that's a long time ago so I'm not really sure that it will happen  . And at last the big one you must use com.mobixess.jodb.core.JodbMini as the entry point instead of com.mobixess.jodb.core.JODB.

So opening a database would look something like this:  JodbMini.open(databaseName) ;

Problems related to android

JODB have had Android support for a long time ( since 2007 ), but Android has changed a bit since they released their JODB Mini. RandomAccessFileBufferFactory.createBuffer in JODB, uses File.createTempFile() which creates a new file in the directory that is defined by the java.io.tmpdir system property. Using this method not a recommended practice in Android, as it will fill up data in a folder that might not be cleaned up, making the application fill up a lot of space, that we won't get back until it cleanup. It seems that Android( at least in 1.6 )  have solved this problem by setting the java.io.tmpdir to the SDCARD. But still we don't want to pollute our SDCARD directory with lots of temporary files, and in this case we want the files to be written and read fast so the SDCARD is not a good option. An other thing is that Android as of 1.6  Applications need explicit permission to be allowed to write to the SDCARD through the android.permission.WRITE_EXTERNAL_STORAGE permission.

To fix this you can:

1. Give the application write access to the SDCARD.
2. Set the "java.io.tmpdir" property to somewhere on the phone where the application can write, like in it's own "home" directory.

The nice thing about setting the "java.io.tmpdir" property property to a directory within the applications own directory is that when you remove the application from the device, all of the related data will also be removed.

I here set the temp directory to a directory with the application directory so I can clean up the temp files by just deleting the contents of that directory.

File dir = this.getDir("TMP", 0);
String newTempDir = dir.getAbsolutePath();
System.setProperty("java.io.tmpdir", newTempDir);

Documentation
JODB help page and javadoc is the only documentation that I found on JODB. JODB mini can be fetched here.

Signing Android applications

Signing
When you're developing Applications for Android you need to sign them. This must be done with a digital certificate that is owned by you, the
application developer.
Why sign?
There are several reasons for signing your applications; the most important reason is that Android needs to know who made the application and what applications an application can trust. Therefore the application must be signed with "your" private key.
Another reason as of the 1.6 SDK is to align your resources in your application packages. As mention in the previous post about Zipaling, the packages should be aligned after the application is signed.There are a lot of other aspects of signing in Android than one would think.
Application upgrading, Android won't allow application to be upgraded unless signed with same certificate the applications are signed with the same key.
Application modularity, Android allows applications that are signed with the same certificate to run in the same processes. This way you can deploy your application as modules.
Code/data sharing through permissions - the Android system provides a signature-based permissions enforcement, so that an application can expose functionality to another application that is signed with same certificate. So by signing multiple applications with the same certificate and using signature-base permissions
checks, your application can share code and data in a secure manner.
Validity
25 years or more is recommended. After the key's validity is expired, users will no longer be able, seamlessly upgrade to new versions of you application. When choosing the validity of your there are several aspects you should consider. As it must exceed all of the applications, including depending applications (modules) added in the future.
Marked
If you plan to put your applications on the Android Marked your validity period must be after 22. October 2033. This date is enforced by the Android Market Server and is put into place to ensure that users can seamlessly upgrade their applications.
When to sign
When your application is ready to be released to the "market", you should check that it is signed for "release", as the debug keys that are generated by the SDK won't do.
How to sign
There are two ways of signing your applications. Manually and from the build tool that you might be using environment in Android this will be Ant or ADT.
Manual approach:
The manual approach where you use the Keytool and Jarsigner from the command-line. These tools are bundled with the java development kit (JDK). When doing this you first compile your application as an unsigned .apk. After this you must a create keystore/key if you don’t allready has one. You can create the keystore/key by running keytool.
Creating keystore/key with Keytool:
The line below will create a keystore/key which you can use to sign your packages. The keystore file is a password protected file that stores keys and certificates that you use in the signing of you application.


keytool -genkey -v -keystore my-release-key.keystore -alias alias_name -keyalg RSA -validity 10000

When you run keytool with the following parameters you it will ask you about information that it needs to create your keystore.

Enter keystore password:
Re-enter new password:
What is your first and last name?
[Unknown]: Jon Doe
What is the name of your organizational unit?
[Unknown]: R&D
What is the name of your organization?
[Unknown]: Corparation
What is the name of your City or Locality?
[Unknown]: Kongsberg
What is the name of your State or Province?
[Unknown]: Buskerud
What is the two-letter country code for this unit?
[Unknown]: NO
Is CN=Jon Doe, OU=R&D, O=Corparation, L=Kongsberg, ST=Buskerud, C=NO correct?
[no]: yes
Generating 1á024 bit RSA key pair and self-signed certificate (SHA1withRSA) with a validity of 10á000 days for: CN=Jon Doe, OU=R&D, O=Corparation, L=Kongsberg, ST=Buskerud, C=NO
Enter key password for alias_name
(RETURN if same as keystore password):
[Storing my-release-key.keystore]

-genkey:tells keytool to create a key pair (public and private)
-v: Enable verbose output.
-keystore: The location of the keystore file, where the information is going to be stored.
-validity: The validity of the key, given in days.
-alias: The alias of your entry inside the keystore file
-keyalg:specifies the algorithm to be used to generate the key pair.

The keystore you just have made, you should put somewhere safe as this will be your identification as an Application developer. If someone gets a hold of your keystore they can sign applications with your identification.

Signing application with Jarsigner:
To sign your application with the key you created with the line above, you can the jarsigner tool.
jarsigner -verbose -keystore my-release-key.keystore my_application.apk alias_name

Enter Passphrase for keystore:
adding: META-INF/ALIAS_NA.SF
adding: META-INF/ALIAS_NA.RSA
signing: res/drawable/icon.png
signing: res/layout/main.xml
signing: res/layout/other.xml
signing: AndroidManifest.xml
signing: resources.arsc
signing: classes.dex

Once your application is signed, don't forget to run zipalign on the APK for additional optimization.

Using ADT
The Android SDK tools have options to assist you in signing your applications when debugging, and when you are ready for a release. Both the Android Development Tool (ADT) plug-in for Eclipse and ANT support debug and release mode signing, not going to go in to ANT signing now.
Debug mode signing

When you are developing and testing, you typically compile in debug mode. In this mode the build tools automatically use the Keytool from the JDK, creates a keystore and a key with a known key and alias. Since the password is known, the tools don't prompt you for the keystore/key password each time you compile. The self-signed certificate that is used to sign your application in default mode, have an expiration date of 365 days after its creation.So after a year you started signing in debug mode you might encounter:
debug:
[echo] Packaging bin/samples-debug.apk, and signing it with a debug key...
[exec] Debug Certificate expired on 8/4/08 3:43 PM
To remedy this you can delete the debug.keystore in the default storage location for AVD.

ADT release signing
Luckily the ADT can do all this for you in the ADT Export Wizard. It can even generate the key for you. And as mentioned in the previous post it will even do the aligning for you.

Creating a new keystore

The first thing you do is find the Export Signed Application Package wizard.

Then it will ask you which project you want to export.

The next step you will be asked to either use an existing keystore, or you can create a new one, we'll create a new one first.

At this stage you asked to enter an alias and password for this alias and validity for the signing, as well as the First and last name of the signer.

Now you just got to tell it where to export the signed package, and you're done.
Using an already existing keystore
If you already have a keystore that you want to use you select the use existing keystore.

To use the existing keystore you must select its location and enter the password you entered when creating it.

Now you can use the key that you that you defined when creating the key.

And now you're done, all signed up and ready to go.

For more info on the keytool application here, signing in Android documentation (recommended)

Aligning resources in Android Packages( using zipalign)

Zipaling: 
The zipalign tool was added to the SDK in version 1.6. Zipalign optimizes the way the application is packed, this optimizes the way android interacts with your application and may increase the performance of an applciation. 

Why Zipaling:
 The purpose of zipalign is to ensure that all uncompressed data starts with a particular alignment relative to the start of the application file. Specifically, it causes all uncompressed data within the .apk such as images or raw files to be aligned on 4-byte boundaries. This allows for all portions of the apk file to be accessed directly by memory-mapping them with a function known as mmap(), rather than copying all of the data out of them. The mmap() function establishes a mapping between a process' address space and a file, shared memory or typed memory object, in this case the .apk file. The result of this is that you can get a reduction in the amount of RAM that your applications consume when they run. If the resources are not aligned, Android has to explicitly read them, which is slower and also consume more memory.

For end users using unaligned application can make the Home application and the unaligned application launch slower than they otherwise should. The worst case, installing several applications with unaligned resource will increase the memory pressure, and cause the system to suffer by having constantly having to start and kill processes. This will have the user end up with a slow device with poor battery life.

Zipalign and signing:
 Zipalign must be run after the application, this is because the signing the application after aligning will disrupt the package alignment. As a result of this all of the "automatic integrated alignment" tools do not align packages until they can sign it. 

How do I use it?   
ADT:

If you are developing with Eclipse and using the ADT plug-in, the zipalign tool is integrated into the "Export Wizard". When you select "Export signed application package", ADT signs and automatically runs zipalign on the exported package. If you choose "Export unsigned application package" it won't be aligned due to reasons mentioned above, so if you want to align it, you must sign and align it manually. If you're wondering where the "Export" functions is, it can be found at the first page of the AndroidManifest.xml editor as well as in the "Android tools"  menu selection in the projects context menu. 

Ant:

With the Ant build script that targets the newest Android API (1.6, API Level 4) have the ability to align the application packages. Older ant build scripts that target older versions of the Android platform will not be aligned and will need to be aligned manually. Debug packages that are built with Ant with 1.6 as a target are aligned and signed by default. Release packages are aligned automatically if Ant has enough information to sign the packages, this cause the aligning has to happen after signing. So in order to have the application signed, and thereby align them, Ant needs to know the location of the keystore and the name of the key in the build.properties. The name of the properties are key.store and key.alias, if these properties are present the signing tool will ask for the store/key password during the build, and after this the tool will sign and align the apk file. 

Manually:
 To manually align a package, you'll find the zipalign tool in the in the "tools" folder in the Android 1.6 SDK. It can be used on any application package irrelevant of target version. As stated previously this should be done after the package is signed. To use the tool you will need to use the following syntax: 

zipalign [-f] [-v] infile.apk outfile.apk 
-f : overwrite existing outfile.zip
-v : verbose output
-c : confirm the alignment of the given file, in other words verify the alignment.
Alignment must always be 4, as Android uses 32 bit alignment, if it is anything else this will be as if it was not aligned.

So:

zipalign -v 4 source.apk destination.apk

Will align your android package. 

Verification: 
You can verify that your package has been aligned properly on can: 

zipalign -c -v 4 application.apk 

It is strongly recommended to align your Android applications as it can improve the performance and memory usage, and as it isn't that hard to do I don't see any good reason not to do it.

Happy aligning.

Android documentation on Zipaling.

Upgrading Android Developer phone 1(ADP1) from 1.5 to 1.6

Discovering that HTC had posted new firmware images on their site, I felt like trying them out to see what new features it had in store, and if it improved performance in any way. However after just after downloading the images, discovered that they were wrong and not signed. And that to get the unsigned images to work one would have to wipe the userdata partition, as some google applications failed to start when opening when "signed" userdata was present. Well a bit later the proper images was posted, I downloaded them happy that I did not have to wipe my userdata.
To update the ADP1 there are two strategies: 1. The recovery image way and the fastboot way, I choose to use the fastboot strategy as I did the Recovery-image strategy when I upgraded to to 1.5. So when the image was all downloaded I booted the device in "fastboot" mode. I assume it is called fastboot mode as it boots right away, if it does not boot right away you probabily did something wrong. For my phone i booted up while holding the camera button down, but it might be something else on other phones on HTC's site it say you have to hold the back button down, as one might imagine that didn't work for me. When the phone is in fast boot mode, it should look something like this.

Some happy looking skateboarding Android characters and in the middle there should say serial something, press the back button until it says FASTBOOT.

 If you want to exit the fast boot screen you have to hold the MENU+SEND+END buttons down, where Send is the CALL button on the ADP1 and END is the power button(end call).
Well now you have to get the fastboot application, it can be found at HTC's site here.
The fastboot application should be downloaded to the \tools directory, as it is dependent on AdbWinApi.dll and will not start without it. If you want to run the fastboot application from another directory you will need to add the \tools directory to your PATH enviroment variable.

now you have to type something like this: 
fastboot.exe update signed-dream_devphone_userdebug-img-14721.zip
archive does not contain 'boot.sig'
archive does not contain 'recovery.sig'
archive does not contain 'system.sig'
'waiting device'

If you do not get past the waiting for device part there is something wrong with the connection between the phone and the fastboot application. This may be that it is not connected, the driver migth not be installed or not installed properly, and the phone might not be in fastboot mode.
 When successfull there should be some output in the commandline looking something like this:
--------------------------------------------
Bootloader Version...: 0.95.3000
Baseband Version.....: 2.22.19.26I
Serial Number........: HT91RLZ00368
--------------------------------------------
checking product... OKAY
checking serialno... OKAY
checking version-bootloader... OKAY
checking version-baseband... OKAY
checking version-cpld... OKAY
sending 'boot' (1838 KB)... OKAY
writing 'boot'... OKAY
sending 'recovery' (2088 KB)... OKAY
writing 'recovery'... OKAY
sending 'system' (58570 KB)... OKAY
writing 'system'... OKAY
rebooting...
When rebooting it might take a bit longer than usual. And then you're done, if you look at the Android market it should look something like this.

TestIndex on Android developer phone1(ADP1) with Berkeley DB

After a long day of testing I got some results on the ADP1. BerkeleyDB does seem to like heap space more than the other database managament systems(DBMS), but it seems to be able to run ok. I think I'll need to think about ways to shrink the memory usage of the different tests.

Well here are the results:

	Perst-ADP1	SQLite-ADP1	DB4O-ADP1	BerkeleyDB- ADP1
insert	21150	600824	88059	113668
search	16176	107514	470561	74417
iterate	11747	6777	90657	62190
delete	37704	562522	155778	215499

TestIndex on Android emulator now also with BerkeleyDB

It's taken a bit longer time than I had expected to get some TestIndex results for BerkeleyDB on Android. A while back I had a running version up and running, but managed somehow to messup the serialization part, so I had to create a new version from scratch that does not use serialization. Instead I used BerkeleyDB's custom TuppleBinding which gives somewhat better performance than serilization. I hope to soon be able to post results using serilization as well, to compare the difference in performance. I will also do some research into the other database management systems that I have tested to see if they have similar optimalizations that can be used.

Results from the test in milliseconds.

	Perst-Emu	SQLite-Emu	DB4O-Emu	BerkeleyDB-Emu
insert	15158	72426	46648	55698
search	12480	100528	290541	37612
iterate	8365	5157	51335	29484
delete	25100	65498	93396	100455

BerkeleyDB for Java on Android

After having tried a couple of databases on Android I thought it would be nice to add another one to the bunch. So after a little searching I found BerkeleyDB for Java, downloaded it and tried to run it on the Android emulator. To get it to work with Android there are some instruction in a HowToAndroid.hmtl file that's bundled with the documentation. It describes the required modifications that are needed to do to the source in order to get it to work. There are a couple of folders that need to be deleted and some extra java files that need to be created, but all-in-all it's quite manageable. After doing the required modification to the source of BerkeleyDB I tried to run a sample small sample that I had for BerkeleyDB and ended up with this exception: "com.sleepycat.je.log.LogException: (JE 3.3.87) java.io.FileNotFoundException: /data/je/je.lck". I concluded that this had to do with BerkeleyDB's need for a folder to store it's enviroment(database files, logs). BerkeleyDB requires you to create this folder, or else you will need end up with the lovely exception above. In the how to there is some instructions on how to create the environment( folders ) for BerkeleyDB, "adb shell mkdir /data/je" and removal of the folder by "adb shell rm /data/je/*". This is a bit quirky, and leaves you with the responsibility of cleaning after you application, as the folder is directly placed in the data folder and not in the "home" folder of your application. So I'd recommend using the Activity.getDir("folderName", 0) or something similar to create the directory for BerkeleyDB, so that it will be cleaned up when the application is removed. So sum it all up I got it up and running and in a couple of days I'll post some results from the benchmark.

Memory limit in Android

It has come to my attention that Android applications have a heap limit of 16MB. Normally this won't pose a problems to ordenary applications, but I have found that it isn't as hard as one would think to reach this limit, especially when you are doing resource intensive things like testing. When the heap size reaches above 16MB you will get an java.lang.OutOfMemoryError exeption.
It is posible to increase this limit, but it seem to require direct intervension into the the Android source. By defining a compile flag, CUSTOM_RUNTIME_HEAP_MAX one can set the parameter that defines the max heap size of dalvik. This flag will then be used by the AndroidRuntime::start method in the frameworks/base/core/jni/AndroidRuntime.cpp to define the -Xmx parmater that sets the max heap size of dalvik.

ifdef CUSTOM_RUNTIME_HEAP_MAX
#define __make_max_heap_opt(val) #val
#define _make_max_heap_opt(val) "-Xmx" __make_max_heap_opt(val)
opt.optionString = _make_max_heap_opt(CUSTOM_RUNTIME_HEAP_MAX);
#undef __make_max_heap_opt
#undef _make_max_heap_opt
#else
/* limit memory use to 16MB */
opt.optionString = "-Xmx16m";
#endif

So if you define CUSTOM_RUNTIME_HEAP_MAX with a value like "20m" you will have increased the maximum heap size to 20MB.

Stopping Android from sleeping

Yesterday I published the source for the extended Android TestIndex application. The primary difference from that my extension and the original is the introduction of wakelocks. The reason for that is that when tests started to use longer and longer time, I felt a need to be able to do the tests without having to hover over the phone and push "menu" to give it input, and thus stopping it from going into sleep mode. Another thing that was starting to happen was that if the test took really long time and I just left the phone running it would restart the application, and thus removing any results that it may or may not have produced. So I started to digg into a way  to keep the CPU from going into sleepmode while the tests was running. After a short while I found the PowerManager sections in the Android documentation, which talks about wakelocks. The wakelocks are aquired from the PowerManager, and controls the different sleep stages that the phone goes into. I choose to use the partial wakelock wich just keeps CPU from sleeping, but allows the screen and keyboard to go into various sleep stages. Below I've pasted the table from the Android documentation that tells what the different wakelocks do.

Flag Value	CPU	Screen	Keyboard
PARTIAL_WAKE_LOCK	On*	Off	Off
SCREEN_DIM_WAKE_LOCK	On	Dim	Off
SCREEN_BRIGHT_WAKE_LOCK	On	Bright	Off
FULL_WAKE_LOCK	On	Bright	Bright

So to solve the problem you just:

PowerManager pm = (PowerManager) getSystemService(Context.POWER_SERVICE);
 PowerManager.WakeLock wl = pm.newWakeLock(PowerManager.PARTIAL_WAKE_LOCK, "MyWakelock");
 wl.acquire();
  // test code 
 wl.release();

TestIndex with DB4O(part two) on Android developer phone(ADP1)

I finished my ADP1 testing early this morning and here are the results.

Here are the results:

	Perst-ADP1	SQLite-ADP1	DB4O-ADP1
insert	20583	444980	83922
search	15982	103047	471148
iterate	11187	6578	88179
delete	28798	432789	156262

Here we also see that DB4O uses a long time on the search as it did on the Emulator, still don't have a good eplanation for this, I can only speculatate that it has something to do with the way it stores its records. But we allso see that SQLite uses a lot of time on the insert and delete part, which I think has to do with free memory, as my ADP1 still has quite a few application installed.

Android TestIndex extended with DB4O source

After a request I've uploaded the source for the Android TestIndex with DB4O here, so that anyone that wants can  have a look at it and perhaps come with sugestions for improvements.The original application can be found at McObjects site here To run the application you will need a copy of Perst, which can be found at McObjects site here. You will also need a copy of DB4O as well which can be found here and the Android development kit which can be downloaded here. .For those who doesn't want to go through the hazzle of setting up the eclipse project, and downloading all of the different components needed to compiling the test I've created an APK that can be found here.

TestIndex with DB4O(part one) on Android Emulator

When I did my last benchmark on the Android I got a question if I had tried out DB4O on Android, something I had not. So I had to try it out, to see how it was to use on the Android platform and compare it to SQLite and Perst. Using object oriented databases makes it alot easyer to develop applications as you can think "objects" all though out the development process, where you don't have to think about converting the results back into obejcts again. Using DB4O on android worked completely painless, and after some searching I found a compatible version of the TestIndex application for DB4O that I could integrate in the android. Below are the results for the emulator, I hope that i can test it on the ADP1 later today. How did I test ? I ran the test index application several times until I got some stable / reliable results.

Here is a graph of the results on the emulator. 

	Perst-Emulator	SQLite-Emulator	DB4O-Emulator
insert	11465	67420	36980
search	9601	90328	226624
iterate	6578	4504	39405
delete	18408	59998	71463

If we look at the results we see that the only thing that is really standing out is the search on the DB4O, I have no good explanation for this.

Android Perst Benchmark on ADP1(1.5) and emulator

I downloaded the new version of the Perst TestIndex bechmark application from McObject today, and it worked like a charm. When you start the test you can choose to do a benchmark of Perst or SQLite. The Perst part of the benchmark finnishes pretty quickly on the ADP1, but the SQLlite part takes 15 min to finish with(35MB memory free). I allso ran it on the Emulator to see how the results would differ.
The test application can be downloaded from McObject's site, and McObject have published a report containing their results and conclusion of running the Applicaiton on a G1 handset and on the emulator.
Bellow are my results from my execution of the test application on the Emulator and the ADP1:

The results of the Perst test on the emulator

The results of the SQLlite test on the emulator

 

Graph of results on emulator

	Perst Emulator	SQLite Emulator
insert	12455	94676
search	9836	98260
iterate	7240	4625
delete	30707	77907

The results of the Perst test on the ADP1 handset with around 7MB space free

The results of the SQLite test on the ADP1 phone with around 7MB space free, I just let it run and looked like it wasn't able to finnish even after more than one hour. So I tried again after some cleanup of various unused applications now with arond 14-20MB free space, still wouldn't finish after 15 minutes, so I shut it down again. Probably a good idea to have an application like "close everything" so the benchmark doen't run the background. So I tried another time now with 35MB free space. Now it finished after about 15 minutes or so, so you'll need some free space on the phone to be able to run the SQLite benchmark within a resonable amout of time.

The results of the Perst test on the ADP1 handset with around 35MB space free

The results of the SQLite test on the ADP1 handset with around 35MB space free

 

Graph of results on ADP with 35MB free space.

	Perst ADP1	SQLite ADP1
insert	22399	471281
search	17528	103076
iterate	11394	6620
delete	56299	445178