Xamarin Android Bitmaps Part 1 – Introduction to Bitmaps

The basics of bitmaps in Xamarin.Android, this will be a springboard into future posts about bitmaps in Xamarin.Android.

Advertisements

Why read about this?

Jump to “What is a bitmap?” to skip the intro.

One of the most annoying things I’ve had to deal with yet in Xamarin, specifically on the Android side of things, is bitmaps. Oddly enough, I rarely tend to see developers post about bitmap related issues, and I think this is either due to the fact that their apps aren’t very complicated, or because they aren’t popping the hood and actually seeing what their apps memory is doing, or the unlikely third possibility, they are accidentally managing their bitmaps perfectly (or perhaps I just suck, and everyone else doesn’t).

Take a project we worked on a while back for example, I was charged with rolling out the Android version of the app, and this was our companies first Xamarin project. After about 2 months I got this thing nearly complete, and my boss sent the app to his wife to test (yep, we didn’t have a company Android device yet). She was saying the app was crashing frequently, and so I popped open the Android device monitor to see what was going on. We had a list view, which had bitmaps; when the listview was clicked, a details page off to the side loaded details about the item clicked. Each time you clicked an item and backed out to the listview, the heap was growing a whopping 10mb each time, indicating a pretty serious memory leak. So you can imagine, this will pretty quickly kill an app running on even the most modern hardware.

To make a long story short, there were two issues. The ultra-high res photos our client sent us were almost 2000×2000 pixels, now imagine we’re loading 20-30 of these into the list view, and then loading 1 into our details screen, and you can see the recipe for disaster. To remedy this, I simply scaled the bitmaps down on load, simple enough. The second issue was basically that in Xamarin, bitmaps exist in both the Mono runtime, and the native Android runtime. This complicates memory management significantly. In total, it took about 40 hours and a ton of trial and error to track down and fix all of the memory issues with these bitmaps because I literally couldn’t find any information on how to manage bitmap memory in Xamarin.

How did things go so wrong for me? Well, I had no real understanding of how bitmaps worked in Xamarin, and from what I’ve discussed with other developers, this seems to be a normal trend, not an outlier. I could at least say I understood bitmaps conceptually as I’ve done some lower level C code where I was directly manipulating bitmaps on per-pixel basis, but not everyone has even that much experience with them. My guess is a sizable chunk of developers experience with bitmaps is loading them into an image tag in html, so I figured a good starting point for this little series of posts would be to actually discuss bitmaps and their interactions in Xamarin.

What is a bitmap?

Probably the best place to start is to explain what a bitmap is, and frankly they are pretty simple. The concept of a “map” in programming is nothing new, maps are basically a collection of data that get interpreted in a certain way, at least that’s how I define it, so don’t go using that in academia. Essentially, bitmaps are arrays where each element in the array has some data associated with it. In the case of a bitmap, each element in the array represents a pixel on the screen, and it’s bit depth. Bit depth is a generic term for the amount of bits used to represent something, in the case of bitmaps, bit depth refers to the color depth, or the number of pixels used to represent a color (higher bit depth = more potential colors). You can actually see the effects of color depth on a bitmap by simply saving a bitmap as a lower quality bitmap (for example, 8 bit, 16 bit, 32 bit, etc.).

Bitmaps as operating system resources

Okay so that’s it right? Bitmaps are just arrays of data that represent pixels and their colors? Yes…but no. For starters, bitmaps have more to them than just the aforementioned things, but what those other things are will depend on the operating system. Based on the operating system, a bitmap gets handled differently, but for the most part you can be sure that a bitmap is going to be considered a system resource. In windows, the kernel itself actually manages bitmaps, and you access these bitmaps via a handle that gets returned back to you after requesting a bitmap be created. Now I’m no operating system specialist, but my assumption as to why the kernel manages bitmaps is that it probably has something to do with bitmaps being stored in protected pages, so trying to read directly from those pages would throw an exception because they are protected, I would love to hear someone’s explanation of this in the comments or via email, I will edit this part to have an actual explanation with credits to said person if anyone has a better explanation that me.

Xamarin’s architechture (this should probably be it’s own post…)

So let’s break down Xamarin’s architecture really quick. Xamarin Android apps are basically two runtimes running side by side in virtual machines, in this case it’s the Android runtime and the Mono runtime. When using .Net code to work with Android, you are actually using an Android binding to invoke native Android code (For example using .NET file IO features to save or read Android files). Both of these environments run on top of the Linux kernel, which is what is doing the heavy lifting behind the scenes.

The idea of using code that is not Java to work with Java applications works because of something called a JNI (Java Native Interface). A JNI is essentially a wrapper around native Java code that allows said code to be invoked by any language that implements the JNI. In Xamarin, a JNI is implemented in two ways, as an ACW (Android Callable Wrapper), and an MCW (Managed Callable Wrapper). The names of these two things give us clues as to what they do. In the case of an MCW, this allows us to use managed code to invoke native code. In this case, Mono runtime code can invoke Android native runtime code. An ACW allows us to invoke Mono runtime code in the Android runtime. In Xamarin, ACW’s are how you are able to override methods and implement Java interfaces via C#.

The TLDR; is ACW’s are wrappers that Android can use to invoke managed Mono side code, and an MCW is a wrapper that Mono can use to invoke Android code, both are simply different implementations of a JNI. This is a very, very cursory glance at how Xamarin works with Android.

How this works with bitmaps

With how long I just blabbed on there, you might have thought I had forgotten what I was talking about in the first place, but never fear. Now that we understand bitmaps and Xamarin independently (kinda) we can discuss how they work together and why problems can arise. So we know bitmaps are managed by the kernel, and the kernel in this case is the Linux kernel. We also know that to reference the bitmap, we receive a handle from the kernel that allows us to work with the bitmap, so we have another layer, which is the resource handle in Android. Now, add Mono to the mix, which gets it’s own handle to the Android handle to the bitmap, and you’ve got 2 levels of indirection to access a bitmap. Weird, but it’s not obvious that this would cause any real issues. the relationship is as follows:

Kernel managed bitmap memory <- Android handle (pointer) <- Mono handle (pointer)

Okay so let’s take a step back for just a second to mention that in Xamarin, you have not one, but two garbage collectors running, the Mono S-Gen garbage collector, and the native Java garbage collector. Garbage collectors map the heap by specifying garbage collector roots, and then “linking” other objects to those roots based on references. So for example, I have an activity, which I could call my root, the activity is using a listview, so I associate the listview as a sort of link in the chain where the activity is the root. The listview is using an image view in each row, and the image view is using a bitmap, so the graph looks something like this:

(root)Activity -> List view -> Image View -> Bitmap

So if we wanted to clean the bitmap, how would we do that? Well, we’d simple dispose the bitmap, call the garbage collector to do a sweep, and that cleans it right? So now we’ve got this:

(root)Activity -> List view -> Image View(empty)

Okay, simple enough, but what happens in Xamarin is not so simple. The main problem is the Mono handle I mentioned a minute ago. Our graph (simplified to show just the mono stuff we care about) looks like this with Xamarin:

(root)Activity -> List view -> Image View -> Bitmap <- Mono handle to bitmap

So let’s say we just dispose the bitmap like we did previously, and we invoke the garbage collector. Here’s what that now looks like.

(root)Activity -> List view -> Image View -x-> Bitmap <- Mono handle to bitmap

So we have a bit of a problem here, the Mono handle to the bitmap doesn’t break it’s reference to the native bitmap, meaning when the activity get’s destroyed, we lose the variable which referenced the bitmap on the mono side (unless it’s static, in which case it will be it’s own garbage collector root which can be accessed at anytime from anywhere), and since we can no longer access the mono bitmap handle, but it still exists, the garbage collector deems that bitmap not ready to be collected because it is still being referenced, and we can never free up our memory, thus creating a memory leak. This is the exact problem I ran into with our app we made that I mentioned at the beginning of this post, I didn’t realize that Xamarin required you to manage resource handles on both sides of the equation, once you know how to handle this it’s not a huge deal or anything, but finding out how to do this was an absolute pain as no one seems to have posts about it.

That’s about all I have for the basics, it’s not really all that complicated. In the next few posts, I will discuss in more detail how to manage these bitmaps, helper methods I’ve written to streamline the process, issues you can run into when cleaning bitmaps, tools for finding memory issues with bitmaps, and a slew of other hopefully useful information. Thanks for reading, and if you see anything that I said that was incorrect, feel free to reach out to me and let me know, and I’ll be sure to correct it.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s

Advertisements
Advertisements
%d bloggers like this: