Android Binder

Binder is an Android-specific interprocess communication mechanism, and remote method invocation system.


 * That is, one Android process can call a routine in another Android process, using binder to indentify the method to invoke and pass the arguments between processes.
 * note that Android does NOT use SysV IPC for interprocess communication


 * the binder implementation is in the kernel source at: drivers/android/binder.c, with include file: include/uapi/linux/android/binder.h

Resources

 * http://www.angryredplanet.com/~hackbod/openbinder/docs/html/ - where it came from
 * http://developer.android.com/reference/android/os/Binder.html - java interface to binder
 * http://developer.android.com/reference/android/os/IBinder.html - java low-level interface to binder feature
 * http://developer.android.com/guide/developing/tools/aidl.html - Android Interface Definition Language


 * A good overview of binder is in the presentation Inter-process method invocation in Android by Tetsuyuki Kobayashi
 * Android IPC Mechanism - Another presentation about Android Binder and framework internals, extending Tetsuyuki Kobayashi's materials by Jim Huang of 0xlab.
 * Android Binder - Thesis by Thorsten Schreiber describing Android Binder in detail. Has good, detailed information about Binder (if you plan to work on Binder, consider this a "must read"!)
 * Deep Dive into Android IPC/Binder Framework - Aleksandar Gargenta of Marakana presented at Android Builders Summit 2013. Very informative.

Answers to questions
Q: Why does [Binder] need to be done in the kernel? Couldn't any of the current Linux IPC mechanisms be re-used to accomplish this?

A: Brian Swetland answers here:

I believe the two notable properties of the binder that are not present in existing IPC mechanisms in the kernel (that I'm aware of) are:
 * avoiding copies by having the kernel copy from the writer into a ring buffer in the reader's address space directly (allocating space if necessary)
 * managing the lifespan of proxied remoted userspace objects that can be shared and passed between processes (upon which the userspace binder library builds its remote reference counting model)

How it's used by Android
This is quoted from: https://lkml.org/lkml/2009/6/25/3, in it's entirety. (I hope that Daniel and Dianne are OK with this.)

2009/6/19 Daniel Walker  > Most of these questions related to the fact that I don't think an > interface like this just slips into the kernel as a driver. Since it's > IPC, it's totally generic, and it's not part of a standard (i.e. POSIX), > we need to have some better and more specific information about it (or > atleast I do).

Hi, sorry I have been slow to respond. I can give a summary of how binder is used in the Android platform and the associated feature set. I won't try to address other options, especially D-Bus, because honestly I haven't been following it for the last 3 or so years so don't really know its current state of art.

In the Android platform, the binder is used for nearly everything that happens across processes in the core platform. Some examples of this, illustrating key features are:

- The window manager and clients talk with each other through Binder. When a client starts up, it does a binder IPC into the window manager to create a new binder connection dedicated to that client. This is a common use of the capability model of the binder, where secure connections are given to clients which they can use for communication with the system.

- The window manager and lower-level surface compositor talk with each other through Binder. There is as simple binder-based API that is used to allocate a surface for a window. This takes advantage of the Binder's fd passing and object identity facilities to allow the surface compositor to allocate area in a shared heap it manages: the window manager makes this request on behalf of a client application, and then passes that binder object over to the client process (it will retrieve the associated fd and map it for each unique heap it receives) for it to draw directly into the associated surface memory. The binder's object identity rules (an object has a single identity as it travels across processes, no matter how many times it does so or where it goes) are very convenient for managing this.

- Separate components, like the window manager or surface flinger, may be switched between running in the same process or different processes with no change to their code. For example, in the current android platform these two components run in the same process, but we also have had run them in other processes and would like to do so on higher-end systems where there is more memory. This is not strictly a feature of the kernel part of the binder, but the IPC semantics it provides greatly ease its implementation: dispatching transactions to thread pools, synchronous calls with recursion across processes, etc.

- The activity (or really application/process) manager also uses the binder for launching and managing components in a process. For applications, it creates a simple binder object for use as a "token" for the application. It gives this token to both the application and the window manager, and the application gives its token to the window manager when it adds windows. Because the binder maintains object identity, this model is used extensively in the system for security: you can hand someone a token, and then can hand that token to others, and you can always check whenever you get a token exactly who it was originally given to without any way for clients to spoof it. So the activity manager can say to the window manager, "all of this token's windows should be hidden," and the window manager can absolutely identify which windows came from that application through the token the app supplied with them.

- The fundamentals of Android's security are a combination of uid-based permissions and binder capabilities. Some capabilities are direct (I give you access to my interface that you can call on), some are indirect (I give you a binder object as a token that you can compare against other tokens you receive to validate who it is). For permissions, every incoming binder transaction has associated with it the uid of the initator, which is used in numerous places where we want to only allow specific uids to access specific features. For example, there are APIs on the window manager to inject high-level input events into the system, and the implementation of those methods checks the calling uid to see if it is an application that has been granted the permission to do this.

- The binder natively supports one-way and two-way calls. Its two-way calls are used extensively by all of the system services for incoming IPCs for better multi-threading: they are dispatched directly from a thread pool and the services acquire specific locks as needed to protect their state (rather than serializing all calls through one thread). More traditional one-way/async calls are used for communicating back with applications (or really for any service to send commands to a higher-level part of the system).

- Many of the system services of course want to clean up state they have associated with a client process. For example, if an application process goes away, all of its windows should be removed. This is made easy by the binder's "link to death" facility, which allows a process to get a callback when another process hosting a binder object goes away. For example, the window manager links to the death of a window's callback interface, and other services have clients send a binder object token just to be able to find out when its process dies. The driver provides this facility by telling a process about the death of any objects it is watching.

- The Input Method Manager is probably one of the better representative examples of how the binder facilities are used in the system: it is a relatively small component, but makes extensive use of binder object identities, capabilities, death links, and other features to arbitrate between N applications and M IMEs securely interacting with each other in a controlled way. A taste of this can be seen in the "Security" section of http://developer.android.com/reference/android/view/inputmethod/InputMethodManager.html . One particular feature it relies on is allowing an application to hand it a binder object for an interface (here an InputConnection), which it can then send to an IME running in another process. That IME can now make direct calls on the InputConnection for just that application (it has been granted that capability) without having to go through the Input Method Manager intermediary process.

One part of the binder protocol that is really nice but doesn't yet have a user space implementation is weak references. This allows a process to maintain knowledge of a remote object, without forcing it to stay around. At any point it can try to promote that to a strong reference (to actively call on the object), which will either succeed or fail based on whether the original object is still around or is not around because all of the strong references (either in-proc or remote) are gone. We never re-implemented the user space code for this because we didn't do weak references in the Java layer, but for native C and C++ code it is a very nice facility for managing object lifetimes.

For a rough idea of the scope of the binder's use in Android, here is a list of the basic system services that are implemented on top of it: package manager, telephony manager, app widgets, audio services, search manager, location manager, notification manager, accessibility manager, connectivity manager, wifi manager, input method manager, clipboard, status bar, window manager, sensor service, alarm manager, content service, activity manager, power manager, surface compositor.

> If for instance the main reason for Google using this interface is cause > a large number of android people once worked at Palm or BeOS, that's not > reason enough for it to go into the kernel. Or if this binder interface > really fits well with Java or C++ people and they just love it, that's > not really acceptable either..

It is true that a lot of the ideas of the binder came from previous work on BeOS and Palm's Cobalt. However, that is mostly inspiration: we started with the Open Binder code for very intial bringup, but entirely rewrote both the user space and driver code to address our needs for Android and to better fit with the Linux-centric design of the platform.

I'm not sure what the relevance is of Java or C++ people liking it. Does this mean that the important thing is that C people love it and other languages don't matter? :) Anyway whether or not you "love" it I don't think is a matter of programming language but just design style, personal preference, and who knows what else.  It has been extremely useful in our implementation of Android, as can be seen in just how much of the system sits on top of it, but that's all.

Finally as far as someone else's comment of Open Binder being dead -- well it's an interesting situation. That particular code is no longer being developed, but basically the active development switched over to the fork/rewrite of it we have now in Android. You could maybe say that Open Binder was a research project, and Android is the shipping implementation. Though really, the main difference between them is that Android has a much simpler user-space implementation (because we didn't need the full features of Open Binder); there isn't any reason the full Open Binder environment couldn't be put back on top of the current binder. The binder shell is certainly a fun toy. :) See http://www.angryredplanet.com/~hackbod/openbinder/docs/html/ for example.  But a lot of the stuff there is just not hugely interesting for Linux/Android.

-- Dianne Hackborn Android framework engineer hackbod@android.com

mainlining feedback
Daniel Walker wrote (in June 2009):

I think the biggest issue I have with the binder implementation is that it's doing far too much in the kernel, it's not just IPC. It's also thread management, memory management, and lots of other stuff that I haven't figured out yet .. A lot of it can already be done in userspace.

Christoph Hellwig wrote (in June 2009):

The binders interface as implemented currently with it's fdtable munging and messing around with the user virtual address space has exactly zero chance to go properly upstream.

Marcel Holtman wrote a response to Dianne's message, with information about D-Bus capabilities that correlated with what binder does.

Some developers lamented that having 2 IPCs seems unnecessary, and it would be nice to migrate to 1 IPC for all of Android.

Some developers asked why D-Bus wasn't chosen as the IPC for Android. The selection of Binder was made before D-Bus was mature. Indeed, years later D-Bus was still missing some semantics that binder includes.

Neil Brown wrote: (see http://lists.linuxfoundation.org/pipermail/ce-android-mainline/2012-January/000042.html)

One of the big problems with the current binder interface is that it creates a strong tie between a file-descriptor and a process, without actually enforcing that tie. So if you open the binder device and then fork, it will almost certainly stop behaving correctly.

Another big problem - for me at least - is the use of a char device and ioctls. This is "wrong" from a philosophical perspective, and we could of course argue philosophy all day without getting anywhere so maybe that isn't a useful observation. Maybe a better way to look at it is that this approach effectively bolts binder on the side of linux rather than integrating it properly. Thus you lose the benefits of a uniform integrated whole.

When I was researching binder for my lwn article: https://lwn.net/Articles/466304/ I spent a while trying to come up with a "good" interface for the same functionality but was never really satisfied with anything I came up with. However I'm tempted to try again .... maybe.

Anyway, my thumbnail sketch would be something like this:

1/ A new 'address family' to be used with sockets. Addresses in this family are comprised of 1 or 2 'cookies' which are either file descriptors or an internal 'object descriptor'.

Thus if you open a file and bind(2) to that descriptor, anyone else who can open the same file can send you a message.

Also you can send a message containing an 'object reference' and it arrives at the other end as an 'object descriptor' which can be used in addresses. This roughly matches the addressing used in binder.

The optional second 'cookie' is just an object reference but it identifies a 'transaction' which supports the transaction handling that is implicit in  the current binder.

2/ New 'control messages' that can be sent with sendmsg and received with recvmsg which pass 'object' references

3/ A normal code path where a writer blocks until a reader is also ready, and then the message is copied from the writers iovec into the readers iovec without ever living in a kernel buffer.

4/ An extension to aio so that sendmsg/recvmsg messages can be sent async too. Then this would be used for async communication.

There are lots more details which I don't recall very clearly at the moment but I'm sure they can be worked out.

One of the biggest differences (apart from the obvious sockets-vs-chardevice difference) is that I wouldn't have any reference counting on object references (I know I said above that they are important .. but bear with me).

With binder, if you get sent a reference to an object that you already hold a reference to, then you get given the same reference. i.e. the same number. So the kernel keeps a count of the number of times it has given you a reference. This is important as one thread could be freeing references while another thread is adding them and there are possible races there which a refcount can avoid.

Note that this is different from sending file descriptors, both with binder and with Unix domain sockets. If you are sent a file descriptor when you already own a descriptor on that file, you still get a new file descriptor.

So in my thumb-nail-sketch, every object reference comes through as a new references. The objects keep a count of the active references, but the references themselves don't have counters.

So if you hold 12 references to the one object, they will be twelve distinct numbers that you hold.

One problem with this is that I believe Android uses presentation of a reference and proof of permission for some things. i.e. an object is used as a credential (a 'capability'??). So code needs to be able to ask 'are these two references for the same object?'. In android this simply a numeric comparison.

In my scheme you would need to ask the kernel to compare two 'addresses'. Some sort of 'setsockopt' might do it.

I appreciate that some of the things that I am suggesting we do to sockets are quite a deviation from the norm and you might argue that I am mis-using sockets just as much as the current binder is mis-using char devices. You might even be right: I am not objective enough to comment.

If there is interest in this (and maybe even if there isn't) I'll try to find time to make it all more concrete and detailed so we can ask some independent by-stander to comment.

relationship to kdbus
Greg Kroah-Hartman wrote on his blog about kdbus, and how it doesn't really do the same thing as binder.
 * http://kroah.com/log/blog/2014/01/15/kdbus-details/

History

 * Originally a feature in BeOS, Binder was a central construct for encapsulating software interfaces. One of the architects behind this mechanism was Dianne Hackborn, who is now a key employee at Android/Google. For more history see this interview.


 * The Linux version of Binder was originally derived from a project by PalmSource to implement a CORBA-like message-passing or method invocation system. Documentation on that system is at: http://www.angryredplanet.com/~hackbod/openbinder/docs/html/index.html