Kernel areas of focus for mainlining

From eLinux.org
Revision as of 14:12, 4 March 2016 by Wmat (talk | contribs) (updating vivante)
Jump to: navigation, search

This page has a list of areas of the kernel where multiple SoCs have code out-of-tree, or that appear to need some work in order to accommodate contribution of device-related software. There is also a maling list for discussion of these issues at http://lists.linuxfoundation.org/mailman/listinfo/device-mainlining


Overview

The following table shows major out-of-tree areas of code, for mobile phone products based on the 3.4 Linux kernel. For sources see Phones Processors and Download Sites. Although this kernel is quite old, it was the one currently shipping in phones in late 2014 and early 2015, when this analysis was done. It demonstrates the major technical areas where vendors used out-of-tree kernel code, for that generation of products.

The table shows different phones, and the number of lines they have out-of-mainline for several different kernel areas. The analysis was done using the upstream-analysis-tools (See https://github.com/tbird20d/upstream-analysis-tools). Since this task was mostly automated, the results here should not be considered definitive. That is, most of the categorization of diffs was done using regex matches on the file paths. Although this can result in some mis-categorizations of diffs, the results appear to show general patterns which may be of interest.

Note that the table shows the number of insertions (or changes). The number of deletions was found to be inconsequential for this analysis.

Manufacturer/phone/SoC => LG/G3/Qcom Samsung/Galaxy 4/Exynos Acer/E2/Mediatek Asus/Zenfone 6/Atom Huawei/P6/Hisilicon Sony/Z3/Qcom
Kernel area
Mach specific 417411 89118 ? ? ? 393767
Media 131811 163540 ? 1466714 153032 124973
Video 221550 176787 37876 38785 238267 214027
Wireless 110946 123308 387214 91449 122501 81502
Sound 216643 86127 12932 74057 19250 213137
Input 107334 51160 2752 53321 35410 62930
Camera 66989 1342 7900 1024348 51065 56744
GPU 53345 172300 114932 36545 198323 53468
Power 61912 7635 94059 66629 45127 46173
USB 99295 35366 30795 34074 121075 98663
Notable other Prima - 565210 firmware - 101069 i2c - 105744 reg - 90034

Technical areas

Here are some lists of things to get mainlined, in different technical areas:


Media

Video

Wireless

action item: Find out if any vendors have the same IP blocks for wireless Here are the findings for 3.4:

  • LG/G3/qcom - Broadcom 4330/4334/43341/4335/4339
    • source include drivers/net/wireless/bcmdhd
      • this code is organized differently from the Acer bcmdhd code
  • Samsung/Galaxy 4/exynos - Broadcom 4335
    • source includes drivers/net/wireless/bcmdhd
    • source includes drivers/net/wireless/ipsecdrvtl
  • Acer/E2/mediatek - Broadcom 4329/30 wireless chipsets
    • source includes drivers/net/wireless/bcmdhd
    • mainline 4.2 has drivers/net/wireless/brcm80211/brcmfmac,brcmsmac,brcmutil
    •  !! also lots of code in mdtk/krnl/drivers/combo/drv_wlan/mt66s*/wlan/mgmt,nic,os/linux
      • code under this tree is not in linux format (it has vendor style)
      • need to see if this is some mediatek on-chip thing, and Acer added external broadcom chip for E2
  • Asus/Zenfone 6/Atom - Broadcom 4329/30
    • source include drivers/net/wireless/bcmdhd
  • Huawei/P6/Hisilicon
    • defines CONFIG_WL12XX=m
  • Sony/Z3/Qcom -
    • Sony has historically used broadcom 4339 and qcom 3660 and qcom 3680.
    • Sony did internal testing of the brcmfmac driver on a phone using a 3.4 kernel with the BCM 4339 chipset

Random notes:

  • Google got burned in early Android - TI had a bad wireless implementation (it was cross-platform and didn't work well in Linux). Google now integrates broadcom code themselves, at drivers/net/wireless/brcmdhd, and requires current Android vendors to use broadcom. This explains why everyone has it.
  • broadcom driver in mainline is a drivers/net/wireless/brcm80211/*
    • this is the one people should try to use and extend if it doesn't work

Possible strategy:

  • backport brcm80211 code to 3.18 or 3.14. These are the versions of kernels likely to be used in the next generation of Android products. Having the brcm80211 code available in that version means that phone vendors can test the driver. If it's close to product grade, they can help improve it - if not, then they can report the issues upstream.
    • Is Google or someone else already doing this - AI - check 3.18 src from SoC vendor for brcm80211 code

action items

  • try brcm80211 driver on 3.14 Android device
    • identify a 3.14 Android device using a broadcom chipset for wifi
  • try brcm80211 driver on dev board running mainline
    • identify a dev board running mainline
    • make sure the board can run the brcm80211 driver
    • test to make sure all required features work
      • figure out what the required features are (is there a wireless test suite?)
  • try brcm80211 driver on phone hardware (Xperia Z3) running mainling top-of-tree (in progress)

Sound

  • HDMI integration with graphics for EDID and hotplug information.

Input

MD

  • [1] Optimisations to make dm-crypt play better with hardware crypto modules.
    • Worked on at Linaro, req based approach not going to fly upstream.
    • Basic idea is to use large blocks for crypto but the block layer really doesn't want to use anything other than PAGE_SIZE as the size, will require changes in the block core.
    • Also working on optimising the crypto layer to be more efficient with smaller block sizes.

Sensors

The Android sensor stack is one of these places where vendors are expected to put in their own sensor library providing some middleware (in userspace) to provide sensor input. Some vendors will utilize IIO (industrial I/O, drivers/iio) as their backend for this, but several still have non-upstream custom char devices to provide the sensor input. The Android x86 project has some initial work on IIO-based sensor input, providing polled (non-buffered) sensory data, but it is not part of the official Android git/repo forest.

Migrating custom sensor driver code to use IIO and provide an IIO library by default in upstream android can advance development to address this problem.

  • LG/G3/qcom -
  • Samsung/Galaxy 4/exynos -
  • Acer/E2/mediatek -
  • Asus/Zenfone 6/Atom -
  • Huawei/P6/Hisilicon -
  • Sony/Z3/qcom - older qualcomms have a custom kernelspace driver but could be migrated to IIO, the newer ones utilize a special sensor processor, the communication to this coprocessor is in userspace and presented directly to Android instead of going through the kernel and IIO, these could also be handled in kernelspace but require the off-chip communication to be handled in-kernel first.

Camera

GPU

  • HDMI integration with sound for EDID and hotplug information.

Mali

The Mali GPU is used on SoCs from Allwinner, Amlogic, CSR, InfoTM, HiSilicon, Leadcore, MediaTek, NetLogic, Nufront, Rockchip, Samsung, Socle-Tech, Spreadtrum, ST-Ericsson, ST Microelectronics, Telechips and Wondermedia (according to the Wikipedia page). Sudip Mukherjee suggested adding Mali 400 for ODROID-U3 to staging. The Odriod-U3 has lots of stuff upstream and uses a Mali 400 and Exynos 4412.

GregKH says he won't take a DRM kernel driver into staging if it doesn't have an open user-space driver. The Mali driver from ARM is apparently closed, and they appear to have no intention to open it. As it stands, Greg would not take the driver into staging.

It was suggested that possibly the Lima driver could be used to fulfill the requirement for an open-source user space. However, the Lima driver project seems to not be active any more.

Heiko Stuebner said:

An additional point would be the possible (in-)stability of the userspace 
interface. While I could sucessfully use a r6 midgard kernel driver with a r5 
userspace lib yesterday on my Rockchip Chromebook using Debian, I don't think 
that is guaranteed to work everytime or that special care is taken for the 
kernel driver to prevent needing upgrades in lockstep.

And that is not even taking into account, that the mali userspace libs seem to 
be implementator-specific somehow too. For example for Rockchip I currently get 
my libs from the ChromeOS tree [0] which can talk to X11 [although not plasma5 
it seems] and ARM seems to provide a variant for the firefly board using the 
legacy fbdev interface of the 3.10-based Android kernel. And then there are 
different variants for Samsung, etc too.

...

[The Lima project is] probably not entirely defunct, but still on life-support:

For one, if you click on the "Source"-link on limadriver.org you get the 
gitorious "we're migrating old projects to archive.org" page.

And secondly there is supposed to be an actual mesa driver for mali400 and 
even bits for the midgard gpus (Mali-T) around, but limited to Luc's harddrive 
for now [1].

Vivante

The Vivante GPU is used in Intel Atom, CuBox, Rockchip, Actions Semiconductor, Freescale, HiSilicon and Marvell SoCs, according to the Wikipedia page.

The Etnaviv project has proposed initial GPU driver support to the upstream kernel.

The Linux kernel driver developed by the Etnaviv project has been merged into Linux 4.5. Lucas Stach gave a presentation on etnaviv at FOSDEM 2016. See the slides and watch the video (not available yet).

Adreno

Rob Clark has been working on freedreno, an open source driver for the Adreno hardware used on QualComm SoCs. He gave a presentation on the status at ELC 2015. See the slides, video and showcase poster.

Rob reported (by private e-mail in September 2015) that there is now the GLES 3.0 code has support for mesa 11. (Much more detail about the status of the project (as of March 2015) is available in the slides.)

PowerVR

Nvidia

MMC

Power

Battery charging

Battery charging to be handled in drivers/power. These can be subdivided into two classes:

  • Smart batteries, resembling laptop batteries, which have a hardware-provided state machine to charge and monitor the status of the battery. These have good framework support upstream already in the struct power_supply_desc and enum power_supply_property found in <linux/power_supply.h>
  • Batteries requireing an external charging algorithm. It was at one point unclear what part of a charging algorithm should go into kernelspace and what parts should possibly go into userspace akin to how Wireless have policy in userspace. It is nowadays widely agreed that any battery handling that may cause damage to the hardware if not attended to need to go into kernelspace, and that includes the major part of the charging algorithm.

The latter class of batteries exist upstream but does not have shared code and frameworks in place to implement the charging algorithm. These algorithms are quite physical and close to standardized, the most well-known concepts are CC/CV (Constant-Current/Constant-Voltage) and trickle charging. These algorithms are then modified to handle critical events such as overtemperature or overvoltage, and for this reason they periodically perform temperature and voltage and current measurements. They are mostly implemented in the form of a state machine. Interaction with IIO drivers for ADC and/or temperature is ideally needed to provide the necessary input.

We may not see the battery charging code in vendor kernels because it has been shimmed to userspace, even though it should technically be inside the kernel.

  • LG/G3/qcom -
  • Samsung/Galaxy 4/exynos -
  • Acer/E2/mediatek -
  • Asus/Zenfone 6/Atom -
  • Huawei/P6/Hisilicon -
  • Sony/Z3/qcom - using the kernel frameworks with a whole suite of alien battery charger drivers in drivers/power, extensions to the power subsystem to handle things like battery aging, shutdown on low battery and battery ID string. These could probably be mainlined easily. Apart from this each battery driver seems to spawn a nexus of custom sysfs files which probably each have a non-standard userspace daemon to talk to them.

USB

  • USB-gadget to charger interface
    • Currently in progress at Linaro.
  • Code for external vbus and ID detection (looks like extcon was created for stuff like this - ish, it's mainly a mainlining of the Android switch API though it probably does fit)
    • The ID pin may have different resistance across it (between it and ground), indicating different charger configurations.
    • For this reason, on some platforms the ID pin is not routed to the controller, but to the PMIC, so it can do the charger detection.
    • The mainline chipidea controller code doesn't handle charging correctly, as it only looks at the ID pin in the PORTSC register

USB controllers for different SoCs

Need to look at other controllers, and see how they handle the ID pin for their OTG ports.

  • LG/G3/qcom
    • HS = chipidea (dual role) controller
    • SS = dwc3 controller
  • Samsung/Galaxy 4/exynos -
    • config uses CONFIG_USB_EHCI_HCD, CONFIG_USB_EHCI_S5P, CONFIG_S5P_DEV_USB_EHCI
    • config uses CONFIG_USB_DWC3_DEBUG, CONFIG_USB_EXYNOD_SS_UDC, CONFIG_USB_XHCI_HCD, CONFIG_USB_XHCI_EXYNOS
    • code can use CONFIG_USB_S3C_OTGD

S5P is related to USB switching - see arch/arm/mach-exynos/dev-usb-switch.c exynos4_device_usb3_drd1 has name "exynos-dwc3"

    • S3C = UDC for OTG for some Samsung SoCs
  • Acer/E2/mediatek -
  • Asus/Zenfone 6/Atom -
  • Huawei/P6/Hisilicon -

See Tims_USB_Notes for more information

GPS (Global Positioning System)

GPS or any other positioning system for that matter, seems to be driven from userspace. In mobile handsets Assisted GPS utilizing cell tower data is commonly used. These chips usually consists of a "baseband" (in the modem terminology, the digital chip) and a RF (radio frequency) frontend that connects to the antenna (also called an air interface). If the "baseband" is a stand-alone chip, it tends to have a large firmware and complex interactions over some local bus such as simple UART. The power states of the chip is also driven from userspace, making power-saving scenarious uncoordinated with kernel power management for the most part. Assisted GPS requires network connectivity over cellular data and special interactions with a cellular modem to work.

Stand-alone GPS chips for devices without cellular modems (such as tablets) also exist. Garmin has open source releases for all their Linux-based GPS devices, based on "basebands" such as OMAP, PXA and Telechips.

Whether the positioning code should be in the kernel or in userspace is a question of debate.

Bluetooth/NFC/GPS

Many parts of these systems use a combination of code from user-space and code in line discipline modules that communicates with the code through the kernel via a serial interface. Here are some problems (quoted from the ksummit discuss list, and not very organized at the moment - my apologies if the attribution is wrong):

Linus Walleij wrote:

It seems BT chips more often than not sit on a UART connection
(often very high speed) and since there is no "uart bus" or "serial bus"
akin to what we have for I2C or SPI, it is actually impossible to
instantiate them properly in the driver model. Instead BT drivers
are poked and peeked from userspace using the line discipline as
if they were some kind of modem, just in-kernel.

Rob Herring appears to have written:

Yes, but then there's always some side band signals for regulator
control, reset, clocks, wake-up, rf-kill, etc. that needs a glue
driver. There's been some work by Neil Brown to create a UART slave
bus[1] and I've gotten several other UART device bindings recently,
but they all suffer from being one off device bindings done in
different ways and I want to see something common here.

Someone wrote:

It also seems the BT folks are working on moving more of the setup
into the kernel and creating a proper subsystem. I'm not sure about
the details on that.

Marcel Holtmann wrote:

We are indeed moving a lot of the initialization and handling of the
Bluetooth controller into the kernel. So that this can also be shared
between USB and UART based controllers.

The work on UART slaves (or whatever it will be called eventually) is
important for Bluetooth and most likely GPS and NFC in the future. It
then allows to define all the nasty behind the curtain details of that
UART via DT or ACPI in vendor drivers.

The Bluetooth driver in theory should not care at all and generally
speaking it doesn't since the Bluetooth SIG has defined a Host Controller
Interface (HCI) that is standard. Meaning all operation are well defined.
Just the transport over UART has many vendor specific tweaks. The two most
obvious ones are deep sleep modes and running the UART at high speeds.

In a perfect world I would prefer we are not using the Bluetooth HCI line
discipline at all. The problem right now is that everybody wants to enable
the UART as /dev/ttyFOO and then move on. However in reality they are not
general purpose TTY devices. The only thing you can ever do with them is
tell the Bluetooth subsystem that there is a TTY device and attach its line
discipline to it.

The majority of these high speed UARTs in phones, tablets and also laptops
with Bluetooth chips attached to it should just expose a special bus that
we can then enumerate. Pretending to a general purpose UART is actually
cumbersome and you will just see more and more of these.

While Bluetooth supports USB and SDIO as well, the most common other low
power interface is the UART (using H:4 or 3-Wire Bluetooth transports).
The usage of PCI or SPI or I2C as interface to Bluetooth never really happened.

Current Bluetooth/WiFi chips usually come in these flavors.
  * WiFi on PCIe + Bluetooth on USB,
  * WiFi + Bluetooth on USB,
  * WiFi + Bluetooth on SDIO and
  * WiFi on SDIO + Bluetooth on UART.
Now if you are building a mobile phone, you can choose the low power version
you like. And this often turns into SDIO for WiFi and UART for Bluetooth.

Or and did I mention that some manufactures actually put FM radio and GPS
controls behind the Bluetooth chip. So access to these slave devices goes
via the Bluetooth HCI. This means that you have this fun dependency:

  FM radio -> Bluetooth HCI -> UART -> UART slave

Some nasty solution here are to double stack the line disciplines and have
some sort of shared transport driver in between. I am trying to actually fix
that so that Bluetooth drivers can expose proper platform buses with correctly
assigned resources that the FM radio driver would just enumerate on. So that
part is in the works as well.

We can also talk about A4WP and the corner cases where your battery is so
drained that the Bluetooth Low Energy portion of your hardware is the only
one that gets powered first before you get your device back to life. In case
this is not obvious, the A4WP wireless charging technology using Bluetooth Low
Energy for its communication.

PCIe

Endpoint / nontransparent bridge

The kernel multiple implementations for communicating directly over PCIe, but none of them are easy to use or generic enough to cover multiple vendors. Examples of existing upstream code include

  • drivers/ntb (initial code meant to be generic, but in fact still specific to Intel hardware)
  • drivers/misc/mic (specific implementation for Intel MIC)
  • drivers/misc/spear13xx_pcie_gadget.c (broken driver for designware pcie)
  • drivers/remoteproc/ (generic framework that could be extended for this)

out of tree examples include

  • an old prototype virtio implementation for PLX NTB (ask Arnd)
  • Roadrunner/Triblade NTB
  • Marvell endpoint mode
  • APM endpoint mode

A requirement for a framwork would be that it can support both arbitrary hardware implementations of NTB or endpoint through device driver modules, and arbitrary protocols (network, rdma, block, file system, console, socket, ...).

template list

skeleton for other comparisons:

  • LG/G3/qcom -
  • Samsung/Galaxy 4/exynos -
  • Acer/E2/mediatek -
  • Asus/Zenfone 6/Atom -
  • Huawei/P6/Hisilicon -
  • Sony/Z3/qcom -

Notes from ksummit-discuss list

  • RGB leds - not the same as 3 leds, and sometimes have hardware acceleration
  • battery over-discharge protection
  • charger (there are 3 different chargers on N900)
  • voice link to modem (Nokia says ALSA is not suitable for modem audio)
    • No need for this, ALSA is in active use for rather a lot of production phones...
  • userspace (mini-distro?) for testing
  • suspend-to-ram