Device tree kernel summit 2017 etherpad
Local cache of Linux Kernel Summit Devicetree Workshop etherpad notes
(as of Thu Oct 26 10:13:17 PDT 2017)
Post-summit notes added at elinux.org are in braces, for example:
[Frank: this is an added note]
Devicetree Workshop Notes
- 1 Agenda
- 2 YAML Format
- 3 Zephyr usage of Device Tree
- 4 Device Tree in U-Boot SPL
- 5 Lunch 12:30-14:00
- 6 DeviceTreats (Arnd)
- 7 DT Maintenance
- 7.1 Device Tree Overlay Deploy (Bill Mills)
- 7.2 Deployment
- 7.3 Which DT Boards in the Kernel?
- 7.4 Avoiding duplicate descriptions (Thomas Petazonni)
- 7.5 How to handle DT foreign bindings (Maxime Ripard?)
- 7.6 Sharing Generic Bindings (Geert Uytterhoeven)
- 7.7 15:30-16:00 Group Photo + Break
- 7.8 API Stability (Lucas Stach)
- 8 DT Health Check (Ben)
- 9 EBBR Discussion (Grant)
- 10 DeviceTree.org (Grant)
- 11 ConfigFS
- 12 Connectors (David)
- 13 Feedback
9:40 Encoding and Schema
- DT Yaml Encoding (Pantelis Antoniou)
- DT Schema (option 1) (Pantelis)
- DT Schema (option 2) (Grant Likely)
11:50 Code Generation from DT (Kumar Gala)
14:30 DT Maintenance
- Overlay maintenance (Bill Mills)
- Avoiding duplicate descriptions (Thomas Petazonni)
- Criteria for accepting board files (TBD)
- Location for maintaining bindings (Maxime Ripard)
- Sharing Generic Bindings (Geert Uytterhoeven)
- API Stabiliy (Lucas Stach)
16:00 DT Health Check 16:20 Devicetree.org update (Grant) 17:35 EBBR Discussion (Grant)
- Connectors (David Gibson)
- Classes of embedded systems
- ConfigFS interface
- DT on servers?
Grant and Pantelis independently worked on this.
Grant is looking at how to make DT work with existing tools and workflows
Why YAML Encoding:
enables tooling. He looked at json schema, would be more work. YAML should allow us to work with existing tools
http://www.konsulko.com/yaml-and-device-tree/ * without special tools, DTS is hard to parse * The current schema is a lossless conversion, using tags in YAML * Labels are supported
show of hands, who is familiar with YAML? (1/3), JSON? (1/2) * YAML is a superset of JSON * YAML is friendlier for humans * YAML supports labels and refs (anchor/alias)
* The native reference support in YAML breaks with a 1-to-1 mapping of DTS references * JSON is really easy to parse, but for example number formats 0x are difficult to use in JSON * JSON schema works on parsed JSON, works also on parsed YAML
Q: Is this a replacement for the C preprocessor
No, it would be used as before
What we have now: DTS → cpp → dtc → DTB/DTS/ASM Prototype for: all DTS → Pantelis' tool →YAML → validation tool (with additional binding schemas) The format of the schemas should go into the DT spec so that we can validate Pantelis' tool can replace DTC currently there is no DTS → YAML migration planned
Q: Why is human readability important?
For this use-case, it is not important. Makes it easier to use for other tools.
Other projects may just use YAML and skip DTS
So we want to have it specced out.
Q: Are there tools to convert indented (human readable) to braced YAML?
Yes. All tools parse both.
Q: Can you summarise the alternatives?
XML has been considered, but isn't nice to work with.
The YAML format is only needed for the bindings?
The schemas are specific to the actual encoding to YAML.
Q: Are includes support in YAML?
Not in YAML itself, but we would have it in our tools
Q: Does it support comments? Are they kept during parsing?
Yes (#). Yes (ruaml in python)
Q: Is the DTB different?
It should be exactly the same (in compatible mode)
Let's move on to where Pantelis is at, so we can talk about schemas. We can't use native * references in YAML, would use $ and a DT sepecific resolution step after parsing
Q: Why to we need to quote "#address-cells"?
# is comment
#address-cells is the reason comments in DTS are the way they are
We are asking people to understand the YAML syntax to work with schema files.
The YAML files correspond to DTS files, are not deduplicated and node-deleted yet.
Would like to run validation on DTBs. That wouldn't work if the schema is fixed on the YAML format.
Use the schemas to decode the DTB to have the correct type info. Only then do validation.
Frank: The following image was drawn on the easel by Frank.
Arrows "1", "2" and "3" are possible data flow paths. Choice of data flow path is an implementation detail.
DTS include files are not shown to reduce clutter. DTS include files can be input to either cpp or dtc when dtc is used. ]
Pantelis: Goes through his web page https://www.konsulko.com/yaml-and-device-tree/
Q: Can the constraint be C code?
Q: Can you show some more constraints?
Examples from website and live demo of error detection (invalid value in property)
Can you reference other properties?
Each schema is for a node. You'd like to have separate schemas for nodes and properties to allow reuse.
Who has looked at JSON schema? 3 persons a little bit
At first discarded JSON schema, because he thought it didn't fit.
After looking at YAML schema, it seems JSON schema does actually work.
Explains dt-schema-core.json file
This would not be written by the schema writer. Would be similar but much more constrained. Have no examples for that. The dt-schema-core would be able to validate a set of standard properties (#*-cells, compatible, interrupt-controller/-parent, …)
Parts of the schema can be reused in other schemas (size-spec, address-spec)
Documentation can be in the JSON schema files.
JSON schema validators have support for custom extensions, would allow adding eBPF constraints
Q: how do the errors look?
A: Terrible. But there is context. Could be improved.
Q: Brings additional requirements to build the kernel.
A: Python, ruamel, JSON schema. Could be moved to dtc. Validation would not be required for build, only useful for development.
Should try to run validation during build if requirements are available.
Can the schema follow a phandle?
Should be possible to implement as an extension.
Q: Can we validate busses?
The bus schema could require that every child has the right format. Would be hooked up by the controller schema.
Validation has two parts: Triggering and Checking Triggering could be done by compabile or by a parent. Or we could start using device type
We need this to work with existing trees.
Q: For a sensor connectable via SPI and I2C, do I need copies?
No, this is supported by JSON schema. Properties can be required depending on the actual type?
Next steps for schema: Pantelis and Grant need to build a combined proposal.
Q: We need to have a page to read an up-to-date summary on what has been agreed
A: Wiki page and discussion on the devicetree spec mailing list.
Zephyr usage of Device Tree
- Zephyr overview
- Kconfig was used to describe HW properties, was a mess and didn't scale
- DT is obvious solution
- DTB is too big, but can generate code
DTS → cpp → dtc → DTB → dtc → DTS → python script → Kconfig definitions + C headers
Q: do you have helper macros?
Want to move to structure generation soon
Support counting to size IRQ, DMA structures to the actual requirements Have I2C, Serial, some others, many still missing Only ARM, x86 supported. ARM completed.
At Xilinx, they are generating headers, and it's not going to work. You'd need another layer to select which instances to use and so on. The C code would get a lot more complex.
TI has a system called ritzy, which is not recommented.
Maybe have a templating step to generate code.
U-Boot has a tool to generate structures from DT
There are too many parsers for DTS/DTB, … we should have common libraries
RioT would possibly benefit from this as well
Device Tree in U-Boot SPL
Size contraints, ~32KiB
Tools: ftdgrep, dtoc
fdtgrep filters out parts that u-boot doesn't use (40K to 3K), based on libfdt+python
dtoc: builds structs in header files and c files to fill them out. It's used together with fdtgrep
The same drivers in normal u-boot and SPL.
The of-to-platdata code (which uses libfdt) gets replaced by simple assignments.
Whishlist: Smaller DTB format (shared propery value)
I we'd do a new DTB revision, that would be on the table.
Was designed to be compact for 2005 servers.
What would you need to generate the C structures?
Common Problems: * how to generate names, * how to handle busses/slaves, * filtering of nodes
We can't generate C code well without schemas, so we have nicknames and type info.
If anyone is interested in working on this, talk to Simon and Kumar
[Frank: Arnd shared some chocolate. Thank you Arnd!]
Device Tree Overlay Deploy (Bill Mills)
Bootloader applied is here:
- Android will use it next year
Android DT Overlay top level: https://source.android.com/devices/architecture/dto/
DT Partition format: https://source.android.com/devices/architecture/dto/partitions
- TI will use it in Jan 2018
Android: SOC (base) + board (overlay)
Upstream: board based
Interested in Board with options
Opinions on the source format?
in Rob Herring's branch, coming soon: the dtc can use a dtsi as an overlay (plugin directive) [Frank: clarification - the device tree source format will be the same for an overlay or for a dtsi include file. The only difference in use is that the dts including an overlay will specify '/plugin/;'.]
two parts: technical capability and policy
Can the schema validation support the maintainance of the overlay sources?
Is using the old patch for the delete node support.
We should reconsider merging that.
How do nested pluggable boards work?
There will be general usecases, we need to have guidance for that. Who will provide that?
Android wants to distribute SoC and board parts separatly (SoC is responsibility of the silicon vendor)
The base should be the most specific thing, so the board.
- Append base to kernel or put it into its own partition
- Overlay is a file in ODM partition or in its own partition
- Android specifies a multi image format for DT partition
- TI uses FIT images for everything for secure boot
- DT should come with the board. Bundle into the UEFI binary?
- Or in EFI partition?
Originally, UEFI should provide a default, but OS can provide another one. We should make sure that we always have appended DT
Appended DT doesn't exisit for ARM64
In the ARM Trusted Firmware world, the necessary configuration for the TF parts may be passed via DT.
The fit configurations refer which overlays to apply.
Different models for U-Boot verified boot and UEFI secure boot, in the future U-Boot may support the UEFI model.
Some ppl in debian would like verified boot on u-boot.
Would like to avoid many approaches to this.
Which DT Boards in the Kernel?
Q: Which DT boards would be rejected?
A: Only for something extremely configurable like ARM fast models.
A DTS for a zynq board was rejected, there would be too many. We don't want to have DTS for configurable logic
Why fix the flat DTS directory in ARM32
For example xilinx/vendor (if a vendor has 35 boards) We might have an external master (git repo) for a subdirectory Will need more to think about this topic for ARM32.
Should move this disussion to a whiteboard later.
Avoiding duplicate descriptions (Thomas Petazonni)
Marvell Armada 7k/8k:
somewhat modular AP (CPU, simple IO) / CP (network, complex IO) currently AP806 dual/quad, CP110 but can have multiple of each (4, potentially 8 AP)
cp110-master and cp110-slave are mostly identical, except of internal references, offsets)
possible solution C preprocessor to generate cpm_ or cps_
one big macro?
The idea of macros has been around for some time... There are mainline python bindings to libfdt
The HW will be taped out in the near future (months)
we could generate this from a scripting language
The decision is between cut&paste and CPP
We are using the CPP approach in QEmu for generating HW descriptions
We accept patches for string expressions for node names.
Would be nice to handle the case of Thunder X of hundreds of CPUs.
Complex CPP macros would worry him for YAML
1. see if dtc can be changed to support relative references 2. CPP macros or copy paste
How to handle DT foreign bindings (Maxime Ripard?)
"bindings defined by other projects" used by Zephyr, U-Boot, Barebox, OpenBSD, ...
u-boot,efi-partition-entries-offset uboot-dm-pre-reloc /chosen/tick-timer
different ARM GIC compatbiles (fixed now), they try to align with linux
u-boot,dm-pre-reloc have been rejected as an implementation detail
We tried to push back on these, but if it's the best solution, we should accept it. Needs good justification
Should be limited
There are zephyr,*
What's with projects forking the linux bindings? Using existing properties differently...
U-Boot uses the regulator names which were intended to be only cosmetic in linux
Seperate Core HW bindings and Linux (and other project) specific bindings in the docs?
Where should we store the bindings? Close to the tools, to make it easy to use validation.
maybe use devicetree.org more
Adding more projects to the Linux review won't increase the volume alot. Use the same release cadence as the kernel. Maybe similar to the separate firmware repo.
Default stance: Bindings belong in the kernel. Accepting the binding together with the user. We are still the primary users.
A release is not needed to depend on it, applying it would be enough.
Having an external tree would make it more explicit when it is accepted.
Maybe generate spec documentation from the stable schema parts. Currently, the kernel tree is the canonical binding source.
Sharing Generic Bindings (Geert Uytterhoeven)
IP block gets reused, now there are explicit power, clock, resets
power-domains clocks resets …
David: seems similar to interrupts
should we define that in a generic binding?
maybe in a device binding that is inherited by all device? have a category of properties which have the same meaning over the whole system?
use a new compatible string for new integrations
the way they relate to the device is specific to that device
The question is wether the integration of an IP should be reflected in the compatible This is also relevant in the validation case A consistent rule at the SoC level seems not possible
We want the common properties in the generic binding We also see integration relevant errata, so we need the string.
How do we document policy? Mailinglist? Volunteers?
Renesas already uses SoC-sepecific compatibles
We have an interface (soc_device_match()) to even match against a specific version of the SoC, it's not widely known
15:30-16:00 Group Photo + Break
API Stability (Lucas Stach)
Cares about backwards compatibility: use new kernel with older DT from the firmware
Infer as much as possible from compatible or actual hardware
ACPI now can use DT properties needs a good taste decision on how much detail goes into a binding → need policy document deviations are possible, but they need to be justified
Are we also trying to run old distro on new HW/FW?
not as little or as much as possible, but the right amount the ideal case would be to also run old kernel on new FW, but old FW with new kernel is much more important
We've become better at designing bindings (like graph bindings) If we break things, have a really good (documented) reason Enterprise ecosystem (HW&SW) has decided: UEFI, ACPI, Datacenter, Support model Distro support is important (debian)
Use both original implementation and soc specific compatibles, allows fixing quirks in newer kernels
Sometimes you don't have enough information in the begining for some socs
It's also possible to rev the compatible
Standard properties for version information? you could have soc, ip version, ip variants all in the compatible string?
originally there was model, which was the sepcific
Use most and least specific, and required ones in between
Is the compatible about features or bugs?
Both. The specific hardware interface.
For configurable HW blocks, it is better to have registers in the HW.
To enable features that didn't work before, updating from a broken DT in FW is reasonable.
Will write a docs patch
We need a guidance document! Could sit at devicetree.org
DT Health Check (Ben)
- does DT help us in total?
- media pipeline status?
Some vendors are lagging behind mainline devel He likes some things about the old board files.
A graphical tool based on the schematics?
Grant pain points?
Power has some quirks that key of the board compatible
ARM64 doesn't have those 50% of ARM32 has those
We see board specific code as a last resort (drivers/soc)
Leftover in ARM32: SMP and cpufreq, code to communicate with a onboard microcontroller which doesn't fit any subsystem
Do we need to have some area to improve?
Is distro support in our goals? Yes. Having everything on the generic boards to boot debian would be very good for the ecosystem. It should be a goal to be able ship DT with the firmware and boot debian from it.
In the far future he'd like a DT repo for all project using DT, but this is difficult for many reasons.
We can't say that the DTs should be shipped with the Firmware when it's build from the kernel.
Developing DTS and Linux in lockstep hides the real problems.
So we should get the tooling in place to have a separate repo for DT. Like firmware.
DT is not like firmware. We shouldn't decide this here.
We need to support diffierent models. Keep DTS in kernel, get tooling in place which support a separate repo.
Yes. Let's do that.
EBBR Discussion (Grant)
ARM wanted to build a spec on how to build more standard embedded systems.
- DT noch ACPI
Not for all embedded systems
What would you like?
Contact Grant afterwards per Mail
A place for the specs
Some people want to have assurances that DeviceTree is going to be "stable"
Be a bridge
Spec is on github, using sphinx
Discussion on kernel.org list
Is a membership org with a steering commitee
SC oversees the release process
SC: IBM ARM Linaro ST NXP
Should the schema work (and tooling) be done via github?
Restart spec process and get it up to the current state.
It is a generic way to apply a DT at runtime.
What is the status of the patches?
Needs a way to specifiy which nodes are allowed to be overlayed. Whitelist in the base tree would be configuration in DT.
Can we constrain which parts of the tree are adjusted by the overlay?
shows slides from bof there are people using overlays in the field there is breakage in mainline when applying overlays accepting overlays would commit us to fixing those
Wants to clean up internal handling of overlay syntax in dtc
We could start allowing appling limited overlays, maybe even with removals.
Applying in the bootloader would work now.
You have the same problem with duplication as with the armada.
You need to have an overlay for each slot where you can connect something.
- allows mapping for interrupt ids from local ids to global ones.
- have a compatible string
- can be available multiple times
- local names for things which are connected (as aliases)
- at the /plugin/ directive it declares into which socket it plugs
- has a toplevel compatible describing the actual device
some FPGA ppl said this would work for them
Doesn't have to be hotplug, could also be used at boot time or in the bootloader
The connector plugin DTB could be untrusted
We'd need specs for many subsystems (irq, clock, dma, gpio, …) to build the translator nodes
We might need to have a driver for the connector for more complex cases, which can handle the enable/disable.
Need support for pinmux
Needs special translation nodes for SPI
Difficult case will be a SoM connector with several soc vendors and board vendors.
You don't want to use overlays to configure many pads on your soc.
Partial reconfiguration has been around for years.
To simplify: maybe have just a set of specific pinmux configs to select from for each connector.
Could we constrain overlays enough that we would be fine with exposing that? Maybe with a platform specific driver?
Platform, I2C and SPI could work.
Xilinx has a 96Boards-like board which might be useful for referencing
Important are FruitPis, Beaglebone, 96Boards
Discussing the agenda was useful.
We need the key ppl. and subsystems present.
Again in 6 months? (50%) 12 months? (50%)