Mainlining improvement ideas

This page has a list of ideas for improving mainlining activities for large out-of-tree code bases (often found in mobile products). These were gathered from meetings and online forums. An attempt has been made to recognize the source of the idea. Please accept my apologies if you thought of something here but are not credited.

from LWN.net article about the LCJ "obstacles" talk
Here are ideas that were posted on LWN.net in response to their coverage of Tim Bird's "Overcoming Obstacles to Mainlining" talk at LinuxCon Japan. See https://lwn.net/Articles/647524/

Summary of ideas:
 * 1. binary blobs should be discouraged, as they freeze downstream to particular kernel versions
 * 2. NDAs are another obstacle to contributing
 * 3. use staging for non-mainlinable code
 * 4. mail patches to LKML for future reference, even if not for mainlining
 * 5. make software costs visible
 * 6. get hard data on costs for out-of-tree code
 * 7. make hardware available to the community
 * make it so that phone owners can work on mainlining
 * development of the Sony_Debug_Assist board is part of this effort
 * 8. make a dumping ground for non-mainlined source
 * possibly use automation to create it
 * 9. vendors should release kernel source ASAP, to reduce customer wait for source
 * 10. create a wiki page with links to source trees
 * see Phones Processors and Download Sites


 * For more details, see LWN_Obstacles_article_details

from ksummit-discuss thread
How hard would it be to get "patch" or "git apply" to apply white-space-damaged patches? Wiggle does a good job of a certain class. I came this --><--- close to getting wiggle to strip trailing '\r', but I never received that third patch to push me over the edge.
 * (Darren Hart) make mainline easier by creating a web form for first-time or infrequent contributors
 * avoids e-mail and formatting hassles
 * (Neil Brown) Create tools to pre-process patches so they're more acceptable to maintainers
 * see http://lists.linuxfoundation.org/pipermail/ksummit-discuss/2015-July/001825.html

How hard would it be to create a pre-commit hook that strips trailing spaces, NormalizesCamelCase, and imposes Reverse Christmas Notation (or whatever it is). It could even add "FOO:" to the start of the patch summary for any patch which modifies the "FOO" subsystem.

How hard would it be to have an SMTP server on submit.kernel.org which only accepts properly formatted patches addressed to "linux@kernel.org", performs basic compile tests, runs get_maintainer and sends it off to the appropriate places. Then Eager Developer could just "./scripts/config-email", answer two questions, and "git submit" (or whatever) would submit their pride and joy to the correct place. ./scripts/config-email would probably install the pre-commit hooks so that bad white space would never even get to git.
 * (Tim Bird) modify patch or wiggle to accept include lines and variable declarations without context

ideas from LinuxCon NA 2015 BOF
[put stuff here]

discussion from ELCE 2015 BOF
Ideas for discussion or that were discussed:

Company incentives
How to show cost reductions from mainlining:
 * best incentive (and one that works) is when customers request/require upstream drivers
 * this might work for SoC vendor, but it doesn't work for phone vendor, because their customers don't care
 * (maybe the carriers care)
 * have to find other incentive for phone vendor
 * reduced maintenance
 * reduced patch count
 * need to measure patch count for code in-tree and out-of-tree
 * size of driver shrinks
 * zforce - touchscreen driver
 * rockchip clock drivers (was 150K, now using common clock framework)
 * etnaviv driver (50-60k, converted to 3 or 4K)
 * According to robert schwebel, etnaviv driver was reduced to 1/10th it's size in mainlining it
 * freedreno driver
 * any mediatek driver (have their own device model)
 * marvell berlin driver
 * marvell ethernet driver (freeelectrons wrote an ethernet driver, then found one already existing upstream)
 * Marvell uses a HAL - with 10s of thousands of stupid functions that do nothing.
 * Thomas is not sure this that this argument ever convinced a company

Tools for mainlining

 * Easier patch mailing (so any e-mail client can be used)
 * some tool to convert attachments to inline and convert flowed-text to regular text, remove HTML, etc. on the server side
 * Its not only the first submission that matters, followup e-mails can't be HTML, flowed-text, etc.
 * could git send-email support exchange
 * have to make sure exchange doesn't re-write e-mail
 * can we put stuff on github and issue pull requests? (No, maintainer prefers patches on e-mail)
 * add something to github to reformat and convey the patches in preferred format

Best Practices

 * Measure software costs
 * Select hardware with upstream drivers
 * Have a small team, off product treadmill, dedicated to upstreaming
 * Train developers in open source methods
 * Use open source methods internally
 * Mail lists, review, git, etc.
 * Periodically review patches and categorize to: upstream, forward port, drop
 * Post patches to get community feedback
 * Don’t make them perfect before sharing
 * can share even an outline or design
 * doesn't have to be on the mailing list
 * can ask maintainer if that's OK
 * this makes engineers visible - is that a problem?
 * if so, your company stinks
 * it reflects more on the reputation of the individual than the company
 * can have dedicated session with maintainers to solve difficult problems
 * community is interaction between people and not companies
 * conferences and networking
 * has to be mulutiple conferences over time

Technical projects

 * Broadcom wireless driver
 * USB charging
 * Android sensors integration with IIO
 * Uart slave

more notes/other ideas

 * submit SoC code to staging
 * still not sure how to do this
 * once mach code is upstream, is all drivers, and those can go into staging
 * how to get clock drivers or pinctrl into staging
 * could build with external headers, doesn't need to actually run
 * if driver needs to be extended, staging is useless
 * good for coding style cleanups
 * additional obstacles that developers face when mainlining