[libcamera-devel] libcamera AOSP Integration

[Nicholas Roth]

> Out of curiosity, is that the PinePhone Pro, or something else?

PinePhone Pro

> Also, which Android version are you targetting ?

SDK 30

> Could you please disable mail link tracking when posting to the list ?

Yes! Oops. I'm sending this from a corp machine where the extension is not
installed, so hopefully that fixes it.



On Thu, Oct 13, 2022 at 4:09 AM Laurent Pinchart <
laurent.pinchart at ideasonboard.com> wrote:

> Hello,
>
> On Wed, Oct 12, 2022 at 11:11:26AM +0100, Kieran Bingham via
> libcamera-devel wrote:
> > Quoting Nicholas Roth via libcamera-devel (2022-10-12 04:16:03)
> > > Hello all,
> > >
> > > TL;DR: I would propose adding a top-level Android.bp Bazel-based build
> and
> > > tests to support building libcamera inline with AOSP to expose the
> > > libcamera HAL.
> >
> > Thanks for taking this on.
>
> Likewise, thank you for your offer to help supporting the Android build
> system.
>
> As a first comment, Android.bp files are meant for the Soong build
> system, not Bazel. They are similar (and the Android.bp syntax is
> intentionally similar to Bazel), but not identical. See
> https://android.googlesource.com/platform/build/soong/+/master/README.md
> for information about Soong.
>
> Then, to make this even more fun, Android has decided in 2020 to migrate
> to Bazel (https://source.android.com/docs/setup/build/bazel/introduction).
> This is work in progress, I don't know when it will land, and it will
> then deprecate both make and Soong.
>
> > > *Short intro because I'm new* (feel free to skip):
> > > I'm Nicholas, a long-time amateur programmer with ~10 years of industry
> > > experience now, although it doesn't seem like it's been that long. I've
> > > published work in distributed scale-out systems and for the last ~6
> years,
> > > I've worked in AI. More relevant to you all, I have free time and a
> phone
> > > with an RK3399 SoC that I'd like to get supported in Waydroid.
>
> Out of curiosity, is that the PinePhone Pro, or something else ?
>
> Also, which Android version are you targetting ?
>
> > Welcome to the team! :-)
> >
> > > *Background*:
> > > There's been back-and-forth discussion on Kieran's personal GitHub
> account
> > > (link: https://github.com/kbingham/libcamera/pull/26
> > > <
> https://mailtrack.io/trace/link/9c70a621d6594480c04ab3722218fff2dec55196?notrack=1&url=https%3A%2F%2Fgithub.com%2Fkbingham%2Flibcamera%2Fpull%2F26&userId=4192442&signature=13231078163330ff
> >)
>
> Could you please disable mail link tracking when posting to the list ?
>
> > > about how to potentially build libcamera inline with AOSP. If I
> understand
> > > correctly, the original contributor proposed some minor changes and
> also a
> >
> > The minor changes/fixes could certainly be considered already when
> > posted to the list.
>
> Yes, those could be posted, reviewed and merged already, before
> completing the work on the build system.
>

Will do. My fork currently does not build on SDK 30 due to additional
compiler errors. My top priority is to get a working build so I can test
changes.


>
> > > major change that would add ~349 lines of Makefiles, including a
> deprecated
> > > Android.mk. The original contributor ran out of bandwidth recently and
> >
> > The build system is probably the hard part, and indeed it sounds like
> > there are two options.
> >
> > 1) an Android.mk 'wrapper' around the meson infrastructure
> > 2) A parallel build system.
> >
> >
> > 1. may become deprecated, but we don't know when. And the one thing I've
> > learnt about open source is - if you don't know when somethings going to
> > happen, it seems best sometimes to assume it won't until it does.
>
> It is officially deprecated already, what we don't know is when support
> for it will be dropped. It may be possible to find out, I recall seeing
> a public migration schedule plan a while ago, but can't find it at the
> moment. Given that Android is migrating to Bazel, they may not move
> forward with dropping support of make before they switch away from Soong
> too. Or maybe then will. It's hard to predict, they're not known for
> taking anybody but themselves into consideration in their plans.
>
> > Because it can be *far away in the future*  (Just like our first
> > libcamera stable API release :D)
>
> As long as an Android.mk wrapper around meson would be low maintenance,
> I don't mind having support for the deprecated make-based build system
> for the time being. Of course, if there was a way to nicely support
> Soong-based builds without a high maintenance effort, that would be
> better, but a "one step at a time" approach is fine with me.
>

>From what I'm seeing so far, I'm assuming we'll start with a Makefile-based
approach. That's what I'm using now.

>
> > 2. Is painful to maintain... but sounds like the route you are
> > interested in...
>
> I don't think we want to maintain two build systems in parallel. A
> proper integrating with Soong or Bazel would require bridging the gap
> between those and meson, as we have no plan to move away from meson for
> the time being.
>
> For Soong, the little I understand is that the Android.bp files are
> fully declarative, and any logic needs to be written in Soong plugins,
> in the Go language. As libcamera generates source files at build time,
> this generation logic would need to be implemented there. A meson
> developer once (during a session about the Android build system at the
> Linux Plumbers Conference a few years ago) publicly offered to write a
> meson frontend in Go, for integration with Soong (a meson frontend means
> a reimplementation of meson in Go, which would parse the meson.build
> files and output the data format expected by Soong to let the Soong
> backend generate ninja files).  The offer was rejected by the Android
> build system maintainer along the lines of "you can do whatever you
> want, we won't care, so it will likely break very quickly as we move
> forward". That wasn't very motivating.
>
> > > decided not to pursue their changes further, and I volunteered to carry
> > > this forward.
> > >
> > > *My Prior Work*
> > > I have tried building libcamera with these proposed changes inline with
> > > AOSP using sdk 30 under "/external/libcamera" (full procedure link:
> > >
> https://docs.google.com/document/d/1Sly_VH3w6wFIdE72WXijQegoHZh8IxEnJ9m0hH7GodU/edit
> > > <
> https://mailtrack.io/trace/link/e6ac3150dcbfa009e3880aa030d8176036dc2199?notrack=1&url=https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1Sly_VH3w6wFIdE72WXijQegoHZh8IxEnJ9m0hH7GodU%2Fedit&userId=4192442&signature=abdf0877da91078c
> >)
> > > and it does not work as intended as far as I can tell. Specifically,
> > > /vendor/lib/hw/camera.libcamera.so
> > > <
> https://mailtrack.io/trace/link/f38ddeb643e9e502355e6d6c7078c4f678867495?url=http%3A%2F%2Fcamera.libcamera.so%2F&userId=4192442&signature=f3a416f93f49123b
> >
> > > does
> > > not get bundled into the Android images. This may be user error on my
> part,
> > > however.
> > >
> > > *Options for AOSP Inline Build*
> > > The way I can see, we have at least these options for building
> libcamera
> > > inline with AOSP. Personally, parallel build systems (see below) seems
> like
> > > the best option to me. Please feel free to jump in if I'm wrong about
> any
> > > of this or if you have options to add:
> > >
> > >    - Transition libcamera to Soong and build intuitively
> > >    Tradeoffs: Simplest and most consistent option, but potentially very
> > >    disruptive
>
> As Soong is already deprecated by Android given their move to Bazel,
> that's a dead end. Even if it wasn't the case, I don't think it would be
> an option, as Soong is Android-specific.
>
> > >    - Invoke Meson from Soong with a genrule, akin to what
> meson_cross.mk
> > >    <
> https://mailtrack.io/trace/link/8b8bf21362b7febf0a481b5bf274ad86d982180e?url=http%3A%2F%2Fmeson_cross.mk%2F&userId=4192442&signature=c34c1a84f1c76a29
> >
> > >     does
> > >    Tradeoffs: complex-- I would be happy to do this but I'll need help
> > >    understanding how meson_cross.mk
> > >    <
> https://mailtrack.io/trace/link/35387558b6d890dee771bdbfb1f7bf8f6132b75f?url=http%3A%2F%2Fmeson_cross.mk%2F&userId=4192442&signature=dfc9af2da8b37fa2
> >
> > > works;
> > >    probably harder than writing meson_cross.mk
> > >    <
> https://mailtrack.io/trace/link/fecb2bde4cbf9c1d4e4afb58415434f6bf142167?url=http%3A%2F%2Fmeson_cross.mk%2F&userId=4192442&signature=3458951a34f69bc0
> >
> > > was;
> > >    possibly infeasible since Soong is quite opinionated
>
> As far as I can tell it's not really possible indeed. As I wrote above,
> you'd need a Soong plugin in Go that generates the internal data format
> that Soong requires. It may be possible to have that plugin call the
> meson parser written in Python, but writing a meson frontend in Go may
> be easier. meson doesn't guarantee the stability of the API between its
> Python frontend and Python backends as far as I know.
>
> > >    - *Parallel build systems*-- Create an Android.bp file that would
> live
> > >    at the top-level of the libcamer repo in addition to the existing
> Meson
> > >    files. We would use Soong for Android, and Meson for other
> use-cases.
> > >    Tradeoffs: Less complex and less error-prone than invoking Meson
> from
> > >    Soong, and less disruptive than moving to Soong exclusively, but
> would
> > >    incur an increased maintenance burden compared to doing nothing or
> using a
> > >    single build system.
>
> This is the option I like the least, as it increases the maintenance
> burden and is pretty much guaranteed to lead to breakages.
>
> > >    - Android.mk-- Use a legacy (link:
> > >    https://source.android.com/docs/setup/build
> > >    <
> https://mailtrack.io/trace/link/a590ddf9d4e0fc2459ff68e2943bcb5feb034056?notrack=1&url=https%3A%2F%2Fsource.android.com%2Fdocs%2Fsetup%2Fbuild&userId=4192442&signature=124f558360d74d95
> >)
> > >    Android.mk file and figure out how to get the proposed Android.mk
> and
> > >    meson_cross.mk
> > >    <
> https://mailtrack.io/trace/link/b47795745b452c00239c6966c5fabd90e9ff7a07?url=http%3A%2F%2Fmeson_cross.mk%2F&userId=4192442&signature=1528cb5f2f1cd6c7
> >
> > > files
> > >    working and tested.
> > >    Tradeoffs: New code would depend on a legacy tool and Makefiles can
> be
> > >    error-prone and hard to read (not great), but the maintenance burden
> > >    *might* be lower than creating a parallel Android.bp build file.
> >
> > Of the above options, the only one that I don't think we could easily
> > merge would be a parallel build system. That would be difficult to
> > maintain.
>
> Another option which isn't listed above is to compile libcamera
> separately using meson, with the Android SDK providing the
> cross-compiler. This would be a manual process though, not integrated in
> the AOSP build system. This is the option that mesa has selected as far
> as I know, they faced the same issue as us as they use meson. They still
> support the historical make-based Android build system
> (https://gitlab.freedesktop.org/mesa/mesa/-/tree/main/android), if that
> doesn't work for any reason, then users have to build mesa manually.
>
> We also asked, in the same Android build system session at LPC that I
> mentioned above, if Android could provide a hook in their build system
> to ease integration of externally-compiled binaries in Android images.
> The answer was negative.
>
> > A wrapper around our build system with either an Android.mk or a Soong
> > genrule (or both!?) would be more reasonable in my opinion ... as long
> > as :
> >
> > > *Test Plan*
> >
> > We have a way to at least compile test it ;-)
>

Working on that right now! Looks like this actually has a few compile
errors. Will ask for help on this list when/if I get stuck, if that's
alright.


> >
> > > I'm not exactly sure how the libcamera community approaches testing.
> That
> > > said, I would write a sanity check based on
> > > https://source.android.com/docs/core/tests/development/blueprints
> > > <
> https://mailtrack.io/trace/link/566af31c86d4ae27635d28c2f28a21e50ddd700b?notrack=1&url=https%3A%2F%2Fsource.android.com%2Fdocs%2Fcore%2Ftests%2Fdevelopment%2Fblueprints&userId=4192442&signature=76612081ac230acc
> >.
> > > Ideally, we would agree to run this before submitting new code. This is
> > > probably the weakest part of my proposal, and I'd love any resources on
> > > libcamera's testing philosophy and thoughts on how to make this
> stronger.
> >
> > We already have infrastructure that builds libcamera across various GCC
> > and clang versions. It also runs CTS (Chrome Test Suite) on a Chromebook
> > (IPU3) which tests the Android HAL layer.
> >
> > I would extend this to support compiling with either the Android.mk or
> > genrule wrappers (again, or both) ... as long as I'm guided on what is
> > required. I have a build machine with plenty of space and resources for
> > such a task.
> >
> > > *Next Steps*
> > > If this looks good to the community (parallel build systems option),
> I'll
> > > figure out how to make Android.bp work. Most likely though, we have
> more to
> > > discuss. I'm ready to hear your thoughts!
> >
> > Parallel build is the one that worries me. libcamera is already complex
> > to build, with many separate and sometimes optional components.
> >
> > Will it be a single file that covers our entire build? as well as the
> > configuration options for different platforms, and specially built IPA
> > proxies, and module signing? That sounds ... like a lot of work, and a
> > lot of maintenance.
> >
> > But now we're on the libcamera list - lets await more feedback.
>
> On thing that could already be done without waiting for the outcome of
> this discussion is to post, review and merge the patches that fix build
> issues with Android (I'm thinking about removing usage of
> std::filesystem for instance).
>

I would like to have a working inline build before I post these. Currently,
"activeState.agc.exposure = 10ms / configuration.sensor.lineDuration;" in
https://github.com/rothn/libcamera/blob/aosp-inline-build/src/ipa/ipu3/algorithms/agc.cpp
causes
a compiler error with SDK30 because libcamera::utils::Duration and `long
long` have no common type. Not sure if it's a compiler bug or what, but I'm
looking into it. I'm setting up my build environment based on these
instructions:
https://docs.google.com/document/d/1Sly_VH3w6wFIdE72WXijQegoHZh8IxEnJ9m0hH7GodU/edit
if
it interests anyone.

>
> --
> Regards,
>
> Laurent Pinchart
>

Thanks, and looking forward to more feedback!

-Nicholas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libcamera.org/pipermail/libcamera-devel/attachments/20221013/8928efa9/attachment.htm>