[libcamera-devel] libcamera AOSP Integration

[Nicholas Roth]

> If possible, please report the exact compiler version and flags used to
reproduce the problem.
Posted as a bug report for posterity:
https://bugs.libcamera.org/show_bug.cgi?id=156

> Is the NDK/SDK available without a full build of AOSP
I believe so, but I know very little about Android. If I recall correctly,
developers use the standalone NDK to compile their apps.

> It would be nice to have a clearly documented procedure.
This works for me (and I've had feedback that it works for others too), but
it's for a full build:
https://docs.google.com/document/d/1Sly_VH3w6wFIdE72WXijQegoHZh8IxEnJ9m0hH7GodU/edit.
Even though it's not what you want, it may be a good place to start. My
focus is on getting the code to compile, but if someone who wants to spend
time fixing the compile errors strongly prefers to start with a standalone
NDK-based build, I can invest some time into figuring that out.

Thanks again everyone!

-Nicholas

On Sat, Oct 15, 2022 at 11:41 AM Laurent Pinchart <
laurent.pinchart at ideasonboard.com> wrote:

> On Fri, Oct 14, 2022 at 10:46:56AM +0100, Kieran Bingham wrote:
> > Quoting Nicholas Roth (2022-10-14 01:35:37)
> > > > Out of curiosity, is that the PinePhone Pro, or something else?
> > >
> > > PinePhone Pro
> > >
> > > > Also, which Android version are you targetting ?
> > >
> > > SDK 30
> > >
> > > > Could you please disable mail link tracking when posting to the list
> ?
> > >
> > > Yes! Oops. I'm sending this from a corp machine where the extension is
> not
> > > installed, so hopefully that fixes it.
> > >
> > > On Thu, Oct 13, 2022 at 4:09 AM Laurent Pinchart wrote:
> > > > On Wed, Oct 12, 2022 at 11:11:26AM +0100, Kieran Bingham via
> > > > libcamera-devel wrote:
> > > > > Quoting Nicholas Roth via libcamera-devel (2022-10-12 04:16:03)
> > > > > > Hello all,
> > > > > >
> > > > > > TL;DR: I would propose adding a top-level Android.bp Bazel-based
> build and
> > > > > > tests to support building libcamera inline with AOSP to expose
> the
> > > > > > libcamera HAL.
> > > > >
> > > > > Thanks for taking this on.
> > > >
> > > > Likewise, thank you for your offer to help supporting the Android
> build
> > > > system.
> > > >
> > > > As a first comment, Android.bp files are meant for the Soong build
> > > > system, not Bazel. They are similar (and the Android.bp syntax is
> > > > intentionally similar to Bazel), but not identical. See
> > > >
> https://android.googlesource.com/platform/build/soong/+/master/README.md
> > > > for information about Soong.
> > > >
> > > > Then, to make this even more fun, Android has decided in 2020 to
> migrate
> > > > to Bazel (
> https://source.android.com/docs/setup/build/bazel/introduction).
> > > > This is work in progress, I don't know when it will land, and it will
> > > > then deprecate both make and Soong.
> > > >
> > > > > > *Short intro because I'm new* (feel free to skip):
> > > > > > I'm Nicholas, a long-time amateur programmer with ~10 years of
> industry
> > > > > > experience now, although it doesn't seem like it's been that
> long. I've
> > > > > > published work in distributed scale-out systems and for the last
> ~6 years,
> > > > > > I've worked in AI. More relevant to you all, I have free time
> and a phone
> > > > > > with an RK3399 SoC that I'd like to get supported in Waydroid.
> > > >
> > > > Out of curiosity, is that the PinePhone Pro, or something else ?
> > > >
> > > > Also, which Android version are you targetting ?
> > > >
> > > > > Welcome to the team! :-)
> > > > >
> > > > > > *Background*:
> > > > > > There's been back-and-forth discussion on Kieran's personal
> GitHub account
> > > > > > (link: https://github.com/kbingham/libcamera/pull/26)
> > > >
> > > > Could you please disable mail link tracking when posting to the list
> ?
> > > >
> > > > > > about how to potentially build libcamera inline with AOSP. If I
> understand
> > > > > > correctly, the original contributor proposed some minor changes
> and also a
> > > > >
> > > > > The minor changes/fixes could certainly be considered already when
> > > > > posted to the list.
> > > >
> > > > Yes, those could be posted, reviewed and merged already, before
> > > > completing the work on the build system.
> > >
> > > Will do. My fork currently does not build on SDK 30 due to additional
> > > compiler errors. My top priority is to get a working build so I can
> test
> > > changes.
> > >
> > > > > > major change that would add ~349 lines of Makefiles, including a
> deprecated
> > > > > > Android.mk. The original contributor ran out of bandwidth
> recently and
> > > > >
> > > > > The build system is probably the hard part, and indeed it sounds
> like
> > > > > there are two options.
> > > > >
> > > > > 1) an Android.mk 'wrapper' around the meson infrastructure
> > > > > 2) A parallel build system.
> > > > >
> > > > >
> > > > > 1. may become deprecated, but we don't know when. And the one
> thing I've
> > > > > learnt about open source is - if you don't know when somethings
> going to
> > > > > happen, it seems best sometimes to assume it won't until it does.
> > > >
> > > > It is officially deprecated already, what we don't know is when
> support
> > > > for it will be dropped. It may be possible to find out, I recall
> seeing
> > > > a public migration schedule plan a while ago, but can't find it at
> the
> > > > moment. Given that Android is migrating to Bazel, they may not move
> > > > forward with dropping support of make before they switch away from
> Soong
> > > > too. Or maybe then will. It's hard to predict, they're not known for
> > > > taking anybody but themselves into consideration in their plans.
> > > >
> > > > > Because it can be *far away in the future*  (Just like our first
> > > > > libcamera stable API release :D)
> > > >
> > > > As long as an Android.mk wrapper around meson would be low
> maintenance,
> > > > I don't mind having support for the deprecated make-based build
> system
> > > > for the time being. Of course, if there was a way to nicely support
> > > > Soong-based builds without a high maintenance effort, that would be
> > > > better, but a "one step at a time" approach is fine with me.
> > >
> > > From what I'm seeing so far, I'm assuming we'll start with a
> Makefile-based
> > > approach. That's what I'm using now.
>
> While I wish we could already target something that would be more
> future-proof, this is likely the most sensible approach for now. There's
> little point in spending time on supporting the Soong build system as
> it's already deprecated.
>
> > > > > 2. Is painful to maintain... but sounds like the route you are
> > > > > interested in...
> > > >
> > > > I don't think we want to maintain two build systems in parallel. A
> > > > proper integrating with Soong or Bazel would require bridging the gap
> > > > between those and meson, as we have no plan to move away from meson
> for
> > > > the time being.
> > > >
> > > > For Soong, the little I understand is that the Android.bp files are
> > > > fully declarative, and any logic needs to be written in Soong
> plugins,
> > > > in the Go language. As libcamera generates source files at build
> time,
> > > > this generation logic would need to be implemented there. A meson
> > > > developer once (during a session about the Android build system at
> the
> > > > Linux Plumbers Conference a few years ago) publicly offered to write
> a
> > > > meson frontend in Go, for integration with Soong (a meson frontend
> means
> > > > a reimplementation of meson in Go, which would parse the meson.build
> > > > files and output the data format expected by Soong to let the Soong
> > > > backend generate ninja files).  The offer was rejected by the Android
> > > > build system maintainer along the lines of "you can do whatever you
> > > > want, we won't care, so it will likely break very quickly as we move
> > > > forward". That wasn't very motivating.
> > > >
> > > > > > decided not to pursue their changes further, and I volunteered
> to carry
> > > > > > this forward.
> > > > > >
> > > > > > *My Prior Work*
> > > > > > I have tried building libcamera with these proposed changes
> inline with
> > > > > > AOSP using sdk 30 under "/external/libcamera" (full procedure
> link:
> > > > > >
> https://docs.google.com/document/d/1Sly_VH3w6wFIdE72WXijQegoHZh8IxEnJ9m0hH7GodU/edit
> )
> > > > > > and it does not work as intended as far as I can tell.
> Specifically,
> > > > > > /vendor/lib/hw/camera.libcamera.so does
> > > > > > not get bundled into the Android images. This may be user error
> on my part,
> > > > > > however.
> > > > > >
> > > > > > *Options for AOSP Inline Build*
> > > > > > The way I can see, we have at least these options for building
> libcamera
> > > > > > inline with AOSP. Personally, parallel build systems (see below)
> seems like
> > > > > > the best option to me. Please feel free to jump in if I'm wrong
> about any
> > > > > > of this or if you have options to add:
> > > > > >
> > > > > >    - Transition libcamera to Soong and build intuitively
> > > > > >    Tradeoffs: Simplest and most consistent option, but
> potentially very
> > > > > >    disruptive
> > > >
> > > > As Soong is already deprecated by Android given their move to Bazel,
> > > > that's a dead end. Even if it wasn't the case, I don't think it
> would be
> > > > an option, as Soong is Android-specific.
> > > >
> > > > > >    - Invoke Meson from Soong with a genrule, akin to what
> meson_cross.mk
> > > > > >     does
> > > > > >    Tradeoffs: complex-- I would be happy to do this but I'll
> need help
> > > > > >    understanding how meson_cross.mk works;
> > > > > >    probably harder than writing meson_cross.mk was;
> > > > > >    possibly infeasible since Soong is quite opinionated
> > > >
> > > > As far as I can tell it's not really possible indeed. As I wrote
> above,
> > > > you'd need a Soong plugin in Go that generates the internal data
> format
> > > > that Soong requires. It may be possible to have that plugin call the
> > > > meson parser written in Python, but writing a meson frontend in Go
> may
> > > > be easier. meson doesn't guarantee the stability of the API between
> its
> > > > Python frontend and Python backends as far as I know.
> > > >
> > > > > >    - *Parallel build systems*-- Create an Android.bp file that
> would live
> > > > > >    at the top-level of the libcamer repo in addition to the
> existing Meson
> > > > > >    files. We would use Soong for Android, and Meson for other
> use-cases.
> > > > > >    Tradeoffs: Less complex and less error-prone than invoking
> Meson from
> > > > > >    Soong, and less disruptive than moving to Soong exclusively,
> but would
> > > > > >    incur an increased maintenance burden compared to doing
> nothing or using a
> > > > > >    single build system.
> > > >
> > > > This is the option I like the least, as it increases the maintenance
> > > > burden and is pretty much guaranteed to lead to breakages.
> > > >
> > > > > >    - Android.mk-- Use a legacy (link:
> https://source.android.com/docs/setup/build)
> > > > > >    Android.mk file and figure out how to get the proposed
> Android.mk and
> > > > > >    meson_cross.mk files working and tested.
> > > > > >    Tradeoffs: New code would depend on a legacy tool and
> Makefiles can be
> > > > > >    error-prone and hard to read (not great), but the maintenance
> burden
> > > > > >    *might* be lower than creating a parallel Android.bp build
> file.
> > > > >
> > > > > Of the above options, the only one that I don't think we could
> easily
> > > > > merge would be a parallel build system. That would be difficult to
> > > > > maintain.
> > > >
> > > > Another option which isn't listed above is to compile libcamera
> > > > separately using meson, with the Android SDK providing the
> > > > cross-compiler. This would be a manual process though, not
> integrated in
> > > > the AOSP build system. This is the option that mesa has selected as
> far
> > > > as I know, they faced the same issue as us as they use meson. They
> still
> > > > support the historical make-based Android build system
> > > > (https://gitlab.freedesktop.org/mesa/mesa/-/tree/main/android), if
> that
> > > > doesn't work for any reason, then users have to build mesa manually.
> > > >
> > > > We also asked, in the same Android build system session at LPC that I
> > > > mentioned above, if Android could provide a hook in their build
> system
> > > > to ease integration of externally-compiled binaries in Android
> images.
> > > > The answer was negative.
> > > >
> > > > > A wrapper around our build system with either an Android.mk or a
> Soong
> > > > > genrule (or both!?) would be more reasonable in my opinion ... as
> long
> > > > > as :
> > > > >
> > > > > > *Test Plan*
> > > > >
> > > > > We have a way to at least compile test it ;-)
> > >
> > > Working on that right now! Looks like this actually has a few compile
> > > errors. Will ask for help on this list when/if I get stuck, if that's
> > > alright.
> >
> > Certainly, feel free to post the compile error messages as a distinct
> > thread - or add them to a bug on bugs.libcamera.org
> >
> > If it is a compile failure we'd want to make sure that's fixed.
>
> If possible, please report the exact compiler version and flags used to
> reproduce the problem.
>
> > > > > > I'm not exactly sure how the libcamera community approaches
> testing. That
> > > > > > said, I would write a sanity check based on
> > > > > >
> https://source.android.com/docs/core/tests/development/blueprints.
> > > > > > Ideally, we would agree to run this before submitting new code.
> This is
> > > > > > probably the weakest part of my proposal, and I'd love any
> resources on
> > > > > > libcamera's testing philosophy and thoughts on how to make this
> stronger.
> > > > >
> > > > > We already have infrastructure that builds libcamera across
> various GCC
> > > > > and clang versions. It also runs CTS (Chrome Test Suite) on a
> Chromebook
> > > > > (IPU3) which tests the Android HAL layer.
> > > > >
> > > > > I would extend this to support compiling with either the
> Android.mk or
> > > > > genrule wrappers (again, or both) ... as long as I'm guided on
> what is
> > > > > required. I have a build machine with plenty of space and
> resources for
> > > > > such a task.
> > > > >
> > > > > > *Next Steps*
> > > > > > If this looks good to the community (parallel build systems
> option), I'll
> > > > > > figure out how to make Android.bp work. Most likely though, we
> have more to
> > > > > > discuss. I'm ready to hear your thoughts!
> > > > >
> > > > > Parallel build is the one that worries me. libcamera is already
> complex
> > > > > to build, with many separate and sometimes optional components.
> > > > >
> > > > > Will it be a single file that covers our entire build? as well as
> the
> > > > > configuration options for different platforms, and specially built
> IPA
> > > > > proxies, and module signing? That sounds ... like a lot of work,
> and a
> > > > > lot of maintenance.
> > > > >
> > > > > But now we're on the libcamera list - lets await more feedback.
> > > >
> > > > On thing that could already be done without waiting for the outcome
> of
> > > > this discussion is to post, review and merge the patches that fix
> build
> > > > issues with Android (I'm thinking about removing usage of
> > > > std::filesystem for instance).
> > >
> > > I would like to have a working inline build before I post these.
> Currently,
> > > "activeState.agc.exposure = 10ms / configuration.sensor.lineDuration;"
> in
> > >
> https://github.com/rothn/libcamera/blob/aosp-inline-build/src/ipa/ipu3/algorithms/agc.cpp
> > > causes
> > > a compiler error with SDK30 because libcamera::utils::Duration and
> `long
> > > long` have no common type. Not sure if it's a compiler bug or what,
> but I'm
> >
> > Interesting, this seems related (but not identical) to another
> > patch that was posted recently:
> >
> >  https://patchwork.libcamera.org/patch/17570/
> >
> >
> > """
> > From: Olivier Gayot <olivier.gayot at canonical.com>
> >
> > On ppc64el, GCC fails to determine the result of some long double
> > expressions at compile time. This makes libcamera fail to build with GCC
> > on that architecture.
> >
> > e.g.:
> >
> > constexpr auto x = 1.0l/30.0;
> >
> > in ‘constexpr’ expansion of ‘std::chrono::operator/<long double,
> std::ratio<1>,
> >   double>(std::literals::chrono_literals::operator""s(1.0e+0l), 6.0e+1)’
> > /usr/include/c++/11/chrono:710:39: error: ‘(1.0e+0l / 6.0e+1)’ is not a
> constant expression
> >   710 |         return __cd(__cd(__d).count() / __s);
> >
> > Tweaking the expressions just a bit makes GCC happy and allows libcamera
> > to build properly on ppc64el.
> >
> > Signed-off-by: Olivier Gayot <olivier.gayot at canonical.com>
> > ---
> >  src/ipa/raspberrypi/raspberrypi.cpp | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/src/ipa/raspberrypi/raspberrypi.cpp
> b/src/ipa/raspberrypi/raspberrypi.cpp
> > index 14b06a4f..3e4383f9 100644
> > --- a/src/ipa/raspberrypi/raspberrypi.cpp
> > +++ b/src/ipa/raspberrypi/raspberrypi.cpp
> > @@ -60,7 +60,7 @@  using utils::Duration;
> >  /* Configure the sensor with these values initially. */
> >  constexpr double defaultAnalogueGain = 1.0;
> >  constexpr Duration defaultExposureTime = 20.0ms;
> > -constexpr Duration defaultMinFrameDuration = 1.0s / 30.0;
> > +constexpr Duration defaultMinFrameDuration = 1s / 30.0;
> >  constexpr Duration defaultMaxFrameDuration = 250.0s;
> >
> >  /*
> > @@ -69,7 +69,7 @@  constexpr Duration defaultMaxFrameDuration = 250.0s;
> >   * we rate-limit the controller Prepare() and Process() calls to lower
> than or
> >   * equal to this rate.
> >   */
> > -constexpr Duration controllerMinFrameDuration = 1.0s / 30.0;
> > +constexpr Duration controllerMinFrameDuration = 1s / 30.0;
> >
> >  /* List of controls handled by the Raspberry Pi IPA */
> >  static const ControlInfoMap::Map ipaControls{
> > """
>
> I've just reviewed that patch, and I don't think it's related.
>
> > > looking into it. I'm setting up my build environment based on these
> > > instructions:
> > >
> https://docs.google.com/document/d/1Sly_VH3w6wFIdE72WXijQegoHZh8IxEnJ9m0hH7GodU/edit
> > > if it interests anyone.
> >
> > I'd like to know the quickest/easiest way to obtain the SDK and
> > replicate the build for the environment.
> >
> > Is the NDK/SDK available without a full build of AOSP  (If not that's
> > ok, but I'd be keen to know if there's a shortcut just to be able to
> > track the compiling of 'just' libcamera).
> >
> > Building for waydroid looks interesting though - and could mean a way to
> > test libcamera on more platforms easily.
>
> I gave it a brief try by following
>
> https://proandroiddev.com/how-to-setup-android-sdk-without-android-studio-6d60d0f2812a?gi=28d85ab4cc0c
> to setup the SDK, and wrote a meson cross file by hand. It failed at
> meson setup stage, when trying to configure cmake-based subprojects. I
> found https://github.com/ppetraki/meson-android-helloworld that may be
> of interest, but haven't investigated. It would be nice to have a
> clearly documented procedure.
>
> --
> Regards,
>
> Laurent Pinchart
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libcamera.org/pipermail/libcamera-devel/attachments/20221015/b54d0e36/attachment.htm>