[libcamera-devel] libcamera AOSP Integration

[Kieran Bingham]

Quoting Nicholas Roth (2022-10-14 01:35:37)
> > Out of curiosity, is that the PinePhone Pro, or something else?
> 
> PinePhone Pro
> 
> > Also, which Android version are you targetting ?
> 
> SDK 30
> 
> > Could you please disable mail link tracking when posting to the list ?
> 
> Yes! Oops. I'm sending this from a corp machine where the extension is not
> installed, so hopefully that fixes it.
> 
> 
> 
> On Thu, Oct 13, 2022 at 4:09 AM Laurent Pinchart <
> laurent.pinchart at ideasonboard.com> wrote:
> 
> > Hello,
> >
> > On Wed, Oct 12, 2022 at 11:11:26AM +0100, Kieran Bingham via
> > libcamera-devel wrote:
> > > Quoting Nicholas Roth via libcamera-devel (2022-10-12 04:16:03)
> > > > Hello all,
> > > >
> > > > TL;DR: I would propose adding a top-level Android.bp Bazel-based build
> > and
> > > > tests to support building libcamera inline with AOSP to expose the
> > > > libcamera HAL.
> > >
> > > Thanks for taking this on.
> >
> > Likewise, thank you for your offer to help supporting the Android build
> > system.
> >
> > As a first comment, Android.bp files are meant for the Soong build
> > system, not Bazel. They are similar (and the Android.bp syntax is
> > intentionally similar to Bazel), but not identical. See
> > https://android.googlesource.com/platform/build/soong/+/master/README.md
> > for information about Soong.
> >
> > Then, to make this even more fun, Android has decided in 2020 to migrate
> > to Bazel (https://source.android.com/docs/setup/build/bazel/introduction).
> > This is work in progress, I don't know when it will land, and it will
> > then deprecate both make and Soong.
> >
> > > > *Short intro because I'm new* (feel free to skip):
> > > > I'm Nicholas, a long-time amateur programmer with ~10 years of industry
> > > > experience now, although it doesn't seem like it's been that long. I've
> > > > published work in distributed scale-out systems and for the last ~6
> > years,
> > > > I've worked in AI. More relevant to you all, I have free time and a
> > phone
> > > > with an RK3399 SoC that I'd like to get supported in Waydroid.
> >
> > Out of curiosity, is that the PinePhone Pro, or something else ?
> >
> > Also, which Android version are you targetting ?
> >
> > > Welcome to the team! :-)
> > >
> > > > *Background*:
> > > > There's been back-and-forth discussion on Kieran's personal GitHub
> > account
> > > > (link: https://github.com/kbingham/libcamera/pull/26
> > > > <
> > https://mailtrack.io/trace/link/9c70a621d6594480c04ab3722218fff2dec55196?notrack=1&url=https%3A%2F%2Fgithub.com%2Fkbingham%2Flibcamera%2Fpull%2F26&userId=4192442&signature=13231078163330ff
> > >)
> >
> > Could you please disable mail link tracking when posting to the list ?
> >
> > > > about how to potentially build libcamera inline with AOSP. If I
> > understand
> > > > correctly, the original contributor proposed some minor changes and
> > also a
> > >
> > > The minor changes/fixes could certainly be considered already when
> > > posted to the list.
> >
> > Yes, those could be posted, reviewed and merged already, before
> > completing the work on the build system.
> >
> 
> Will do. My fork currently does not build on SDK 30 due to additional
> compiler errors. My top priority is to get a working build so I can test
> changes.
> 
> 
> >
> > > > major change that would add ~349 lines of Makefiles, including a
> > deprecated
> > > > Android.mk. The original contributor ran out of bandwidth recently and
> > >
> > > The build system is probably the hard part, and indeed it sounds like
> > > there are two options.
> > >
> > > 1) an Android.mk 'wrapper' around the meson infrastructure
> > > 2) A parallel build system.
> > >
> > >
> > > 1. may become deprecated, but we don't know when. And the one thing I've
> > > learnt about open source is - if you don't know when somethings going to
> > > happen, it seems best sometimes to assume it won't until it does.
> >
> > It is officially deprecated already, what we don't know is when support
> > for it will be dropped. It may be possible to find out, I recall seeing
> > a public migration schedule plan a while ago, but can't find it at the
> > moment. Given that Android is migrating to Bazel, they may not move
> > forward with dropping support of make before they switch away from Soong
> > too. Or maybe then will. It's hard to predict, they're not known for
> > taking anybody but themselves into consideration in their plans.
> >
> > > Because it can be *far away in the future*  (Just like our first
> > > libcamera stable API release :D)
> >
> > As long as an Android.mk wrapper around meson would be low maintenance,
> > I don't mind having support for the deprecated make-based build system
> > for the time being. Of course, if there was a way to nicely support
> > Soong-based builds without a high maintenance effort, that would be
> > better, but a "one step at a time" approach is fine with me.
> >
> 
> From what I'm seeing so far, I'm assuming we'll start with a Makefile-based
> approach. That's what I'm using now.
> 
> >
> > > 2. Is painful to maintain... but sounds like the route you are
> > > interested in...
> >
> > I don't think we want to maintain two build systems in parallel. A
> > proper integrating with Soong or Bazel would require bridging the gap
> > between those and meson, as we have no plan to move away from meson for
> > the time being.
> >
> > For Soong, the little I understand is that the Android.bp files are
> > fully declarative, and any logic needs to be written in Soong plugins,
> > in the Go language. As libcamera generates source files at build time,
> > this generation logic would need to be implemented there. A meson
> > developer once (during a session about the Android build system at the
> > Linux Plumbers Conference a few years ago) publicly offered to write a
> > meson frontend in Go, for integration with Soong (a meson frontend means
> > a reimplementation of meson in Go, which would parse the meson.build
> > files and output the data format expected by Soong to let the Soong
> > backend generate ninja files).  The offer was rejected by the Android
> > build system maintainer along the lines of "you can do whatever you
> > want, we won't care, so it will likely break very quickly as we move
> > forward". That wasn't very motivating.
> >
> > > > decided not to pursue their changes further, and I volunteered to carry
> > > > this forward.
> > > >
> > > > *My Prior Work*
> > > > I have tried building libcamera with these proposed changes inline with
> > > > AOSP using sdk 30 under "/external/libcamera" (full procedure link:
> > > >
> > https://docs.google.com/document/d/1Sly_VH3w6wFIdE72WXijQegoHZh8IxEnJ9m0hH7GodU/edit
> > > > <
> > https://mailtrack.io/trace/link/e6ac3150dcbfa009e3880aa030d8176036dc2199?notrack=1&url=https%3A%2F%2Fdocs.google.com%2Fdocument%2Fd%2F1Sly_VH3w6wFIdE72WXijQegoHZh8IxEnJ9m0hH7GodU%2Fedit&userId=4192442&signature=abdf0877da91078c
> > >)
> > > > and it does not work as intended as far as I can tell. Specifically,
> > > > /vendor/lib/hw/camera.libcamera.so
> > > > <
> > https://mailtrack.io/trace/link/f38ddeb643e9e502355e6d6c7078c4f678867495?url=http%3A%2F%2Fcamera.libcamera.so%2F&userId=4192442&signature=f3a416f93f49123b
> > >
> > > > does
> > > > not get bundled into the Android images. This may be user error on my
> > part,
> > > > however.
> > > >
> > > > *Options for AOSP Inline Build*
> > > > The way I can see, we have at least these options for building
> > libcamera
> > > > inline with AOSP. Personally, parallel build systems (see below) seems
> > like
> > > > the best option to me. Please feel free to jump in if I'm wrong about
> > any
> > > > of this or if you have options to add:
> > > >
> > > >    - Transition libcamera to Soong and build intuitively
> > > >    Tradeoffs: Simplest and most consistent option, but potentially very
> > > >    disruptive
> >
> > As Soong is already deprecated by Android given their move to Bazel,
> > that's a dead end. Even if it wasn't the case, I don't think it would be
> > an option, as Soong is Android-specific.
> >
> > > >    - Invoke Meson from Soong with a genrule, akin to what
> > meson_cross.mk
> > > >    <
> > https://mailtrack.io/trace/link/8b8bf21362b7febf0a481b5bf274ad86d982180e?url=http%3A%2F%2Fmeson_cross.mk%2F&userId=4192442&signature=c34c1a84f1c76a29
> > >
> > > >     does
> > > >    Tradeoffs: complex-- I would be happy to do this but I'll need help
> > > >    understanding how meson_cross.mk
> > > >    <
> > https://mailtrack.io/trace/link/35387558b6d890dee771bdbfb1f7bf8f6132b75f?url=http%3A%2F%2Fmeson_cross.mk%2F&userId=4192442&signature=dfc9af2da8b37fa2
> > >
> > > > works;
> > > >    probably harder than writing meson_cross.mk
> > > >    <
> > https://mailtrack.io/trace/link/fecb2bde4cbf9c1d4e4afb58415434f6bf142167?url=http%3A%2F%2Fmeson_cross.mk%2F&userId=4192442&signature=3458951a34f69bc0
> > >
> > > > was;
> > > >    possibly infeasible since Soong is quite opinionated
> >
> > As far as I can tell it's not really possible indeed. As I wrote above,
> > you'd need a Soong plugin in Go that generates the internal data format
> > that Soong requires. It may be possible to have that plugin call the
> > meson parser written in Python, but writing a meson frontend in Go may
> > be easier. meson doesn't guarantee the stability of the API between its
> > Python frontend and Python backends as far as I know.
> >
> > > >    - *Parallel build systems*-- Create an Android.bp file that would
> > live
> > > >    at the top-level of the libcamer repo in addition to the existing
> > Meson
> > > >    files. We would use Soong for Android, and Meson for other
> > use-cases.
> > > >    Tradeoffs: Less complex and less error-prone than invoking Meson
> > from
> > > >    Soong, and less disruptive than moving to Soong exclusively, but
> > would
> > > >    incur an increased maintenance burden compared to doing nothing or
> > using a
> > > >    single build system.
> >
> > This is the option I like the least, as it increases the maintenance
> > burden and is pretty much guaranteed to lead to breakages.
> >
> > > >    - Android.mk-- Use a legacy (link:
> > > >    https://source.android.com/docs/setup/build
> > > >    <
> > https://mailtrack.io/trace/link/a590ddf9d4e0fc2459ff68e2943bcb5feb034056?notrack=1&url=https%3A%2F%2Fsource.android.com%2Fdocs%2Fsetup%2Fbuild&userId=4192442&signature=124f558360d74d95
> > >)
> > > >    Android.mk file and figure out how to get the proposed Android.mk
> > and
> > > >    meson_cross.mk
> > > >    <
> > https://mailtrack.io/trace/link/b47795745b452c00239c6966c5fabd90e9ff7a07?url=http%3A%2F%2Fmeson_cross.mk%2F&userId=4192442&signature=1528cb5f2f1cd6c7
> > >
> > > > files
> > > >    working and tested.
> > > >    Tradeoffs: New code would depend on a legacy tool and Makefiles can
> > be
> > > >    error-prone and hard to read (not great), but the maintenance burden
> > > >    *might* be lower than creating a parallel Android.bp build file.
> > >
> > > Of the above options, the only one that I don't think we could easily
> > > merge would be a parallel build system. That would be difficult to
> > > maintain.
> >
> > Another option which isn't listed above is to compile libcamera
> > separately using meson, with the Android SDK providing the
> > cross-compiler. This would be a manual process though, not integrated in
> > the AOSP build system. This is the option that mesa has selected as far
> > as I know, they faced the same issue as us as they use meson. They still
> > support the historical make-based Android build system
> > (https://gitlab.freedesktop.org/mesa/mesa/-/tree/main/android), if that
> > doesn't work for any reason, then users have to build mesa manually.
> >
> > We also asked, in the same Android build system session at LPC that I
> > mentioned above, if Android could provide a hook in their build system
> > to ease integration of externally-compiled binaries in Android images.
> > The answer was negative.
> >
> > > A wrapper around our build system with either an Android.mk or a Soong
> > > genrule (or both!?) would be more reasonable in my opinion ... as long
> > > as :
> > >
> > > > *Test Plan*
> > >
> > > We have a way to at least compile test it ;-)
> >
> 
> Working on that right now! Looks like this actually has a few compile
> errors. Will ask for help on this list when/if I get stuck, if that's
> alright.

Certainly, feel free to post the compile error messages as a distinct
thread - or add them to a bug on bugs.libcamera.org

If it is a compile failure we'd want to make sure that's fixed.

> > > > I'm not exactly sure how the libcamera community approaches testing.
> > That
> > > > said, I would write a sanity check based on
> > > > https://source.android.com/docs/core/tests/development/blueprints
> > > > <
> > https://mailtrack.io/trace/link/566af31c86d4ae27635d28c2f28a21e50ddd700b?notrack=1&url=https%3A%2F%2Fsource.android.com%2Fdocs%2Fcore%2Ftests%2Fdevelopment%2Fblueprints&userId=4192442&signature=76612081ac230acc
> > >.
> > > > Ideally, we would agree to run this before submitting new code. This is
> > > > probably the weakest part of my proposal, and I'd love any resources on
> > > > libcamera's testing philosophy and thoughts on how to make this
> > stronger.
> > >
> > > We already have infrastructure that builds libcamera across various GCC
> > > and clang versions. It also runs CTS (Chrome Test Suite) on a Chromebook
> > > (IPU3) which tests the Android HAL layer.
> > >
> > > I would extend this to support compiling with either the Android.mk or
> > > genrule wrappers (again, or both) ... as long as I'm guided on what is
> > > required. I have a build machine with plenty of space and resources for
> > > such a task.
> > >
> > > > *Next Steps*
> > > > If this looks good to the community (parallel build systems option),
> > I'll
> > > > figure out how to make Android.bp work. Most likely though, we have
> > more to
> > > > discuss. I'm ready to hear your thoughts!
> > >
> > > Parallel build is the one that worries me. libcamera is already complex
> > > to build, with many separate and sometimes optional components.
> > >
> > > Will it be a single file that covers our entire build? as well as the
> > > configuration options for different platforms, and specially built IPA
> > > proxies, and module signing? That sounds ... like a lot of work, and a
> > > lot of maintenance.
> > >
> > > But now we're on the libcamera list - lets await more feedback.
> >
> > On thing that could already be done without waiting for the outcome of
> > this discussion is to post, review and merge the patches that fix build
> > issues with Android (I'm thinking about removing usage of
> > std::filesystem for instance).
> >
> 
> I would like to have a working inline build before I post these. Currently,
> "activeState.agc.exposure = 10ms / configuration.sensor.lineDuration;" in
> https://github.com/rothn/libcamera/blob/aosp-inline-build/src/ipa/ipu3/algorithms/agc.cpp
> causes
> a compiler error with SDK30 because libcamera::utils::Duration and `long
> long` have no common type. Not sure if it's a compiler bug or what, but I'm

Interesting, this seems related (but not identical) to another
patch that was posted recently:

 https://patchwork.libcamera.org/patch/17570/


"""
From: Olivier Gayot <olivier.gayot at canonical.com>

On ppc64el, GCC fails to determine the result of some long double
expressions at compile time. This makes libcamera fail to build with GCC
on that architecture.

e.g.:

constexpr auto x = 1.0l/30.0;

in ‘constexpr’ expansion of ‘std::chrono::operator/<long double, std::ratio<1>,
  double>(std::literals::chrono_literals::operator""s(1.0e+0l), 6.0e+1)’
/usr/include/c++/11/chrono:710:39: error: ‘(1.0e+0l / 6.0e+1)’ is not a constant expression
  710 |         return __cd(__cd(__d).count() / __s);

Tweaking the expressions just a bit makes GCC happy and allows libcamera
to build properly on ppc64el.

Signed-off-by: Olivier Gayot <olivier.gayot at canonical.com>
---
 src/ipa/raspberrypi/raspberrypi.cpp | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp
index 14b06a4f..3e4383f9 100644
--- a/src/ipa/raspberrypi/raspberrypi.cpp
+++ b/src/ipa/raspberrypi/raspberrypi.cpp
@@ -60,7 +60,7 @@  using utils::Duration;
 /* Configure the sensor with these values initially. */
 constexpr double defaultAnalogueGain = 1.0;
 constexpr Duration defaultExposureTime = 20.0ms;
-constexpr Duration defaultMinFrameDuration = 1.0s / 30.0;
+constexpr Duration defaultMinFrameDuration = 1s / 30.0;
 constexpr Duration defaultMaxFrameDuration = 250.0s;
 
 /*
@@ -69,7 +69,7 @@  constexpr Duration defaultMaxFrameDuration = 250.0s;
  * we rate-limit the controller Prepare() and Process() calls to lower than or
  * equal to this rate.
  */
-constexpr Duration controllerMinFrameDuration = 1.0s / 30.0;
+constexpr Duration controllerMinFrameDuration = 1s / 30.0;
 
 /* List of controls handled by the Raspberry Pi IPA */
 static const ControlInfoMap::Map ipaControls{
"""



> looking into it. I'm setting up my build environment based on these
> instructions:
> https://docs.google.com/document/d/1Sly_VH3w6wFIdE72WXijQegoHZh8IxEnJ9m0hH7GodU/edit
> if
> it interests anyone.

I'd like to know the quickest/easiest way to obtain the SDK and
replicate the build for the environment.

Is the NDK/SDK available without a full build of AOSP  (If not that's
ok, but I'd be keen to know if there's a shortcut just to be able to
track the compiling of 'just' libcamera).

Building for waydroid looks interesting though - and could mean a way to
test libcamera on more platforms easily.
--
Kieran