[libcamera-devel] libcamera AOSP Integration

Nicholas Roth nicholas at rothemail.net
Thu Oct 20 07:51:47 CEST 2022 

I was able to get a little farther, to where the .so seems to load but
still no cameras:
```
10-20 00:44:41.142    71    71 I CamPrvdr at 2.4-legacy: Loaded "libcamera
camera HALv3 module" camera module
10-20 00:44:41.142    71    71 I CamPrvdr at 2.4-legacy: setUpVendorTags: No
vendor tags defined for this device.
```

New logcat: https://github.com/waydroid/waydroid/files/9826696/logcat.log

On Wed, Oct 19, 2022 at 11:31 PM Nicholas Roth <nicholas at rothemail.net>
wrote:

> Thanks all for the help so far!
>
> I've got everything building and producing files in the proper places, but
> I can't get Android to pick up the HAL! I wouldn't feel good about
> committing this without validation, so I'd love your help testing this so
> we can get it in.
>
> From https://github.com/waydroid/waydroid/issues/519:
>
> The files are there:
> * /vendor/lib64/hw/camera.libcamera.so
> * /vendor/lib64/libcamera.so
> * /vendor/lib64/libcamera-base.so
>
> And yet, according to Android, based on `lshal` and `dumpsys
> media.camera`, this HAL does not exist and there are no cameras in the
> system!
>
> Any ideas or guesses as to what I might be missing? I'm really stumped and
> would appreciate your help.
>
> libcamera_debug.log:
> https://github.com/waydroid/waydroid/files/9825989/libcamera_debug.log
> logcat.log: https://github.com/waydroid/waydroid/files/9825990/logcat.log
>
>
>
> On Sat, Oct 15, 2022 at 6:38 PM Laurent Pinchart <
> laurent.pinchart at ideasonboard.com> wrote:
>
>> On Sat, Oct 15, 2022 at 07:40:38PM +0300, Laurent Pinchart via
>> libcamera-devel wrote:
>> > On Fri, Oct 14, 2022 at 10:46:56AM +0100, Kieran Bingham wrote:
>> > > Quoting Nicholas Roth (2022-10-14 01:35:37)
>> > > > > Out of curiosity, is that the PinePhone Pro, or something else?
>> > > >
>> > > > PinePhone Pro
>> > > >
>> > > > > Also, which Android version are you targetting ?
>> > > >
>> > > > SDK 30
>> > > >
>> > > > > Could you please disable mail link tracking when posting to the
>> list ?
>> > > >
>> > > > Yes! Oops. I'm sending this from a corp machine where the extension
>> is not
>> > > > installed, so hopefully that fixes it.
>> > > >
>> > > > On Thu, Oct 13, 2022 at 4:09 AM Laurent Pinchart wrote:
>> > > > > On Wed, Oct 12, 2022 at 11:11:26AM +0100, Kieran Bingham via
>> > > > > libcamera-devel wrote:
>> > > > > > Quoting Nicholas Roth via libcamera-devel (2022-10-12 04:16:03)
>> > > > > > > Hello all,
>> > > > > > >
>> > > > > > > TL;DR: I would propose adding a top-level Android.bp
>> Bazel-based build and
>> > > > > > > tests to support building libcamera inline with AOSP to
>> expose the
>> > > > > > > libcamera HAL.
>> > > > > >
>> > > > > > Thanks for taking this on.
>> > > > >
>> > > > > Likewise, thank you for your offer to help supporting the Android
>> build
>> > > > > system.
>> > > > >
>> > > > > As a first comment, Android.bp files are meant for the Soong build
>> > > > > system, not Bazel. They are similar (and the Android.bp syntax is
>> > > > > intentionally similar to Bazel), but not identical. See
>> > > > >
>> https://android.googlesource.com/platform/build/soong/+/master/README.md
>> > > > > for information about Soong.
>> > > > >
>> > > > > Then, to make this even more fun, Android has decided in 2020 to
>> migrate
>> > > > > to Bazel (
>> https://source.android.com/docs/setup/build/bazel/introduction).
>> > > > > This is work in progress, I don't know when it will land, and it
>> will
>> > > > > then deprecate both make and Soong.
>> > > > >
>> > > > > > > *Short intro because I'm new* (feel free to skip):
>> > > > > > > I'm Nicholas, a long-time amateur programmer with ~10 years
>> of industry
>> > > > > > > experience now, although it doesn't seem like it's been that
>> long. I've
>> > > > > > > published work in distributed scale-out systems and for the
>> last ~6 years,
>> > > > > > > I've worked in AI. More relevant to you all, I have free time
>> and a phone
>> > > > > > > with an RK3399 SoC that I'd like to get supported in Waydroid.
>> > > > >
>> > > > > Out of curiosity, is that the PinePhone Pro, or something else ?
>> > > > >
>> > > > > Also, which Android version are you targetting ?
>> > > > >
>> > > > > > Welcome to the team! :-)
>> > > > > >
>> > > > > > > *Background*:
>> > > > > > > There's been back-and-forth discussion on Kieran's personal
>> GitHub account
>> > > > > > > (link: https://github.com/kbingham/libcamera/pull/26)
>> > > > >
>> > > > > Could you please disable mail link tracking when posting to the
>> list ?
>> > > > >
>> > > > > > > about how to potentially build libcamera inline with AOSP. If
>> I understand
>> > > > > > > correctly, the original contributor proposed some minor
>> changes and also a
>> > > > > >
>> > > > > > The minor changes/fixes could certainly be considered already
>> when
>> > > > > > posted to the list.
>> > > > >
>> > > > > Yes, those could be posted, reviewed and merged already, before
>> > > > > completing the work on the build system.
>> > > >
>> > > > Will do. My fork currently does not build on SDK 30 due to
>> additional
>> > > > compiler errors. My top priority is to get a working build so I can
>> test
>> > > > changes.
>> > > >
>> > > > > > > major change that would add ~349 lines of Makefiles,
>> including a deprecated
>> > > > > > > Android.mk. The original contributor ran out of bandwidth
>> recently and
>> > > > > >
>> > > > > > The build system is probably the hard part, and indeed it
>> sounds like
>> > > > > > there are two options.
>> > > > > >
>> > > > > > 1) an Android.mk 'wrapper' around the meson infrastructure
>> > > > > > 2) A parallel build system.
>> > > > > >
>> > > > > >
>> > > > > > 1. may become deprecated, but we don't know when. And the one
>> thing I've
>> > > > > > learnt about open source is - if you don't know when somethings
>> going to
>> > > > > > happen, it seems best sometimes to assume it won't until it
>> does.
>> > > > >
>> > > > > It is officially deprecated already, what we don't know is when
>> support
>> > > > > for it will be dropped. It may be possible to find out, I recall
>> seeing
>> > > > > a public migration schedule plan a while ago, but can't find it
>> at the
>> > > > > moment. Given that Android is migrating to Bazel, they may not
>> move
>> > > > > forward with dropping support of make before they switch away
>> from Soong
>> > > > > too. Or maybe then will. It's hard to predict, they're not known
>> for
>> > > > > taking anybody but themselves into consideration in their plans.
>> > > > >
>> > > > > > Because it can be *far away in the future*  (Just like our first
>> > > > > > libcamera stable API release :D)
>> > > > >
>> > > > > As long as an Android.mk wrapper around meson would be low
>> maintenance,
>> > > > > I don't mind having support for the deprecated make-based build
>> system
>> > > > > for the time being. Of course, if there was a way to nicely
>> support
>> > > > > Soong-based builds without a high maintenance effort, that would
>> be
>> > > > > better, but a "one step at a time" approach is fine with me.
>> > > >
>> > > > From what I'm seeing so far, I'm assuming we'll start with a
>> Makefile-based
>> > > > approach. That's what I'm using now.
>> >
>> > While I wish we could already target something that would be more
>> > future-proof, this is likely the most sensible approach for now. There's
>> > little point in spending time on supporting the Soong build system as
>> > it's already deprecated.
>>
>> [1] seems to be the official source of information regarding the
>> deprecation of the make-based build system. [2] provides the same
>> information in a bit nicer to read form. BUILD_SHARED_LIBRARY is still
>> supported, and I haven't found any indication as to when it would be
>> dropped.
>>
>> [1]
>> https://android.googlesource.com/platform/build/+/master/core/deprecation.mk
>> [2]
>> https://android.googlesource.com/platform/build/+/master/Deprecation.md
>>
>> > > > > > 2. Is painful to maintain... but sounds like the route you are
>> > > > > > interested in...
>> > > > >
>> > > > > I don't think we want to maintain two build systems in parallel. A
>> > > > > proper integrating with Soong or Bazel would require bridging the
>> gap
>> > > > > between those and meson, as we have no plan to move away from
>> meson for
>> > > > > the time being.
>> > > > >
>> > > > > For Soong, the little I understand is that the Android.bp files
>> are
>> > > > > fully declarative, and any logic needs to be written in Soong
>> plugins,
>> > > > > in the Go language. As libcamera generates source files at build
>> time,
>> > > > > this generation logic would need to be implemented there. A meson
>> > > > > developer once (during a session about the Android build system
>> at the
>> > > > > Linux Plumbers Conference a few years ago) publicly offered to
>> write a
>> > > > > meson frontend in Go, for integration with Soong (a meson
>> frontend means
>> > > > > a reimplementation of meson in Go, which would parse the
>> meson.build
>> > > > > files and output the data format expected by Soong to let the
>> Soong
>> > > > > backend generate ninja files).  The offer was rejected by the
>> Android
>> > > > > build system maintainer along the lines of "you can do whatever
>> you
>> > > > > want, we won't care, so it will likely break very quickly as we
>> move
>> > > > > forward". That wasn't very motivating.
>> > > > >
>> > > > > > > decided not to pursue their changes further, and I
>> volunteered to carry
>> > > > > > > this forward.
>> > > > > > >
>> > > > > > > *My Prior Work*
>> > > > > > > I have tried building libcamera with these proposed changes
>> inline with
>> > > > > > > AOSP using sdk 30 under "/external/libcamera" (full procedure
>> link:
>> > > > > > >
>> https://docs.google.com/document/d/1Sly_VH3w6wFIdE72WXijQegoHZh8IxEnJ9m0hH7GodU/edit
>> )
>> > > > > > > and it does not work as intended as far as I can tell.
>> Specifically,
>> > > > > > > /vendor/lib/hw/camera.libcamera.so does
>> > > > > > > not get bundled into the Android images. This may be user
>> error on my part,
>> > > > > > > however.
>> > > > > > >
>> > > > > > > *Options for AOSP Inline Build*
>> > > > > > > The way I can see, we have at least these options for
>> building libcamera
>> > > > > > > inline with AOSP. Personally, parallel build systems (see
>> below) seems like
>> > > > > > > the best option to me. Please feel free to jump in if I'm
>> wrong about any
>> > > > > > > of this or if you have options to add:
>> > > > > > >
>> > > > > > >    - Transition libcamera to Soong and build intuitively
>> > > > > > >    Tradeoffs: Simplest and most consistent option, but
>> potentially very
>> > > > > > >    disruptive
>> > > > >
>> > > > > As Soong is already deprecated by Android given their move to
>> Bazel,
>> > > > > that's a dead end. Even if it wasn't the case, I don't think it
>> would be
>> > > > > an option, as Soong is Android-specific.
>> > > > >
>> > > > > > >    - Invoke Meson from Soong with a genrule, akin to what
>> meson_cross.mk
>> > > > > > >     does
>> > > > > > >    Tradeoffs: complex-- I would be happy to do this but I'll
>> need help
>> > > > > > >    understanding how meson_cross.mk works;
>> > > > > > >    probably harder than writing meson_cross.mk was;
>> > > > > > >    possibly infeasible since Soong is quite opinionated
>> > > > >
>> > > > > As far as I can tell it's not really possible indeed. As I wrote
>> above,
>> > > > > you'd need a Soong plugin in Go that generates the internal data
>> format
>> > > > > that Soong requires. It may be possible to have that plugin call
>> the
>> > > > > meson parser written in Python, but writing a meson frontend in
>> Go may
>> > > > > be easier. meson doesn't guarantee the stability of the API
>> between its
>> > > > > Python frontend and Python backends as far as I know.
>> > > > >
>> > > > > > >    - *Parallel build systems*-- Create an Android.bp file
>> that would live
>> > > > > > >    at the top-level of the libcamer repo in addition to the
>> existing Meson
>> > > > > > >    files. We would use Soong for Android, and Meson for other
>> use-cases.
>> > > > > > >    Tradeoffs: Less complex and less error-prone than invoking
>> Meson from
>> > > > > > >    Soong, and less disruptive than moving to Soong
>> exclusively, but would
>> > > > > > >    incur an increased maintenance burden compared to doing
>> nothing or using a
>> > > > > > >    single build system.
>> > > > >
>> > > > > This is the option I like the least, as it increases the
>> maintenance
>> > > > > burden and is pretty much guaranteed to lead to breakages.
>> > > > >
>> > > > > > >    - Android.mk-- Use a legacy (link:
>> https://source.android.com/docs/setup/build)
>> > > > > > >    Android.mk file and figure out how to get the proposed
>> Android.mk and
>> > > > > > >    meson_cross.mk files working and tested.
>> > > > > > >    Tradeoffs: New code would depend on a legacy tool and
>> Makefiles can be
>> > > > > > >    error-prone and hard to read (not great), but the
>> maintenance burden
>> > > > > > >    *might* be lower than creating a parallel Android.bp build
>> file.
>> > > > > >
>> > > > > > Of the above options, the only one that I don't think we could
>> easily
>> > > > > > merge would be a parallel build system. That would be difficult
>> to
>> > > > > > maintain.
>> > > > >
>> > > > > Another option which isn't listed above is to compile libcamera
>> > > > > separately using meson, with the Android SDK providing the
>> > > > > cross-compiler. This would be a manual process though, not
>> integrated in
>> > > > > the AOSP build system. This is the option that mesa has selected
>> as far
>> > > > > as I know, they faced the same issue as us as they use meson.
>> They still
>> > > > > support the historical make-based Android build system
>> > > > > (https://gitlab.freedesktop.org/mesa/mesa/-/tree/main/android),
>> if that
>> > > > > doesn't work for any reason, then users have to build mesa
>> manually.
>> > > > >
>> > > > > We also asked, in the same Android build system session at LPC
>> that I
>> > > > > mentioned above, if Android could provide a hook in their build
>> system
>> > > > > to ease integration of externally-compiled binaries in Android
>> images.
>> > > > > The answer was negative.
>> > > > >
>> > > > > > A wrapper around our build system with either an Android.mk or
>> a Soong
>> > > > > > genrule (or both!?) would be more reasonable in my opinion ...
>> as long
>> > > > > > as :
>> > > > > >
>> > > > > > > *Test Plan*
>> > > > > >
>> > > > > > We have a way to at least compile test it ;-)
>> > > >
>> > > > Working on that right now! Looks like this actually has a few
>> compile
>> > > > errors. Will ask for help on this list when/if I get stuck, if
>> that's
>> > > > alright.
>> > >
>> > > Certainly, feel free to post the compile error messages as a distinct
>> > > thread - or add them to a bug on bugs.libcamera.org
>> > >
>> > > If it is a compile failure we'd want to make sure that's fixed.
>> >
>> > If possible, please report the exact compiler version and flags used to
>> > reproduce the problem.
>> >
>> > > > > > > I'm not exactly sure how the libcamera community approaches
>> testing. That
>> > > > > > > said, I would write a sanity check based on
>> > > > > > >
>> https://source.android.com/docs/core/tests/development/blueprints.
>> > > > > > > Ideally, we would agree to run this before submitting new
>> code. This is
>> > > > > > > probably the weakest part of my proposal, and I'd love any
>> resources on
>> > > > > > > libcamera's testing philosophy and thoughts on how to make
>> this stronger.
>> > > > > >
>> > > > > > We already have infrastructure that builds libcamera across
>> various GCC
>> > > > > > and clang versions. It also runs CTS (Chrome Test Suite) on a
>> Chromebook
>> > > > > > (IPU3) which tests the Android HAL layer.
>> > > > > >
>> > > > > > I would extend this to support compiling with either the
>> Android.mk or
>> > > > > > genrule wrappers (again, or both) ... as long as I'm guided on
>> what is
>> > > > > > required. I have a build machine with plenty of space and
>> resources for
>> > > > > > such a task.
>> > > > > >
>> > > > > > > *Next Steps*
>> > > > > > > If this looks good to the community (parallel build systems
>> option), I'll
>> > > > > > > figure out how to make Android.bp work. Most likely though,
>> we have more to
>> > > > > > > discuss. I'm ready to hear your thoughts!
>> > > > > >
>> > > > > > Parallel build is the one that worries me. libcamera is already
>> complex
>> > > > > > to build, with many separate and sometimes optional components.
>> > > > > >
>> > > > > > Will it be a single file that covers our entire build? as well
>> as the
>> > > > > > configuration options for different platforms, and specially
>> built IPA
>> > > > > > proxies, and module signing? That sounds ... like a lot of
>> work, and a
>> > > > > > lot of maintenance.
>> > > > > >
>> > > > > > But now we're on the libcamera list - lets await more feedback.
>> > > > >
>> > > > > On thing that could already be done without waiting for the
>> outcome of
>> > > > > this discussion is to post, review and merge the patches that fix
>> build
>> > > > > issues with Android (I'm thinking about removing usage of
>> > > > > std::filesystem for instance).
>> > > >
>> > > > I would like to have a working inline build before I post these.
>> Currently,
>> > > > "activeState.agc.exposure = 10ms /
>> configuration.sensor.lineDuration;" in
>> > > >
>> https://github.com/rothn/libcamera/blob/aosp-inline-build/src/ipa/ipu3/algorithms/agc.cpp
>> > > > causes
>> > > > a compiler error with SDK30 because libcamera::utils::Duration and
>> `long
>> > > > long` have no common type. Not sure if it's a compiler bug or what,
>> but I'm
>> > >
>> > > Interesting, this seems related (but not identical) to another
>> > > patch that was posted recently:
>> > >
>> > >  https://patchwork.libcamera.org/patch/17570/
>> > >
>> > >
>> > > """
>> > > From: Olivier Gayot <olivier.gayot at canonical.com>
>> > >
>> > > On ppc64el, GCC fails to determine the result of some long double
>> > > expressions at compile time. This makes libcamera fail to build with
>> GCC
>> > > on that architecture.
>> > >
>> > > e.g.:
>> > >
>> > > constexpr auto x = 1.0l/30.0;
>> > >
>> > > in ‘constexpr’ expansion of ‘std::chrono::operator/<long double,
>> std::ratio<1>,
>> > >   double>(std::literals::chrono_literals::operator""s(1.0e+0l),
>> 6.0e+1)’
>> > > /usr/include/c++/11/chrono:710:39: error: ‘(1.0e+0l / 6.0e+1)’ is not
>> a constant expression
>> > >   710 |         return __cd(__cd(__d).count() / __s);
>> > >
>> > > Tweaking the expressions just a bit makes GCC happy and allows
>> libcamera
>> > > to build properly on ppc64el.
>> > >
>> > > Signed-off-by: Olivier Gayot <olivier.gayot at canonical.com>
>> > > ---
>> > >  src/ipa/raspberrypi/raspberrypi.cpp | 4 ++--
>> > >  1 file changed, 2 insertions(+), 2 deletions(-)
>> > >
>> > > diff --git a/src/ipa/raspberrypi/raspberrypi.cpp
>> b/src/ipa/raspberrypi/raspberrypi.cpp
>> > > index 14b06a4f..3e4383f9 100644
>> > > --- a/src/ipa/raspberrypi/raspberrypi.cpp
>> > > +++ b/src/ipa/raspberrypi/raspberrypi.cpp
>> > > @@ -60,7 +60,7 @@  using utils::Duration;
>> > >  /* Configure the sensor with these values initially. */
>> > >  constexpr double defaultAnalogueGain = 1.0;
>> > >  constexpr Duration defaultExposureTime = 20.0ms;
>> > > -constexpr Duration defaultMinFrameDuration = 1.0s / 30.0;
>> > > +constexpr Duration defaultMinFrameDuration = 1s / 30.0;
>> > >  constexpr Duration defaultMaxFrameDuration = 250.0s;
>> > >
>> > >  /*
>> > > @@ -69,7 +69,7 @@  constexpr Duration defaultMaxFrameDuration =
>> 250.0s;
>> > >   * we rate-limit the controller Prepare() and Process() calls to
>> lower than or
>> > >   * equal to this rate.
>> > >   */
>> > > -constexpr Duration controllerMinFrameDuration = 1.0s / 30.0;
>> > > +constexpr Duration controllerMinFrameDuration = 1s / 30.0;
>> > >
>> > >  /* List of controls handled by the Raspberry Pi IPA */
>> > >  static const ControlInfoMap::Map ipaControls{
>> > > """
>> >
>> > I've just reviewed that patch, and I don't think it's related.
>> >
>> > > > looking into it. I'm setting up my build environment based on these
>> > > > instructions:
>> > > >
>> https://docs.google.com/document/d/1Sly_VH3w6wFIdE72WXijQegoHZh8IxEnJ9m0hH7GodU/edit
>> > > > if it interests anyone.
>> > >
>> > > I'd like to know the quickest/easiest way to obtain the SDK and
>> > > replicate the build for the environment.
>> > >
>> > > Is the NDK/SDK available without a full build of AOSP  (If not that's
>> > > ok, but I'd be keen to know if there's a shortcut just to be able to
>> > > track the compiling of 'just' libcamera).
>> > >
>> > > Building for waydroid looks interesting though - and could mean a way
>> to
>> > > test libcamera on more platforms easily.
>> >
>> > I gave it a brief try by following
>> >
>> https://proandroiddev.com/how-to-setup-android-sdk-without-android-studio-6d60d0f2812a?gi=28d85ab4cc0c
>> > to setup the SDK, and wrote a meson cross file by hand. It failed at
>> > meson setup stage, when trying to configure cmake-based subprojects. I
>> > found https://github.com/ppetraki/meson-android-helloworld that may be
>> > of interest, but haven't investigated. It would be nice to have a
>> > clearly documented procedure.
>>
>> --
>> Regards,
>>
>> Laurent Pinchart
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libcamera.org/pipermail/libcamera-devel/attachments/20221020/016e04a0/attachment.htm>

More information about the libcamera-devel mailing list