[libcamera-devel] [IPU3-IPA PATCH v3 4/6] ipu3: Apply auto focus and send lens controls to pipeline handler

[Kieran Bingham]

Quoting Kieran Bingham (2021-11-17 17:58:00)
> Quoting Han-Lin Chen (2021-11-11 10:49:06)
> > Apply auto focus and send lens controls to pipeline handler.
> > 
> > Signed-off-by: Han-Lin Chen <hanlinchen at chromium.org>
> > ---
> >  aiq/aiq.cpp                  |  3 +--
> >  aiq/aiq.h                    |  4 +---
> >  aiq/aiq_input_parameters.cpp |  2 +-
> >  ipu3.cpp                     | 26 ++++++++++++++++++++++++++
> >  4 files changed, 29 insertions(+), 6 deletions(-)
> > 
> > diff --git a/aiq/aiq.cpp b/aiq/aiq.cpp
> > index 24c61cb..52a9c07 100644
> > --- a/aiq/aiq.cpp
> > +++ b/aiq/aiq.cpp
> > @@ -138,8 +138,7 @@ int AIQ::setStatistics(unsigned int frame,
> >   * might run asycnronously, or after receipt of statistics, with the filling
> >   * of the parameter buffer being the only part handled when called for.
> >   */
> > -int AIQ::run2a(unsigned int frame, AiqInputParameters &params,
> > -              AiqResults &results)
> > +int AIQ::run2a(unsigned int frame, AiqInputParameters &params, AiqResults &results)
> >  {
> >         (void)frame;
> >  
> > diff --git a/aiq/aiq.h b/aiq/aiq.h
> > index fcd02d2..49eee2d 100644
> > --- a/aiq/aiq.h
> > +++ b/aiq/aiq.h
> > @@ -39,9 +39,7 @@ public:
> >         int setStatistics(unsigned int frame,
> >                           int64_t timestamp, AiqResults &results,
> >                           const ipu3_uapi_stats_3a *stats);
> > -
> > -       int run2a(unsigned int frame, AiqInputParameters &params,
> > -                 AiqResults &results);
> > +       int run2a(unsigned int frame, AiqInputParameters &params, AiqResults &results);
> >  
> >  private:
> >         std::string decodeError(ia_err err);
> > diff --git a/aiq/aiq_input_parameters.cpp b/aiq/aiq_input_parameters.cpp
> > index 46553a6..5dd2f6c 100644
> > --- a/aiq/aiq_input_parameters.cpp
> > +++ b/aiq/aiq_input_parameters.cpp
> > @@ -166,7 +166,7 @@ void AiqInputParameters::setAeAwbAfDefaults()
> >                 ia_aiq_af_range_normal,
> >                 ia_aiq_af_metering_mode_auto,
> >                 ia_aiq_flash_mode_off,
> > -               NULL, NULL, false
> > +               &focusRect, &manualFocusParams, false
> >         };
> >  
> >         /* GBCE Params */
> > diff --git a/ipu3.cpp b/ipu3.cpp
> > index 9d07268..f38440d 100644
> > --- a/ipu3.cpp
> > +++ b/ipu3.cpp
> > @@ -77,6 +77,10 @@ private:
> >         uint32_t gain_;
> >         uint32_t minGain_;
> >         uint32_t maxGain_;
> > +       int32_t lensPosition_;
> > +
> > +       /* Intel AF library relies on timestamp to wait for lens movement */
> > +       uint64_t lensMovementStartTime_;
> >  
> >         /* Intel Library Instances. */
> >         aiq::AIQ aiq_;
> > @@ -257,6 +261,9 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo,
> >         maxGain_ = itGain->second.max().get<int32_t>();
> >         gain_ = maxGain_;
> >  
> > +       lensMovementStartTime_ = 0;
> > +       lensPosition_ = 0;
> > +
> >         int ret;
> >  
> >         ret = aiq_.configure();
> > @@ -383,6 +390,10 @@ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params)
> >         resultsHistory_.extendOne();
> >         aiq::AiqResults& latestResults = resultsHistory_.latest();
> >  
> > +       /* Todo: Refactor AiqInputParameters interface to set following parameters. */
> > +       aiqInputParams_.afParams.lens_position = lensPosition_;
> > +       aiqInputParams_.afParams.lens_movement_start_timestamp = lensMovementStartTime_;
> > +
> >         aiq_.run2a(frame, aiqInputParams_, latestResults);
> >         aic_.updateRuntimeParams(latestResults);
> >         aic_.run(params);
> > @@ -390,6 +401,19 @@ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params)
> >         exposure_ = latestResults.ae()->exposures[0].sensor_exposure->coarse_integration_time;
> >         gain_ = latestResults.ae()->exposures[0].sensor_exposure->analog_gain_code_global;
> >  
> > +       /*
> > +        * Af algorithm compares the timestamp of start of lens movement and the
> > +        * that of the statistics generated to estimate whether next lens
> > +        * positon should be produced.
> 
> /positon/position/
> 
> > +        * Todo: Use the lens movement start time reported by the pipeline handler.
> 
> Is this the time the control is set? or something that's expected to be
> identified from the VCM?
> 
> > +        */
> > +       if (lensPosition_ != latestResults.af()->next_lens_position) {
> > +               utils::time_point time = utils::clock::now();
> > +               uint64_t msecs = std::chrono::duration_cast<std::chrono::microseconds>(time.time_since_epoch()).count();
> > +               lensMovementStartTime_ = msecs;

Small note here. microseconds is usually referred to as us, (usecs?) ...
msecs to me would imply milli-seconds...?

The defintiion and use are closely grouped here, so it's clear
eitherway, but thought it was worth mentioning here.



> > +       }
> > +       lensPosition_ = latestResults.af()->next_lens_position;
> > +
> >         setControls(frame);
> >  
> >         IPU3Action op;
> > @@ -473,6 +497,8 @@ void IPAIPU3::setControls(unsigned int frame)
> >  
> >         op.sensorControls = sensorCtrls;
> >  
> > +       op.lensControls.set(V4L2_CID_FOCUS_ABSOLUTE, lensPosition_);
> > +
> 
> This one is going to need the corresponding update in libcamera of
> course.
> 
> Perhaps merging the interface change is possible before/while we sort
> out the other parts, as that's distinct and could unblock this patch.
> 
> But when this can compile:

I'll merge this series with the exception of these two commits:

 ipu3: Apply auto focus and send lens controls to pipeline handler
 ipu3: Run AIQ for the first frame to avoid blanking

As they require the lens updates on the libcamera side.  "Run AIQ..."
doesn't, but it seems to need to be on top of the other patch, and I
can't cherry-pick it cleanly, (which I was able to do with ipu3: Use
gbce and tone mapping from the tuning files) so I'll leave it out for
now.

Your patch queue should be easier to handle with those down anyway.


> Reviewed-by: Kieran Bingham <kieran.bingham at ideasonboard.com>
> 
> >         queueFrameAction.emit(frame, op);
> >  }
> >  
> > -- 
> > 2.34.0.rc1.387.gb447b232ab-goog
> >