[libcamera-devel] [PATCH v2] android: Add infrastructure for determining capabilities and hardware level
Jacopo Mondi
jacopo at jmondi.org
Fri Jun 18 10:45:15 CEST 2021
Hi Paul,
thanks for addressing this.
On Thu, Jun 17, 2021 at 04:38:56PM +0900, Paul Elder wrote:
> Add the infrastructure for checking and reporting capabilities. Use
> these capabilities to determine the hardware level as well.
>
> Since the raw capability is set to true when support is determined to be
> available, leave that as default false and set to true, since copying
> the pattern of the other capabilities would cause redundant code.
>
> Bug: https://bugs.libcamera.org/show_bug.cgi?id=55
> Signed-off-by: Paul Elder <paul.elder at ideasonboard.com>
>
> ---
> Changes in v2:
> - add a flag for FULL, since there are a few requirements that are not
> obtained from capabilities alone
> - add burst capture capability, since that is required for FULL as well
>
> This is my vision of how we would support the various capabilities.
> Although we don't have anything for FULL yet; I imagine that we would
> start the flags for manual sensor and manual post processing with true,
> and then if a required control is unavailable, then we would set the
> flag to false.
>
> I considered declaring an enum in CameraDevice to mirror the android
> ones, just for shorthand, but it seemed like a lot of code for not much
> gain. Unless the shorthand would be valuable because these constant
> names are so long?
>
> I think the available keys lists will have to be moved to the head of
> the function, and then as available controls are discovered add them to
> that list.
> ---
More than a comment on the patch itself, let me try to formalize the
problem a bit
We have 4 different 'items':
- supported streams
- static metadata
- capabilities
- hw level
- supported streams: self descriptive, we already collect them in
initializeStreamConfigurations()
- static metadata: List of static properties as collected in
getStaticMetadata with an associated set of values
This is what we collect in getStaticMetadata by inspecting the
libcamera::Camera capabilities plus some other information from the
configuration file.
A static metadata might be present/not present, and if it is present
it might have associated values
- capabilities: A list of static metadata which has to be present in the static
metadata pack with associated values and a list of required streams
In example, from the android.request.availableCapabilities
documentation the RAW capability requires:
- An output stream in RAW_FORMAT format with an output size equal to
the sensor pixel array
- DNG related metadata (the list to be formalized)
For MANUAL_SENSOR capability:
- android.control.aeMode contains OFF
- android.control.awbMode contains OFF
- android.control.afMode contains OFF
- ANDROID_REQUEST_AVAILABLE_RESULT_KEYS contains android.sensor.frameDuration
- ANDROID_REQUEST_AVAILABLE_CHARACTERISTICS_KEYS contains
android.sensor.info.maxFrameDuration
- ...
- hw level: a list of requested 'capabilities', a list of static
metadata that has to be present in the static metadata pack and a
list of request streams
In example, FULL requires
- The stream list required for FULL, LIMITED and LEGACY levels
(see the table here
https://developer.android.com/reference/android/hardware/camera2/CameraDevice.html#regular-capture)
- The following capabilities:
- BURST_CAPTURE, MANUAL_SENSOR, MANUAL_POST_PROCESSING
- A list of static metadata and associated requirements for values
- android.sync.maxLatency == PER_FRAME_CONTROL
- max(android.sensor.info.exposureTimeRange) >= 100ms
- max(android.sensor.info.maxFrameDuration) >= 100ms
- android.scaler.croppingType contains CENTERED
It's not easy to formalize this in a table and I'm afraid all
these checks should be open coded and built iteratively
* Build the list of supported streams like we're doing
* Build the list of static metadata like we're doing
* Build the list of capabilities inspecting the supported streams and
static metadata
* Infer the HW level by inspecting capabilities, static metadata and
the list of supported streams
I think we're good for the first two steps, the last two have to be
done and as I've said I suspect it's not easy to formalize all the
requirements in a table (not impossible though, we can associate to
each capability a list of streams and metadata, to each HW level a
list of capabilities, streams and metadata... Although how to express
constrains like "maxFrameDuration should be 1s and MUST be > 100ms"
might end up requiring more complexity than open coding)
Now, that's potentially quite some code, and I would really take the
occasion to break out from camera_device.cpp which is already ~2400
loc the parts that deal with:
- stream list contruction (initializeStreamConfigurations() function)
- static metadata pack contructions (getStaticMetadata() function)
- capabilities building
- HW level inferring
If we move all of this to a CameraCapabilities (?) class we can logically
separate it from the CameraDevice. I don't see huge blockers if not
that the CameraCapabilities class should receive quite some parameters
from CameraDevice, but we can work it out.
Should we start moving the existing code to a new class and the build
the last two steps (capabilities and hw level) there ?
What do you think ?
Thanks
j
> src/android/camera_device.cpp | 45 +++++++++++++++++++++++++++--------
> 1 file changed, 35 insertions(+), 10 deletions(-)
>
> diff --git a/src/android/camera_device.cpp b/src/android/camera_device.cpp
> index 8c71fd06..0dcdda23 100644
> --- a/src/android/camera_device.cpp
> +++ b/src/android/camera_device.cpp
> @@ -13,6 +13,7 @@
> #include <array>
> #include <cmath>
> #include <fstream>
> +#include <map>
> #include <sys/mman.h>
> #include <tuple>
> #include <unistd.h>
> @@ -840,6 +841,19 @@ const camera_metadata_t *CameraDevice::getStaticMetadata()
> const ControlInfoMap &controlsInfo = camera_->controls();
> const ControlList &properties = camera_->properties();
>
> + std::map<camera_metadata_enum_android_request_available_capabilities, bool>
> + capabilities = {
> + { ANDROID_REQUEST_AVAILABLE_CAPABILITIES_BACKWARD_COMPATIBLE, true },
> + /* \todo Change these three to true when we have checks for them */
> + { ANDROID_REQUEST_AVAILABLE_CAPABILITIES_BURST_CAPTURE, false },
> + { ANDROID_REQUEST_AVAILABLE_CAPABILITIES_MANUAL_SENSOR, false },
> + { ANDROID_REQUEST_AVAILABLE_CAPABILITIES_MANUAL_POST_PROCESSING, false },
> + { ANDROID_REQUEST_AVAILABLE_CAPABILITIES_RAW, false },
> + };
> +
> + /* \todo Change this to true when we have checks for it */
> + bool fullSupport = false;
> +
> /* Color correction static metadata. */
> {
> std::vector<uint8_t> data;
> @@ -1298,11 +1312,6 @@ const camera_metadata_t *CameraDevice::getStaticMetadata()
> uint8_t croppingType = ANDROID_SCALER_CROPPING_TYPE_CENTER_ONLY;
> staticMetadata_->addEntry(ANDROID_SCALER_CROPPING_TYPE, croppingType);
>
> - /* Info static metadata. */
> - uint8_t supportedHWLevel = ANDROID_INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED;
> - staticMetadata_->addEntry(ANDROID_INFO_SUPPORTED_HARDWARE_LEVEL,
> - supportedHWLevel);
> -
> /* Request static metadata. */
> int32_t partialResultCount = 1;
> staticMetadata_->addEntry(ANDROID_REQUEST_PARTIAL_RESULT_COUNT,
> @@ -1323,10 +1332,6 @@ const camera_metadata_t *CameraDevice::getStaticMetadata()
> staticMetadata_->addEntry(ANDROID_REQUEST_MAX_NUM_INPUT_STREAMS,
> maxNumInputStreams);
>
> - std::vector<uint8_t> availableCapabilities = {
> - ANDROID_REQUEST_AVAILABLE_CAPABILITIES_BACKWARD_COMPATIBLE,
> - };
> -
> /* Report if camera supports RAW. */
> bool rawStreamAvailable = false;
> std::unique_ptr<CameraConfiguration> cameraConfig =
> @@ -1338,7 +1343,7 @@ const camera_metadata_t *CameraDevice::getStaticMetadata()
> if (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW &&
> info.bitsPerPixel == 16) {
> rawStreamAvailable = true;
> - availableCapabilities.push_back(ANDROID_REQUEST_AVAILABLE_CAPABILITIES_RAW);
> + capabilities[ANDROID_REQUEST_AVAILABLE_CAPABILITIES_RAW] = true;
> }
> }
>
> @@ -1347,9 +1352,29 @@ const camera_metadata_t *CameraDevice::getStaticMetadata()
> staticMetadata_->addEntry(ANDROID_REQUEST_MAX_NUM_OUTPUT_STREAMS,
> numOutStreams);
>
> + /* Check capabilities */
> + std::vector<uint8_t> availableCapabilities;
> + for (auto cap : capabilities) {
> + if (cap.second)
> + availableCapabilities.push_back(CapabilityTable.at(cap.first));
> + }
> staticMetadata_->addEntry(ANDROID_REQUEST_AVAILABLE_CAPABILITIES,
> availableCapabilities);
>
> + uint8_t supportedHWLevel = ANDROID_INFO_SUPPORTED_HARDWARE_LEVEL_LIMITED;
> + if (capabilities[ANDROID_REQUEST_AVAILABLE_CAPABILITIES_MANUAL_SENSOR] &&
> + capabilities[ANDROID_REQUEST_AVAILABLE_CAPABILITIES_MANUAL_POST_PROCESSING] &&
> + capabilities[ANDROID_REQUEST_AVAILABLE_CAPABILITIES_BURST_CAPTURE] &&
> + fullSupport)
> + supportedHWLevel = ANDROID_INFO_SUPPORTED_HARDWARE_LEVEL_FULL;
> + staticMetadata_->addEntry(ANDROID_INFO_SUPPORTED_HARDWARE_LEVEL,
> + supportedHWLevel);
> +
> + LOG(HAL, Info)
> + << "Hardware level: "
> + << supportedHWLevel == ANDROID_INFO_SUPPORTED_HARDWARE_LEVEL_FULL
> + ? "FULL" : "LIMITED";
> +
> std::vector<int32_t> availableCharacteristicsKeys = {
> ANDROID_COLOR_CORRECTION_AVAILABLE_ABERRATION_MODES,
> ANDROID_CONTROL_AE_AVAILABLE_ANTIBANDING_MODES,
> --
> 2.27.0
>
More information about the libcamera-devel
mailing list