[PATCH v2 4/7] Documentation: Breakout docs.rst
Dan Scally
dan.scally at ideasonboard.com
Thu Aug 15 11:55:48 CEST 2024
On 15/08/2024 09:29, Daniel Scally wrote:
> In preparation for including more of the Documentation for libcamera
> on the website, break out the Camera Stack and Feature Requirements
> sections of docs.rst file into separate files for each section. Add
> all of the new files to documentation-contents.rst so they're
> included on the website too.
>
> Signed-off-by: Daniel Scally <dan.scally at ideasonboard.com>
> ---
> Changes since v1:
>
> - Included the new files in meson's docs_sources array
>
>
> Documentation/camera_stack.rst | 78 ++++++++
> Documentation/docs.rst | 219 -----------------------
> Documentation/documentation-contents.rst | 2 +
> Documentation/feature_requirements.rst | 145 +++++++++++++++
> Documentation/index.rst | 2 +
> Documentation/meson.build | 2 +
> 6 files changed, 229 insertions(+), 219 deletions(-)
> create mode 100644 Documentation/camera_stack.rst
> create mode 100644 Documentation/feature_requirements.rst
>
> diff --git a/Documentation/camera_stack.rst b/Documentation/camera_stack.rst
> new file mode 100644
> index 00000000..381385cb
> --- /dev/null
> +++ b/Documentation/camera_stack.rst
> @@ -0,0 +1,78 @@
> +.. SPDX-License-Identifier: CC-BY-SA-4.0
> +
> +.. include:: documentation-contents.rst
> +
> +Camera Stack
> +============
> +
> +::
> +
> + a c / +-------------+ +-------------+ +-------------+ +-------------+
> + p a | | Native | | Framework | | Native | | Android |
> + p t | | V4L2 | | Application | | libcamera | | Camera |
> + l i | | Application | | (gstreamer) | | Application | | Framework |
> + i o \ +-------------+ +-------------+ +-------------+ +-------------+
> + n ^ ^ ^ ^
> + | | | |
> + l a | | | |
> + i d v v | v
> + b a / +-------------+ +-------------+ | +-------------+
> + c p | | V4L2 | | Camera | | | Android |
> + a t | | Compat. | | Framework | | | Camera |
> + m a | | | | (gstreamer) | | | HAL |
> + e t \ +-------------+ +-------------+ | +-------------+
> + r i ^ ^ | ^
> + a o | | | |
> + n | | | |
> + / | ,................................................
> + | | ! : Language : !
> + l f | | ! : Bindings : !
> + i r | | ! : (optional) : !
> + b a | | \...............................................'
> + c m | | | | |
> + a e | | | | |
> + m w | v v v v
> + e o | +----------------------------------------------------------------+
> + r r | | |
> + a k | | libcamera |
> + | | |
> + \ +----------------------------------------------------------------+
> + ^ ^ ^
> + Userspace | | |
> + ------------------------ | ---------------- | ---------------- | ---------------
> + Kernel | | |
> + v v v
> + +-----------+ +-----------+ +-----------+
> + | Media | <--> | Video | <--> | V4L2 |
> + | Device | | Device | | Subdev |
> + +-----------+ +-----------+ +-----------+
> +
> +The camera stack comprises four software layers. From bottom to top:
> +
> +* The kernel drivers control the camera hardware and expose a
> + low-level interface to userspace through the Linux kernel V4L2
> + family of APIs (Media Controller API, V4L2 Video Device API and
> + V4L2 Subdev API).
> +
> +* The libcamera framework is the core part of the stack. It
> + handles all control of the camera devices in its core component,
> + libcamera, and exposes a native C++ API to upper layers. Optional
> + language bindings allow interfacing to libcamera from other
> + programming languages.
> +
> + Those components live in the same source code repository and
> + all together constitute the libcamera framework.
> +
> +* The libcamera adaptation is an umbrella term designating the
> + components that interface to libcamera in other frameworks.
> + Notable examples are a V4L2 compatibility layer, a gstreamer
> + libcamera element, and an Android camera HAL implementation based
> + on libcamera.
> +
> + Those components can live in the libcamera project source code
> + in separate repositories, or move to their respective project's
> + repository (for instance the gstreamer libcamera element).
> +
> +* The applications and upper level frameworks are based on the
> + libcamera framework or libcamera adaptation, and are outside of
> + the scope of the libcamera project.
> diff --git a/Documentation/docs.rst b/Documentation/docs.rst
> index 10f07a9e..d65b2b4f 100644
> --- a/Documentation/docs.rst
> +++ b/Documentation/docs.rst
> @@ -21,225 +21,6 @@ The libcamera API is extensively documented using Doxygen. The :ref:`API
> nightly build <api>` contains the most up-to-date API documentation, built from
> the latest master branch.
>
> -Feature Requirements
> -====================
> -
> -Device enumeration
> -------------------
> -
> -The library shall support enumerating all camera devices available in the
> -system, including both fixed cameras and hotpluggable cameras. It shall
> -support cameras plugged and unplugged after the initialization of the
> -library, and shall offer a mechanism to notify applications of camera plug
> -and unplug.
> -
> -The following types of cameras shall be supported:
> -
> -* Internal cameras designed for point-and-shoot still image and video
> - capture usage, either controlled directly by the CPU, or exposed through
> - an internal USB bus as a UVC device.
> -
> -* External UVC cameras designed for video conferencing usage.
> -
> -Other types of camera, including analog cameras, depth cameras, thermal
> -cameras, external digital picture or movie cameras, are out of scope for
> -this project.
> -
> -A hardware device that includes independent camera sensors, such as front
> -and back sensors in a phone, shall be considered as multiple camera devices
> -for the purpose of this library.
> -
> -Independent Camera Devices
> ---------------------------
> -
> -When multiple cameras are present in the system and are able to operate
> -independently from each other, the library shall expose them as multiple
> -camera devices and support parallel operation without any additional usage
> -restriction apart from the limitations inherent to the hardware (such as
> -memory bandwidth, CPU usage or number of CSI-2 receivers for instance).
> -
> -Independent processes shall be able to use independent cameras devices
> -without interfering with each other. A single camera device shall be
> -usable by a single process at a time.
> -
> -Multiple streams support
> -------------------------
> -
> -The library shall support multiple video streams running in parallel
> -for each camera device, within the limits imposed by the system.
> -
> -Per frame controls
> -------------------
> -
> -The library shall support controlling capture parameters for each stream
> -on a per-frame basis, on a best effort basis based on the capabilities of the
> -hardware and underlying software stack (including kernel drivers and
> -firmware). It shall apply capture parameters to the frame they target, and
> -report the value of the parameters that have effectively been used for each
> -captured frame.
> -
> -When a camera device supports multiple streams, the library shall allow both
> -control of each stream independently, and control of multiple streams
> -together. Streams that are controlled together shall be synchronized. No
> -synchronization is required for streams controlled independently.
> -
> -Capability Enumeration
> -----------------------
> -
> -The library shall expose capabilities of each camera device in a way that
> -allows applications to discover those capabilities dynamically. Applications
> -shall be allowed to cache capabilities for as long as they are using the
> -library. If capabilities can change at runtime, the library shall offer a
> -mechanism to notify applications of such changes. Applications shall not
> -cache capabilities in long term storage between runs.
> -
> -Capabilities shall be discovered dynamically at runtime from the device when
> -possible, and may come, in part or in full, from platform configuration
> -data.
> -
> -Device Profiles
> ----------------
> -
> -The library may define different camera device profiles, each with a minimum
> -set of required capabilities. Applications may use those profiles to quickly
> -determine the level of features exposed by a device without parsing the full
> -list of capabilities. Camera devices may implement additional capabilities
> -on top of the minimum required set for the profile they expose.
> -
> -3A and Image Enhancement Algorithms
> ------------------------------------
> -
> -The camera devices shall implement auto exposure, auto gain and auto white
> -balance. Camera devices that include a focus lens shall implement auto
> -focus. Additional image enhancement algorithms, such as noise reduction or
> -video stabilization, may be implemented.
> -
> -All algorithms may be implemented in hardware or firmware outside of the
> -library, or in software in the library. They shall all be controllable by
> -applications.
> -
> -The library shall be architectured to isolate the 3A and image enhancement
> -algorithms in a component with a documented API, respectively called the 3A
> -component and the 3A API. The 3A API shall be stable, and shall allow both
> -open-source and closed-source implementations of the 3A component.
> -
> -The library may include statically-linked open-source 3A components, and
> -shall support dynamically-linked open-source and closed-source 3A
> -components.
> -
> -Closed-source 3A Component Sandboxing
> --------------------------------------
> -
> -For security purposes, it may be desired to run closed-source 3A components
> -in a separate process. The 3A API would in such a case be transported over
> -IPC. The 3A API shall make it possible to use any IPC mechanism that
> -supports passing file descriptors.
> -
> -The library may implement an IPC mechanism, and shall support third-party
> -platform-specific IPC mechanisms through the implementation of a
> -platform-specific 3A API wrapper. No modification to the library shall be
> -needed to use such third-party IPC mechanisms.
> -
> -The 3A component shall not directly access any device node on the system.
> -Such accesses shall instead be performed through the 3A API. The library
> -shall validate all accesses and restrict them to what is absolutely required
> -by 3A components.
> -
> -V4L2 Compatibility Layer
> -------------------------
> -
> -The project shall support traditional V4L2 application through an additional
> -libcamera wrapper library. The wrapper library shall trap all accesses to
> -camera devices through `LD_PRELOAD`, and route them through libcamera to
> -emulate a high-level V4L2 camera device. It shall expose camera device
> -features on a best-effort basis, and aim for the level of features
> -traditionally available from a UVC camera designed for video conferencing.
> -
> -Android Camera HAL v3 Compatibility
> ------------------------------------
> -
> -The library API shall expose all the features required to implement an
> -Android Camera HAL v3 on top of libcamera. Some features of the HAL may be
> -omitted as long as they can be implemented separately in the HAL, such as
> -JPEG encoding, or YUV reprocessing.
> -
> -
> -Camera Stack
> -============
> -
> -::
> -
> - a c / +-------------+ +-------------+ +-------------+ +-------------+
> - p a | | Native | | Framework | | Native | | Android |
> - p t | | V4L2 | | Application | | libcamera | | Camera |
> - l i | | Application | | (gstreamer) | | Application | | Framework |
> - i o \ +-------------+ +-------------+ +-------------+ +-------------+
> - n ^ ^ ^ ^
> - | | | |
> - l a | | | |
> - i d v v | v
> - b a / +-------------+ +-------------+ | +-------------+
> - c p | | V4L2 | | Camera | | | Android |
> - a t | | Compat. | | Framework | | | Camera |
> - m a | | | | (gstreamer) | | | HAL |
> - e t \ +-------------+ +-------------+ | +-------------+
> - r i ^ ^ | ^
> - a o | | | |
> - n | | | |
> - / | ,................................................
> - | | ! : Language : !
> - l f | | ! : Bindings : !
> - i r | | ! : (optional) : !
> - b a | | \...............................................'
> - c m | | | | |
> - a e | | | | |
> - m w | v v v v
> - e o | +----------------------------------------------------------------+
> - r r | | |
> - a k | | libcamera |
> - | | |
> - \ +----------------------------------------------------------------+
> - ^ ^ ^
> - Userspace | | |
> - ------------------------ | ---------------- | ---------------- | ---------------
> - Kernel | | |
> - v v v
> - +-----------+ +-----------+ +-----------+
> - | Media | <--> | Video | <--> | V4L2 |
> - | Device | | Device | | Subdev |
> - +-----------+ +-----------+ +-----------+
> -
> -The camera stack comprises four software layers. From bottom to top:
> -
> -* The kernel drivers control the camera hardware and expose a
> - low-level interface to userspace through the Linux kernel V4L2
> - family of APIs (Media Controller API, V4L2 Video Device API and
> - V4L2 Subdev API).
> -
> -* The libcamera framework is the core part of the stack. It
> - handles all control of the camera devices in its core component,
> - libcamera, and exposes a native C++ API to upper layers. Optional
> - language bindings allow interfacing to libcamera from other
> - programming languages.
> -
> - Those components live in the same source code repository and
> - all together constitute the libcamera framework.
> -
> -* The libcamera adaptation is an umbrella term designating the
> - components that interface to libcamera in other frameworks.
> - Notable examples are a V4L2 compatibility layer, a gstreamer
> - libcamera element, and an Android camera HAL implementation based
> - on libcamera.
> -
> - Those components can live in the libcamera project source code
> - in separate repositories, or move to their respective project's
> - repository (for instance the gstreamer libcamera element).
> -
> -* The applications and upper level frameworks are based on the
> - libcamera framework or libcamera adaptation, and are outside of
> - the scope of the libcamera project.
> -
> -
> libcamera Architecture
> ======================
>
> diff --git a/Documentation/documentation-contents.rst b/Documentation/documentation-contents.rst
> index a6915e05..613366d1 100644
> --- a/Documentation/documentation-contents.rst
> +++ b/Documentation/documentation-contents.rst
> @@ -4,9 +4,11 @@
>
> * :doc:`/api-html/index`
> * :doc:`/camera-sensor-model`
> + * :doc:`/camera_stack`
> * :doc:`/code-of-conduct`
> * :doc:`/coding-style`
> * :doc:`/environment_variables`
> + * :doc:`/feature_requirements`
> * :doc:`/guides/application-developer`
> * :doc:`/guides/introduction`
> * :doc:`/guides/ipa`
> diff --git a/Documentation/feature_requirements.rst b/Documentation/feature_requirements.rst
> new file mode 100644
> index 00000000..cae7e9ab
> --- /dev/null
> +++ b/Documentation/feature_requirements.rst
> @@ -0,0 +1,145 @@
> +.. SPDX-License-Identifier: CC-BY-SA-4.0
> +
> +.. include:: documentation-contents.rst
> +
> +Feature Requirements
> +====================
> +
> +Device enumeration
> +------------------
> +
> +The library shall support enumerating all camera devices available in the
> +system, including both fixed cameras and hotpluggable cameras. It shall
> +support cameras plugged and unplugged after the initialization of the
> +library, and shall offer a mechanism to notify applications of camera plug
> +and unplug.
> +
> +The following types of cameras shall be supported:
> +
> +* Internal cameras designed for point-and-shoot still image and video
> + capture usage, either controlled directly by the CPU, or exposed through
> + an internal USB bus as a UVC device.
> +
> +* External UVC cameras designed for video conferencing usage.
> +
> +Other types of camera, including analog cameras, depth cameras, thermal
> +cameras, external digital picture or movie cameras, are out of scope for
> +this project.
> +
> +A hardware device that includes independent camera sensors, such as front
> +and back sensors in a phone, shall be considered as multiple camera devices
> +for the purpose of this library.
> +
> +Independent Camera Devices
> +--------------------------
> +
> +When multiple cameras are present in the system and are able to operate
> +independently from each other, the library shall expose them as multiple
> +camera devices and support parallel operation without any additional usage
> +restriction apart from the limitations inherent to the hardware (such as
> +memory bandwidth, CPU usage or number of CSI-2 receivers for instance).
> +
> +Independent processes shall be able to use independent cameras devices
> +without interfering with each other. A single camera device shall be
> +usable by a single process at a time.
> +
> +Multiple streams support
> +------------------------
> +
> +The library shall support multiple video streams running in parallel
> +for each camera device, within the limits imposed by the system.
> +
> +Per frame controls
> +------------------
> +
> +The library shall support controlling capture parameters for each stream
> +on a per-frame basis, on a best effort basis based on the capabilities of the
> +hardware and underlying software stack (including kernel drivers and
> +firmware). It shall apply capture parameters to the frame they target, and
> +report the value of the parameters that have effectively been used for each
> +captured frame.
> +
> +When a camera device supports multiple streams, the library shall allow both
> +control of each stream independently, and control of multiple streams
> +together. Streams that are controlled together shall be synchronized. No
> +synchronization is required for streams controlled independently.
> +
> +Capability Enumeration
> +----------------------
> +
> +The library shall expose capabilities of each camera device in a way that
> +allows applications to discover those capabilities dynamically. Applications
> +shall be allowed to cache capabilities for as long as they are using the
> +library. If capabilities can change at runtime, the library shall offer a
> +mechanism to notify applications of such changes. Applications shall not
> +cache capabilities in long term storage between runs.
> +
> +Capabilities shall be discovered dynamically at runtime from the device when
> +possible, and may come, in part or in full, from platform configuration
> +data.
> +
> +Device Profiles
> +---------------
> +
> +The library may define different camera device profiles, each with a minimum
> +set of required capabilities. Applications may use those profiles to quickly
> +determine the level of features exposed by a device without parsing the full
> +list of capabilities. Camera devices may implement additional capabilities
> +on top of the minimum required set for the profile they expose.
> +
> +3A and Image Enhancement Algorithms
> +-----------------------------------
> +
> +The camera devices shall implement auto exposure, auto gain and auto white
> +balance. Camera devices that include a focus lens shall implement auto
> +focus. Additional image enhancement algorithms, such as noise reduction or
> +video stabilization, may be implemented.
> +
> +All algorithms may be implemented in hardware or firmware outside of the
> +library, or in software in the library. They shall all be controllable by
> +applications.
> +
> +The library shall be architectured to isolate the 3A and image enhancement
> +algorithms in a component with a documented API, respectively called the 3A
> +component and the 3A API. The 3A API shall be stable, and shall allow both
> +open-source and closed-source implementations of the 3A component.
> +
> +The library may include statically-linked open-source 3A components, and
> +shall support dynamically-linked open-source and closed-source 3A
> +components.
> +
> +Closed-source 3A Component Sandboxing
> +-------------------------------------
> +
> +For security purposes, it may be desired to run closed-source 3A components
> +in a separate process. The 3A API would in such a case be transported over
> +IPC. The 3A API shall make it possible to use any IPC mechanism that
> +supports passing file descriptors.
> +
> +The library may implement an IPC mechanism, and shall support third-party
> +platform-specific IPC mechanisms through the implementation of a
> +platform-specific 3A API wrapper. No modification to the library shall be
> +needed to use such third-party IPC mechanisms.
> +
> +The 3A component shall not directly access any device node on the system.
> +Such accesses shall instead be performed through the 3A API. The library
> +shall validate all accesses and restrict them to what is absolutely required
> +by 3A components.
> +
> +V4L2 Compatibility Layer
> +------------------------
> +
> +The project shall support traditional V4L2 application through an additional
> +libcamera wrapper library. The wrapper library shall trap all accesses to
> +camera devices through `LD_PRELOAD`, and route them through libcamera to
> +emulate a high-level V4L2 camera device. It shall expose camera device
> +features on a best-effort basis, and aim for the level of features
> +traditionally available from a UVC camera designed for video conferencing.
> +
> +Android Camera HAL v3 Compatibility
> +-----------------------------------
> +
> +The library API shall expose all the features required to implement an
> +Android Camera HAL v3 on top of libcamera. Some features of the HAL may be
> +omitted as long as they can be implemented separately in the HAL, such as
> +JPEG encoding, or YUV reprocessing.
> diff --git a/Documentation/index.rst b/Documentation/index.rst
> index 52ddc494..59416906 100644
> --- a/Documentation/index.rst
> +++ b/Documentation/index.rst
> @@ -16,8 +16,10 @@
>
> Application Writer's Guide <guides/application-developer>
> Camera Sensor Model <camera-sensor-model>
> + Camera Stack <camera_stack>
> Developer Guide <guides/introduction>
> Environment variables <environment_variables>
> + Feature Requirements <feature_requirements>
> IPA Writer's guide <guides/ipa>
> Lens driver requirements <lens_driver_requirements>
> Pipeline Handler Writer's Guide <guides/pipeline-handler>
> diff --git a/Documentation/meson.build b/Documentation/meson.build
> index c5736991..74cffc11 100644
> --- a/Documentation/meson.build
> +++ b/Documentation/meson.build
> @@ -124,6 +124,7 @@ endif
> if sphinx.found()
> docs_sources = [
> 'camera-sensor-model.rst',
> + 'camera_stack.rst',
> 'code-of-conduct.rst',
> 'coding-style.rst',
> 'conf.py',
> @@ -131,6 +132,7 @@ if sphinx.found()
> 'docs.rst',
> 'documentation-contents.rst',
> 'environment_variables.rst',
> + 'feature_requirements.rst','
Oops, a stray quotation mark there breaks the build with ninja -C build...but not with sphinx-build
which is what I was testing; my bad. I'll send a new version of this patch.
> 'guides/application-developer.rst',
> 'guides/introduction.rst',
> 'guides/ipa.rst',
More information about the libcamera-devel
mailing list