[PATCH v3 5/8] libcamera: simple: Consider raw output configurations
Robert Mader
robert.mader at collabora.com
Wed Mar 26 23:54:41 CET 2025
I just tried the series on the Librem5 (in order to check if Camshark)
and run into the same issue, starting with this commit:
> purism-librem5:~/git/libcamera$ cam -c 1 -I
> [0:12:04.495109800] [3837] INFO Camera camera_manager.cpp:329
> libcamera v0.4.0+194-06e2de17
> [0:12:04.611452658] [3838] WARN IPASoft soft_simple.cpp:98 IPASoft:
> Failed to create camera sensor helper for hi846
> [0:12:04.622258198] [3838] WARN CameraSensorProperties
> camera_sensor_properties.cpp:463 No static properties available for
> 's5k3l6xx'
> [0:12:04.622611614] [3838] WARN CameraSensorProperties
> camera_sensor_properties.cpp:465 Please consider updating the camera
> sensor properties database
> [0:12:04.623478774] [3838] WARN CameraSensor
> camera_sensor_legacy.cpp:669 's5k3l6xx 3-002d': Unsupported ancillary
> entity function 131074
> [0:12:04.635553373] [3838] WARN IPASoft soft_simple.cpp:98 IPASoft:
> Failed to create camera sensor helper for s5k3l6xx
> Using camera /base/soc at 0/bus at 30800000/i2c at 30a40000/camera at 20 as cam0
> 0: 636x480-ABGR8888
> * Pixelformat: ABGR8888 (2x2)-(1628x1224)/(+0,+0)
> [0:12:04.643438097] [3837] ERROR Stream stream.cpp:215 Range format is
> ambiguous
> * Pixelformat: XBGR8888 (2x2)-(1628x1224)/(+0,+0)
> [0:12:04.643696229] [3837] ERROR Stream stream.cpp:215 Range format is
> ambiguous
> * Pixelformat: BGR888 (2x2)-(1628x1224)/(+0,+0)
> [0:12:04.643748792] [3837] ERROR Stream stream.cpp:215 Range format is
> ambiguous
> * Pixelformat: RGB888 (2x2)-(1628x1224)/(+0,+0)
> [0:12:04.643861237] [3837] ERROR Stream stream.cpp:215 Range format is
> ambiguous
> * Pixelformat: ARGB8888 (2x2)-(1628x1224)/(+0,+0)
> [0:12:04.643918479] [3837] ERROR Stream stream.cpp:215 Range format is
> ambiguous
> * Pixelformat: XRGB8888 (2x2)-(1628x1224)/(+0,+0)
> [0:12:04.643971162] [3837] ERROR Stream stream.cpp:215 Range format is
> ambiguous
Interestingly I don't see the same on the OP6 - i.e. not every sensor
appears to be affected.
On 20.03.25 23:43, Kieran Bingham wrote:
> Quoting Milan Zamazal (2025-03-05 19:26:12)
>> When generating simple pipeline configuration, raw output configurations
>> are generated but later filtered out. Let's include processed and/or
>> raw output configurations depending on the requested stream roles.
>>
>> Raw and processed formats are handled separately. The intention is that
>> in case both raw and processed formats are requested then the raw
>> formats should be left intact to the extent possible and not be
>> influenced by the processed formats (implying that raw parameters
>> compatible with the processed output requirements must be requested by
>> the application).
>>
>> Raw output configurations are marked by setting their colorSpace to
>> ColorSpace::Raw, which is the only place there suitable for the purpose.
>>
>> This is another preparatory patch without making raw outputs working.
> I've identified two interesting effects which I've bisected to this
> patch:
>
> First:
>
> Without this patch on x13s:
>
> [libcamera] kbingham at charm:~/iob/libcamera/build/gcc$ cam -c1 --metadata -C10
> [1:44:39.464209549] [37696] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path
> [1:44:39.471858239] [37696] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+149-628e8c80
> [1:44:39.497573824] [37697] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees
> [1:44:39.502155101] [37697] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> [1:44:39.502180672] [37697] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> [1:44:39.502468206] [37697] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> [1:44:39.512547578] [37697] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml'
> Using camera /base/soc at 0/cci at ac4c000/i2c-bus at 1/camera at 10 as cam0
> [1:44:39.514127115] [37696] INFO Camera camera.cpp:1203 configuring streams: (0) 2584x1944-ABGR8888
> [1:44:39.520536247] [37697] WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults.
> [1:44:39.520899922] [37697] INFO IPASoft soft_simple.cpp:251 IPASoft: Exposure 4-2016, gain 1-15.9922 (0.149922)
> cam0: Capture 10 frames
> 6279.664529 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 20093184
> SensorTimestamp = 6279664529000
> 6279.697607 (30.23 fps) cam0-stream0 seq: 000001 bytesused: 20093184
> SensorTimestamp = 6279697607000
> 6279.731336 (29.65 fps) cam0-stream0 seq: 000002 bytesused: 20093184
> SensorTimestamp = 6279731336000
> 6279.831491 (9.98 fps) cam0-stream0 seq: 000003 bytesused: 20093184
> SensorTimestamp = 6279831491000
> 6279.898368 (14.95 fps) cam0-stream0 seq: 000004 bytesused: 20093184
> SensorTimestamp = 6279898368000
> 6279.965133 (14.98 fps) cam0-stream0 seq: 000005 bytesused: 20093184
> SensorTimestamp = 6279965133000
> 6280.031911 (14.97 fps) cam0-stream0 seq: 000006 bytesused: 20093184
> SensorTimestamp = 6280031911000
> 6280.098577 (15.00 fps) cam0-stream0 seq: 000007 bytesused: 20093184
> SensorTimestamp = 6280098577000
> 6280.165320 (14.98 fps) cam0-stream0 seq: 000008 bytesused: 20093184
> SensorTimestamp = 6280165320000
> 6280.232225 (14.95 fps) cam0-stream0 seq: 000009 bytesused: 20093184
> SensorTimestamp = 6280232225000
>
> (Note the 15 FPS), and pipewire/gstreamer/cheese are all 'streaming'...
>
>
> But with this patch the rate jumps to a more reasonable 30 FPS seen by
> qcam and cam:
>
> [libcamera] kbingham at charm:~/iob/libcamera/build/gcc$ cam -c1 --metadata -C10
> [1:46:59.040578435] [38346] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path
> [1:46:59.048420757] [38346] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+150-d0e1caa0
> [1:46:59.074259713] [38347] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees
> [1:46:59.078796773] [38347] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> [1:46:59.078822605] [38347] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> [1:46:59.079110243] [38347] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> [1:46:59.087987917] [38347] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml'
> Using camera /base/soc at 0/cci at ac4c000/i2c-bus at 1/camera at 10 as cam0
> [1:46:59.089522196] [38346] INFO Camera camera.cpp:1203 configuring streams: (0) 1288x972-ABGR8888
> [1:46:59.096422962] [38347] WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults.
> [1:46:59.096786533] [38347] INFO IPASoft soft_simple.cpp:251 IPASoft: Exposure 4-2016, gain 1-15.9922 (0.149922)
> cam0: Capture 10 frames
> 6419.234779 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 5007744
> SensorTimestamp = 6419234779000
> 6419.268213 (29.91 fps) cam0-stream0 seq: 000001 bytesused: 5007744
> SensorTimestamp = 6419268213000
> 6419.301359 (30.17 fps) cam0-stream0 seq: 000002 bytesused: 5007744
> SensorTimestamp = 6419301359000
> 6419.334744 (29.95 fps) cam0-stream0 seq: 000003 bytesused: 5007744
> SensorTimestamp = 6419334744000
> 6419.368129 (29.95 fps) cam0-stream0 seq: 000004 bytesused: 5007744
> SensorTimestamp = 6419368129000
> 6419.401516 (29.95 fps) cam0-stream0 seq: 000005 bytesused: 5007744
> SensorTimestamp = 6419401516000
> 6419.434902 (29.95 fps) cam0-stream0 seq: 000006 bytesused: 5007744
> SensorTimestamp = 6419434902000
> 6419.468289 (29.95 fps) cam0-stream0 seq: 000007 bytesused: 5007744
> SensorTimestamp = 6419468289000
> 6419.501673 (29.95 fps) cam0-stream0 seq: 000008 bytesused: 5007744
> SensorTimestamp = 6419501673000
> 6419.535058 (29.95 fps) cam0-stream0 seq: 000009 bytesused: 5007744
> SensorTimestamp = 6419535058000
>
>
> But alas I note that now gstreamer (and cheese, and pipewire) now fail:
>
> [libcamera] kbingham at charm:~/iob/libcamera/build/gcc$ gst-launch-1.0 libcamerasrc ! autovideosink
> Setting pipeline to PAUSED ...
> [1:46:09.897725501] [38325] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path
> [1:46:09.903176776] [38325] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+150-d0e1caa0
> [1:46:09.935374613] [38330] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees
> [1:46:09.941677813] [38330] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> [1:46:09.941734112] [38330] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
> [1:46:09.942172574] [38330] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
> [1:46:09.949343846] [38330] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml'
> Pipeline is live and does not need PREROLL ...
> Pipeline is PREROLLED ...
> Setting pipeline to PLAYING ...
> New clock: GstSystemClock
> [1:46:09.950895937] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous
> [1:46:09.950940309] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous
> [1:46:09.950962495] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous
> [1:46:09.950982129] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous
> [1:46:09.951001711] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous
> [1:46:09.951020564] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous
> ERROR: from element /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0: Internal data stream error.
> Additional debug info:
> ../../src/gstreamer/gstlibcamerasrc.cpp(662): gst_libcamera_src_task_enter (): /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0:
> streaming stopped, reason not-negotiated (-4)
> Execution ended after 0:00:00.000438306
> Setting pipeline to NULL ...
> Freeing pipeline ...
>
> Debugging a little, at stream.ccp:215
>
> diff --git a/src/libcamera/stream.cpp b/src/libcamera/stream.cpp
> index 978d72752b00..22f348c2df87 100644
> --- a/src/libcamera/stream.cpp
> +++ b/src/libcamera/stream.cpp
> @@ -213,6 +213,8 @@ std::vector<Size> StreamFormats::sizes(const PixelFormat &pixelformat) const
> if (!discrete) {
> if (ranges.size() != 1) {
> LOG(Stream, Error) << "Range format is ambiguous";
> + for (auto range: ranges)
> + LOG(Stream, Error) << range;
> return {};
> }
>
> Reports:
>
> [1:51:01.855863590] [40484] ERROR Stream stream.cpp:215 Range format is ambiguous
> [1:51:01.855901452] [40484] ERROR Stream stream.cpp:217 (4x2)-(1288x972)/(+4,+2)
> [1:51:01.855920305] [40484] ERROR Stream stream.cpp:217 (4x2)-(2584x1944)/(+4,+2)
>
> So we're somehow setting up the stream capabilities incorrectly here
> which breaks things... (note that ranges.size is 2, not 1?)
>
> Perhaps we're incorrectly adding the raw stream sizes to the processed stream
> sizes ?
>
> --
> Kieran
>
>
>> Signed-off-by: Milan Zamazal <mzamazal at redhat.com>
>> ---
>> src/libcamera/pipeline/simple/simple.cpp | 72 ++++++++++++++++--------
>> 1 file changed, 49 insertions(+), 23 deletions(-)
>>
>> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp
>> index 0c6e0f44..92b9608b 100644
>> --- a/src/libcamera/pipeline/simple/simple.cpp
>> +++ b/src/libcamera/pipeline/simple/simple.cpp
>> @@ -25,6 +25,7 @@
>> #include <libcamera/base/log.h>
>>
>> #include <libcamera/camera.h>
>> +#include <libcamera/color_space.h>
>> #include <libcamera/control_ids.h>
>> #include <libcamera/pixel_format.h>
>> #include <libcamera/request.h>
>> @@ -389,6 +390,8 @@ private:
>> const MediaPad *acquirePipeline(SimpleCameraData *data);
>> void releasePipeline(SimpleCameraData *data);
>>
>> + void setUpFormatSizes(std::map<PixelFormat, std::vector<SizeRange>> formats);
>> +
>> std::map<const MediaEntity *, EntityData> entities_;
>>
>> MediaDevice *converter_;
>> @@ -1191,15 +1194,56 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span<const StreamRo
>> data->processedRequested_ = true;
>> }
>>
>> - /* Create the formats map. */
>> - std::map<PixelFormat, std::vector<SizeRange>> formats;
>> + /* Create the formats maps. */
>> + std::map<PixelFormat, std::vector<SizeRange>> processedFormats;
>> + std::map<PixelFormat, std::vector<SizeRange>> rawFormats;
>>
>> for (const SimpleCameraData::Configuration &cfg : data->configs_)
>> - if (!cfg.raw)
>> - for (PixelFormat format : cfg.outputFormats)
>> - formats[format].push_back(cfg.outputSizes);
>> + for (PixelFormat format : cfg.outputFormats)
>> + (cfg.raw ? rawFormats : processedFormats)[format].push_back(cfg.outputSizes);
>> +
>> + if (data->processedRequested_ && processedFormats.empty()) {
>> + LOG(SimplePipeline, Error)
>> + << "Processed stream requsted but no corresponding output configuration found";
>> + return nullptr;
>> + }
>> + if (data->rawRequested_ && rawFormats.empty()) {
>> + LOG(SimplePipeline, Error)
>> + << "Raw stream requsted but no corresponding output configuration found";
>> + return nullptr;
>> + }
>> +
>> + setUpFormatSizes(processedFormats);
>> + setUpFormatSizes(rawFormats);
>> +
>> + /*
>> + * Create the stream configurations. Take the first entry in the formats
>> + * map as the default, for lack of a better option.
>> + *
>> + * \todo Implement a better way to pick the default format
>> + */
>> + for (StreamRole role : roles) {
>> + bool raw = (role == StreamRole::Raw);
>> + auto formats = (raw ? rawFormats : processedFormats);
>> + StreamConfiguration cfg{ StreamFormats{ formats } };
>> + cfg.pixelFormat = formats.begin()->first;
>> + cfg.size = formats.begin()->second[0].max;
>> + if (raw)
>> + cfg.colorSpace = ColorSpace::Raw;
>> +
>> + config->addConfiguration(cfg);
>> + }
>> +
>> + config->validate();
>> +
>> + return config;
>> +}
>>
>> +void SimplePipelineHandler::setUpFormatSizes(
>> + std::map<PixelFormat, std::vector<SizeRange>> formats)
>> +{
>> /* Sort the sizes and merge any consecutive overlapping ranges. */
>> +
>> for (auto &[format, sizes] : formats) {
>> std::sort(sizes.begin(), sizes.end(),
>> [](SizeRange &a, SizeRange &b) {
>> @@ -1219,24 +1263,6 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span<const StreamRo
>>
>> sizes.erase(++cur, sizes.end());
>> }
>> -
>> - /*
>> - * Create the stream configurations. Take the first entry in the formats
>> - * map as the default, for lack of a better option.
>> - *
>> - * \todo Implement a better way to pick the default format
>> - */
>> - for ([[maybe_unused]] StreamRole role : roles) {
>> - StreamConfiguration cfg{ StreamFormats{ formats } };
>> - cfg.pixelFormat = formats.begin()->first;
>> - cfg.size = formats.begin()->second[0].max;
>> -
>> - config->addConfiguration(cfg);
>> - }
>> -
>> - config->validate();
>> -
>> - return config;
>> }
>>
>> int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c)
>> --
>> 2.48.1
>>
More information about the libcamera-devel
mailing list