[PATCH v3 5/8] libcamera: simple: Consider raw output configurations
Milan Zamazal
mzamazal at redhat.com
Wed Apr 2 20:43:59 CEST 2025
Hi Robert,
thank you for testing, see my response to Kieran what's the problem.
Robert Mader <robert.mader at collabora.com> writes:
> I just tried the series on the Librem5 (in order to check if Camshark) and run into the same issue, starting with this commit:
>
>> purism-librem5:~/git/libcamera$ cam -c 1 -I
A really easy reproducer, thanks. :-)
>> [0:12:04.495109800] [3837] INFO Camera camera_manager.cpp:329 libcamera v0.4.0+194-06e2de17
>> [0:12:04.611452658] [3838] WARN IPASoft soft_simple.cpp:98 IPASoft: Failed to create camera sensor helper for hi846
>> [0:12:04.622258198] [3838] WARN CameraSensorProperties camera_sensor_properties.cpp:463 No static properties available for 's5k3l6xx'
>> [0:12:04.622611614] [3838] WARN CameraSensorProperties camera_sensor_properties.cpp:465 Please consider updating the camera sensor properties database
>> [0:12:04.623478774] [3838] WARN CameraSensor camera_sensor_legacy.cpp:669 's5k3l6xx 3-002d': Unsupported ancillary entity function 131074
>> [0:12:04.635553373] [3838] WARN IPASoft soft_simple.cpp:98 IPASoft: Failed to create camera sensor helper for s5k3l6xx
>> Using camera /base/soc at 0/bus at 30800000/i2c at 30a40000/camera at 20 as cam0
>> 0: 636x480-ABGR8888
>> * Pixelformat: ABGR8888 (2x2)-(1628x1224)/(+0,+0)
>> [0:12:04.643438097] [3837] ERROR Stream stream.cpp:215 Range format is ambiguous
>> * Pixelformat: XBGR8888 (2x2)-(1628x1224)/(+0,+0)
>> [0:12:04.643696229] [3837] ERROR Stream stream.cpp:215 Range format is ambiguous
>> * Pixelformat: BGR888 (2x2)-(1628x1224)/(+0,+0)
>> [0:12:04.643748792] [3837] ERROR Stream stream.cpp:215 Range format is ambiguous
>> * Pixelformat: RGB888 (2x2)-(1628x1224)/(+0,+0)
>> [0:12:04.643861237] [3837] ERROR Stream stream.cpp:215 Range format is ambiguous
>> * Pixelformat: ARGB8888 (2x2)-(1628x1224)/(+0,+0)
>> [0:12:04.643918479] [3837] ERROR Stream stream.cpp:215 Range format is ambiguous
>> * Pixelformat: XRGB8888 (2x2)-(1628x1224)/(+0,+0)
>> [0:12:04.643971162] [3837] ERROR Stream stream.cpp:215 Range format is ambiguous
>
> Interestingly I don't see the same on the OP6 - i.e. not every sensor appears to be affected.
>
> On 20.03.25 23:43, Kieran Bingham wrote:
>> Quoting Milan Zamazal (2025-03-05 19:26:12)
>>> When generating simple pipeline configuration, raw output configurations
>>> are generated but later filtered out. Let's include processed and/or
>>> raw output configurations depending on the requested stream roles.
>>>
>>> Raw and processed formats are handled separately. The intention is that
>>> in case both raw and processed formats are requested then the raw
>>> formats should be left intact to the extent possible and not be
>>> influenced by the processed formats (implying that raw parameters
>>> compatible with the processed output requirements must be requested by
>>> the application).
>>>
>>> Raw output configurations are marked by setting their colorSpace to
>>> ColorSpace::Raw, which is the only place there suitable for the purpose.
>>>
>>> This is another preparatory patch without making raw outputs working.
>> I've identified two interesting effects which I've bisected to this
>> patch:
>>
>> First:
>>
>> Without this patch on x13s:
>>
>> [libcamera] kbingham at charm:~/iob/libcamera/build/gcc$ cam -c1 --metadata -C10
>> [1:44:39.464209549] [37696] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path
>> [1:44:39.471858239] [37696] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+149-628e8c80
>> [1:44:39.497573824] [37697] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees
>> [1:44:39.502155101] [37697] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
>> [1:44:39.502180672] [37697] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
>> [1:44:39.502468206] [37697] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
>> [1:44:39.512547578] [37697] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml'
>> Using camera /base/soc at 0/cci at ac4c000/i2c-bus at 1/camera at 10 as cam0
>> [1:44:39.514127115] [37696] INFO Camera camera.cpp:1203 configuring streams: (0) 2584x1944-ABGR8888
>> [1:44:39.520536247] [37697] WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults.
>> [1:44:39.520899922] [37697] INFO IPASoft soft_simple.cpp:251 IPASoft: Exposure 4-2016, gain 1-15.9922 (0.149922)
>> cam0: Capture 10 frames
>> 6279.664529 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 20093184
>> SensorTimestamp = 6279664529000
>> 6279.697607 (30.23 fps) cam0-stream0 seq: 000001 bytesused: 20093184
>> SensorTimestamp = 6279697607000
>> 6279.731336 (29.65 fps) cam0-stream0 seq: 000002 bytesused: 20093184
>> SensorTimestamp = 6279731336000
>> 6279.831491 (9.98 fps) cam0-stream0 seq: 000003 bytesused: 20093184
>> SensorTimestamp = 6279831491000
>> 6279.898368 (14.95 fps) cam0-stream0 seq: 000004 bytesused: 20093184
>> SensorTimestamp = 6279898368000
>> 6279.965133 (14.98 fps) cam0-stream0 seq: 000005 bytesused: 20093184
>> SensorTimestamp = 6279965133000
>> 6280.031911 (14.97 fps) cam0-stream0 seq: 000006 bytesused: 20093184
>> SensorTimestamp = 6280031911000
>> 6280.098577 (15.00 fps) cam0-stream0 seq: 000007 bytesused: 20093184
>> SensorTimestamp = 6280098577000
>> 6280.165320 (14.98 fps) cam0-stream0 seq: 000008 bytesused: 20093184
>> SensorTimestamp = 6280165320000
>> 6280.232225 (14.95 fps) cam0-stream0 seq: 000009 bytesused: 20093184
>> SensorTimestamp = 6280232225000
>>
>> (Note the 15 FPS), and pipewire/gstreamer/cheese are all 'streaming'...
>>
>>
>> But with this patch the rate jumps to a more reasonable 30 FPS seen by
>> qcam and cam:
>>
>> [libcamera] kbingham at charm:~/iob/libcamera/build/gcc$ cam -c1 --metadata -C10
>> [1:46:59.040578435] [38346] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path
>> [1:46:59.048420757] [38346] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+150-d0e1caa0
>> [1:46:59.074259713] [38347] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees
>> [1:46:59.078796773] [38347] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
>> [1:46:59.078822605] [38347] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
>> [1:46:59.079110243] [38347] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
>> [1:46:59.087987917] [38347] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml'
>> Using camera /base/soc at 0/cci at ac4c000/i2c-bus at 1/camera at 10 as cam0
>> [1:46:59.089522196] [38346] INFO Camera camera.cpp:1203 configuring streams: (0) 1288x972-ABGR8888
>> [1:46:59.096422962] [38347] WARN CameraSensor camera_sensor_legacy.cpp:501 'ov5675 24-0010': No sensor delays found in static properties. Assuming unverified defaults.
>> [1:46:59.096786533] [38347] INFO IPASoft soft_simple.cpp:251 IPASoft: Exposure 4-2016, gain 1-15.9922 (0.149922)
>> cam0: Capture 10 frames
>> 6419.234779 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 5007744
>> SensorTimestamp = 6419234779000
>> 6419.268213 (29.91 fps) cam0-stream0 seq: 000001 bytesused: 5007744
>> SensorTimestamp = 6419268213000
>> 6419.301359 (30.17 fps) cam0-stream0 seq: 000002 bytesused: 5007744
>> SensorTimestamp = 6419301359000
>> 6419.334744 (29.95 fps) cam0-stream0 seq: 000003 bytesused: 5007744
>> SensorTimestamp = 6419334744000
>> 6419.368129 (29.95 fps) cam0-stream0 seq: 000004 bytesused: 5007744
>> SensorTimestamp = 6419368129000
>> 6419.401516 (29.95 fps) cam0-stream0 seq: 000005 bytesused: 5007744
>> SensorTimestamp = 6419401516000
>> 6419.434902 (29.95 fps) cam0-stream0 seq: 000006 bytesused: 5007744
>> SensorTimestamp = 6419434902000
>> 6419.468289 (29.95 fps) cam0-stream0 seq: 000007 bytesused: 5007744
>> SensorTimestamp = 6419468289000
>> 6419.501673 (29.95 fps) cam0-stream0 seq: 000008 bytesused: 5007744
>> SensorTimestamp = 6419501673000
>> 6419.535058 (29.95 fps) cam0-stream0 seq: 000009 bytesused: 5007744
>> SensorTimestamp = 6419535058000
>>
>>
>> But alas I note that now gstreamer (and cheese, and pipewire) now fail:
>>
>> [libcamera] kbingham at charm:~/iob/libcamera/build/gcc$ gst-launch-1.0 libcamerasrc ! autovideosink
>> Setting pipeline to PAUSED ...
>> [1:46:09.897725501] [38325] INFO IPAManager ipa_manager.cpp:137 libcamera is not installed. Adding '/home/kbingham/iob/libcamera/build/gcc/src/ipa' to the IPA search path
>> [1:46:09.903176776] [38325] INFO Camera camera_manager.cpp:327 libcamera v0.4.0+150-d0e1caa0
>> [1:46:09.935374613] [38330] WARN CameraSensor camera_sensor_legacy.cpp:616 'ov5675 24-0010': Rotation control not available, default to 0 degrees
>> [1:46:09.941677813] [38330] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
>> [1:46:09.941734112] [38330] WARN IPAProxy ipa_proxy.cpp:177 Configuration file 'ov5675.yaml' not found for IPA module 'simple', falling back to 'uncalibrated.yaml'
>> [1:46:09.942172574] [38330] INFO IPAProxy ipa_proxy.cpp:151 libcamera is not installed. Loading IPA configuration from '/home/kbingham/iob/libcamera/src/ipa/simple/data'
>> [1:46:09.949343846] [38330] INFO Pipeline pipeline_handler.cpp:618 libcamera is not installed. Loading platform configuration file from '/home/kbingham/iob/libcamera/src/libcamera/pipeline/virtual/data/virtual.yaml'
>> Pipeline is live and does not need PREROLL ...
>> Pipeline is PREROLLED ...
>> Setting pipeline to PLAYING ...
>> New clock: GstSystemClock
>> [1:46:09.950895937] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous
>> [1:46:09.950940309] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous
>> [1:46:09.950962495] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous
>> [1:46:09.950982129] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous
>> [1:46:09.951001711] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous
>> [1:46:09.951020564] [38331] ERROR Stream stream.cpp:215 Range format is ambiguous
>> ERROR: from element /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0: Internal data stream error.
>> Additional debug info:
>> ../../src/gstreamer/gstlibcamerasrc.cpp(662): gst_libcamera_src_task_enter (): /GstPipeline:pipeline0/GstLibcameraSrc:libcamerasrc0:
>> streaming stopped, reason not-negotiated (-4)
>> Execution ended after 0:00:00.000438306
>> Setting pipeline to NULL ...
>> Freeing pipeline ...
>>
>> Debugging a little, at stream.ccp:215
>>
>> diff --git a/src/libcamera/stream.cpp b/src/libcamera/stream.cpp
>> index 978d72752b00..22f348c2df87 100644
>> --- a/src/libcamera/stream.cpp
>> +++ b/src/libcamera/stream.cpp
>> @@ -213,6 +213,8 @@ std::vector<Size> StreamFormats::sizes(const PixelFormat &pixelformat) const
>> if (!discrete) {
>> if (ranges.size() != 1) {
>> LOG(Stream, Error) << "Range format is ambiguous";
>> + for (auto range: ranges)
>> + LOG(Stream, Error) << range;
>> return {};
>> }
>>
>> Reports:
>>
>> [1:51:01.855863590] [40484] ERROR Stream stream.cpp:215 Range format is ambiguous
>> [1:51:01.855901452] [40484] ERROR Stream stream.cpp:217 (4x2)-(1288x972)/(+4,+2)
>> [1:51:01.855920305] [40484] ERROR Stream stream.cpp:217 (4x2)-(2584x1944)/(+4,+2)
>>
>> So we're somehow setting up the stream capabilities incorrectly here
>> which breaks things... (note that ranges.size is 2, not 1?)
>>
>> Perhaps we're incorrectly adding the raw stream sizes to the processed stream
>> sizes ?
>>
>> --
>> Kieran
>>
>>
>>> Signed-off-by: Milan Zamazal <mzamazal at redhat.com>
>>> ---
>>> src/libcamera/pipeline/simple/simple.cpp | 72 ++++++++++++++++--------
>>> 1 file changed, 49 insertions(+), 23 deletions(-)
>>>
>>> diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp
>>> index 0c6e0f44..92b9608b 100644
>>> --- a/src/libcamera/pipeline/simple/simple.cpp
>>> +++ b/src/libcamera/pipeline/simple/simple.cpp
>>> @@ -25,6 +25,7 @@
>>> #include <libcamera/base/log.h>
>>> #include <libcamera/camera.h>
>>> +#include <libcamera/color_space.h>
>>> #include <libcamera/control_ids.h>
>>> #include <libcamera/pixel_format.h>
>>> #include <libcamera/request.h>
>>> @@ -389,6 +390,8 @@ private:
>>> const MediaPad *acquirePipeline(SimpleCameraData *data);
>>> void releasePipeline(SimpleCameraData *data);
>>> + void setUpFormatSizes(std::map<PixelFormat, std::vector<SizeRange>> formats);
>>> +
>>> std::map<const MediaEntity *, EntityData> entities_;
>>> MediaDevice *converter_;
>>> @@ -1191,15 +1194,56 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span<const StreamRo
>>> data->processedRequested_ = true;
>>> }
>>> - /* Create the formats map. */
>>> - std::map<PixelFormat, std::vector<SizeRange>> formats;
>>> + /* Create the formats maps. */
>>> + std::map<PixelFormat, std::vector<SizeRange>> processedFormats;
>>> + std::map<PixelFormat, std::vector<SizeRange>> rawFormats;
>>> for (const SimpleCameraData::Configuration &cfg : data->configs_)
>>> - if (!cfg.raw)
>>> - for (PixelFormat format : cfg.outputFormats)
>>> - formats[format].push_back(cfg.outputSizes);
>>> + for (PixelFormat format : cfg.outputFormats)
>>> + (cfg.raw ? rawFormats : processedFormats)[format].push_back(cfg.outputSizes);
>>> +
>>> + if (data->processedRequested_ && processedFormats.empty()) {
>>> + LOG(SimplePipeline, Error)
>>> + << "Processed stream requsted but no corresponding output configuration found";
>>> + return nullptr;
>>> + }
>>> + if (data->rawRequested_ && rawFormats.empty()) {
>>> + LOG(SimplePipeline, Error)
>>> + << "Raw stream requsted but no corresponding output configuration found";
>>> + return nullptr;
>>> + }
>>> +
>>> + setUpFormatSizes(processedFormats);
>>> + setUpFormatSizes(rawFormats);
>>> +
>>> + /*
>>> + * Create the stream configurations. Take the first entry in the formats
>>> + * map as the default, for lack of a better option.
>>> + *
>>> + * \todo Implement a better way to pick the default format
>>> + */
>>> + for (StreamRole role : roles) {
>>> + bool raw = (role == StreamRole::Raw);
>>> + auto formats = (raw ? rawFormats : processedFormats);
>>> + StreamConfiguration cfg{ StreamFormats{ formats } };
>>> + cfg.pixelFormat = formats.begin()->first;
>>> + cfg.size = formats.begin()->second[0].max;
>>> + if (raw)
>>> + cfg.colorSpace = ColorSpace::Raw;
>>> +
>>> + config->addConfiguration(cfg);
>>> + }
>>> +
>>> + config->validate();
>>> +
>>> + return config;
>>> +}
>>> +void SimplePipelineHandler::setUpFormatSizes(
>>> + std::map<PixelFormat, std::vector<SizeRange>> formats)
>>> +{
>>> /* Sort the sizes and merge any consecutive overlapping ranges. */
>>> +
>>> for (auto &[format, sizes] : formats) {
>>> std::sort(sizes.begin(), sizes.end(),
>>> [](SizeRange &a, SizeRange &b) {
>>> @@ -1219,24 +1263,6 @@ SimplePipelineHandler::generateConfiguration(Camera *camera, Span<const StreamRo
>>> sizes.erase(++cur, sizes.end());
>>> }
>>> -
>>> - /*
>>> - * Create the stream configurations. Take the first entry in the formats
>>> - * map as the default, for lack of a better option.
>>> - *
>>> - * \todo Implement a better way to pick the default format
>>> - */
>>> - for ([[maybe_unused]] StreamRole role : roles) {
>>> - StreamConfiguration cfg{ StreamFormats{ formats } };
>>> - cfg.pixelFormat = formats.begin()->first;
>>> - cfg.size = formats.begin()->second[0].max;
>>> -
>>> - config->addConfiguration(cfg);
>>> - }
>>> -
>>> - config->validate();
>>> -
>>> - return config;
>>> }
>>> int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c)
>>> -- 2.48.1
>>>
More information about the libcamera-devel
mailing list