XISP : Creating new Pipeline Handler for AMD/Xilinx HLS-based ISP

[Mario Bergeron]

Barnabas,
Oops, I was specifying "-c1" instead of "-C1" ... very embarrassing ...
thank you for clearing that up for me ...
I am back on track, thanks to you.
Cheers,
Mario Bergeron (AlbertaBeef)

On Tue, Jan 14, 2025 at 9:55 AM Barnabás Pőcze <pobrn at protonmail.com> wrote:

> Hi
>
>
> 2025. január 14., kedd 15:43 keltezéssel, Mario Bergeron <
> grouby177 at gmail.com> írta:
>
> > Hello,
> >
> > I have a design with four (4) capture pipelines implemented in the PL of
> a Zynq-UltraScale+ device. The capture pipelines include the following
> components:
> > - CAM : imx219, imx708, imx500, imx477
> > - CSI : MIPI CSI-2 receiver
> > - XISP : HLS-based ISP
> > - VPSS : for color space conversion and image resizing
> > - VCAP : Video Frame Buffer Write (DMA)
> >
> > These come up with the following names:
> >
> > // Additional context to what is being searched
> > // driver = "xilinx-video"
> > //
> > // Capture Pipeline 0
> > // V4l2VideoDevice = "vcap_mipi_0_v_proc output 0"
> > // V4l2Subdevice = "imx219 1-0010"
> > // V4l2Subdevice = "80050000.mipi_csi2_rx_subsystem"
> > // V4l2Subdevice = "a0010000.ISPPipeline_accel"
> > // V4l2Subdevice = "a0040000.v_proc_ss"
> > //
> > // Capture Pipeline 1
> > // V4l2VideoDevice = "vcap_mipi_1_v_proc output 0"
> > // V4l2Subdevice = "imx708"
> > // V4l2Subdevice = "dw9807 2-000c"
> > // V4l2Subdevice = "80051000.mipi_csi2_rx_subsystem"
> > // V4l2Subdevice = "a0030000.ISPPipeline_accel"
> > // V4l2Subdevice = "a00c0000.v_proc_ss"
> > //
> > // Capture Pipeline 2
> > // V4l2VideoDevice = "vcap_mipi_2_v_proc output 0"
> > // V4l2Subdevice = "imx500 3-001a"
> > // V4l2Subdevice = "80052000.mipi_csi2_rx_subsystem"
> > // V4l2Subdevice = "a0090000.ISPPipeline_accel"
> > // V4l2Subdevice = "a0100000.v_proc_ss"
> > //
> > // Capture Pipeline 3
> > // V4l2VideoDevice = "vcap_mipi_3_v_proc output 0"
> > // V4l2Subdevice = "imx477 4-001a"
> > // V4l2Subdevice = "80053000.mipi_csi2_rx_subsystem"
> > // V4l2Subdevice = "a00b0000.ISPPipeline_accel"
> > // V4l2Subdevice = "a0180000.v_proc_ss"
> >
> > I am able to match the V4L2VideoDevice and V4L2Subdevices during "match"
> phase, and enumerate my camera pipelines:
> >
> > # LIBCAMERA_LOG_LEVELS=XISP:0 cam -l
> > [43:24:27.675100583] [5666] INFO IPAManager ipa_manager.cpp:137
> libcamera is not installed. Adding
> '/home/root/libcamera.xisp_take01/build/src/ipa' to the IPA search path
> > [43:24:27.675839535] [5666] WARN IPAManager ipa_manager.cpp:148 No IPA
> found in '/usr/local/lib/libcamera'
> > [43:24:27.675941856] [5666] INFO Camera camera_manager.cpp:327 libcamera
> v0.4.0+19-91de5502-dirty (2025-01-14T14:36:27UTC)
> > [43:24:27.745595071] [5667] DEBUG XISP xisp.cpp:1153
> [PipelineHandlerXISP::match] Looking for capture pipeline
> > [43:24:27.745819631] [5667] DEBUG XISP xisp.cpp:1234 VCAP :
> vcap_mipi_0_v_proc output 0
> > [43:24:27.745914892] [5667] DEBUG XISP xisp.cpp:1221 CSI :
> 80050000.mipi_csi2_rx_subsystem
> > [43:24:27.745987572] [5667] DEBUG XISP xisp.cpp:1185 CAM : imx219 1-0010
> > [43:24:27.746046542] [5667] DEBUG XISP xisp.cpp:1190 IMX219 :
> 1920x1080-SRGGB10_1X10
> > [43:24:27.746110092] [5667] DEBUG XISP xisp.cpp:1226 XISP :
> a0010000.ISPPipeline_accel
> > [43:24:27.746171352] [5667] DEBUG XISP xisp.cpp:1230 VPSS :
> a0040000.v_proc_ss
> > [43:24:27.755114607] [5667] DEBUG XISP xisp.cpp:1153
> [PipelineHandlerXISP::match] Looking for capture pipeline
> > [43:24:27.755278268] [5667] DEBUG XISP xisp.cpp:1234 VCAP :
> vcap_mipi_1_v_proc output 0
> > [43:24:27.755350048] [5667] DEBUG XISP xisp.cpp:1221 CSI :
> 80051000.mipi_csi2_rx_subsystem
> > [43:24:27.755412538] [5667] DEBUG XISP xisp.cpp:1185 CAM : imx708
> > [43:24:27.755471358] [5667] DEBUG XISP xisp.cpp:1197 IMX708 :
> 1536x864-SRGGB10_1X10
> > [43:24:27.755533129] [5667] DEBUG XISP xisp.cpp:1217 VCM : dw9807 2-000c
> > [43:24:27.755596879] [5667] DEBUG XISP xisp.cpp:1226 XISP :
> a0030000.ISPPipeline_accel
> > [43:24:27.755657829] [5667] DEBUG XISP xisp.cpp:1230 VPSS :
> a00c0000.v_proc_ss
> > [43:24:27.765771027] [5667] DEBUG XISP xisp.cpp:1153
> [PipelineHandlerXISP::match] Looking for capture pipeline
> > [43:24:27.765926408] [5667] DEBUG XISP xisp.cpp:1234 VCAP :
> vcap_mipi_2_v_proc output 0
> > [43:24:27.765996008] [5667] DEBUG XISP xisp.cpp:1185 CAM : imx500 3-001a
> > [43:24:27.766056888] [5667] DEBUG XISP xisp.cpp:1211 IMX500 :
> 2028x1520-SRGGB10_1X10
> > [43:24:27.766120928] [5667] DEBUG XISP xisp.cpp:1221 CSI :
> 80052000.mipi_csi2_rx_subsystem
> > [43:24:27.766183828] [5667] DEBUG XISP xisp.cpp:1226 XISP :
> a0090000.ISPPipeline_accel
> > [43:24:27.766244829] [5667] DEBUG XISP xisp.cpp:1230 VPSS :
> a0100000.v_proc_ss
> > [43:24:27.773600649] [5667] WARN CameraSensorProperties
> camera_sensor_properties.cpp:463 No static properties available for 'imx500'
> > [43:24:27.773693479] [5667] WARN CameraSensorProperties
> camera_sensor_properties.cpp:465 Please consider updating the camera sensor
> properties database
> > [43:24:27.773874120] [5667] DEBUG XISP xisp.cpp:1153
> [PipelineHandlerXISP::match] Looking for capture pipeline
> > [43:24:27.774010170] [5667] DEBUG XISP xisp.cpp:1234 VCAP :
> vcap_mipi_3_v_proc output 0
> > [43:24:27.774081241] [5667] DEBUG XISP xisp.cpp:1221 CSI :
> 80053000.mipi_csi2_rx_subsystem
> > [43:24:27.774146091] [5667] DEBUG XISP xisp.cpp:1185 CAM : imx477 4-001a
> > [43:24:27.774205011] [5667] DEBUG XISP xisp.cpp:1204 IMX477 :
> 1332x990-SRGGB10_1X10
> > [43:24:27.774268071] [5667] DEBUG XISP xisp.cpp:1226 XISP :
> a00b0000.ISPPipeline_accel
> > [43:24:27.774331881] [5667] DEBUG XISP xisp.cpp:1230 VPSS :
> a0180000.v_proc_ss
> > [43:24:27.782596404] [5667] DEBUG XISP xisp.cpp:1153
> [PipelineHandlerXISP::match] Looking for capture pipeline
> > Available cameras:
> > 1: 'imx219' (/base/pl-bus/i2c at 80040000/sensor at 10)
> > 2: 'imx708' (/base/pl-bus/i2c at 80070000/sensor at 10)
> > 3: 'imx500' (/base/pl-bus/i2c at 80090000/sensor at 1a)
> > 4: 'imx477' (/base/pl-bus/i2c at 800b0000/sensor at 1a)
> >
> > I have unexpected "Defer due to missing dependencies" messages, which is
> the V4L2VideoDevice entity:
> >
> > ...
> > [32:28:31.627641623] [4979] DEBUG Camera camera_manager.cpp:73 Starting
> camera manager
> > [32:28:31.678492579] [4979] DEBUG DeviceEnumerator
> device_enumerator.cpp:242 New media device "xilinx-video" created from
> /dev/media0
> > [32:28:31.681069857] [4979] DEBUG DeviceEnumerator
> device_enumerator_udev.cpp:96 Defer media device /dev/media0 due to 1
> missing dependencies
> > [32:28:31.681175337] [4979] DEBUG DeviceEnumerator
> device_enumerator_udev.cpp:105 Missing dependency = vcap_mipi_0_v_proc
> output 0
> > [32:28:31.682177024] [4979] DEBUG DeviceEnumerator
> device_enumerator_udev.cpp:330 All dependencies for media device
> /dev/media0 found
> > [32:28:31.682256265] [4979] DEBUG DeviceEnumerator
> device_enumerator.cpp:270 Added device /dev/media0: xilinx-video
> > [32:28:31.682849299] [4979] DEBUG DeviceEnumerator
> device_enumerator.cpp:242 New media device "xilinx-video" created from
> /dev/media1
> > [32:28:31.686341484] [4979] DEBUG DeviceEnumerator
> device_enumerator_udev.cpp:96 Defer media device /dev/media1 due to 1
> missing dependencies
> > [32:28:31.686433004] [4979] DEBUG DeviceEnumerator
> device_enumerator_udev.cpp:105 Missing dependency = vcap_mipi_1_v_proc
> output 0
> > [32:28:31.687418801] [4979] DEBUG DeviceEnumerator
> device_enumerator_udev.cpp:330 All dependencies for media device
> /dev/media1 found
> > [32:28:31.687493362] [4979] DEBUG DeviceEnumerator
> device_enumerator.cpp:270 Added device /dev/media1: xilinx-video
> > [32:28:31.688111086] [4979] DEBUG DeviceEnumerator
> device_enumerator.cpp:242 New media device "xilinx-video" created from
> /dev/media2
> > [32:28:31.690556563] [4979] DEBUG DeviceEnumerator
> device_enumerator_udev.cpp:96 Defer media device /dev/media2 due to 1
> missing dependencies
> > [32:28:31.690670334] [4979] DEBUG DeviceEnumerator
> device_enumerator_udev.cpp:105 Missing dependency = vcap_mipi_2_v_proc
> output 0
> > [32:28:31.691687431] [4979] DEBUG DeviceEnumerator
> device_enumerator_udev.cpp:330 All dependencies for media device
> /dev/media2 found
> > [32:28:31.691797452] [4979] DEBUG DeviceEnumerator
> device_enumerator.cpp:270 Added device /dev/media2: xilinx-video
> > [32:28:31.692409806] [4979] DEBUG DeviceEnumerator
> device_enumerator.cpp:242 New media device "xilinx-video" created from
> /dev/media3
> > [32:28:31.694842583] [4979] DEBUG DeviceEnumerator
> device_enumerator_udev.cpp:96 Defer media device /dev/media3 due to 1
> missing dependencies
> > [32:28:31.694955104] [4979] DEBUG DeviceEnumerator
> device_enumerator_udev.cpp:105 Missing dependency = vcap_mipi_3_v_proc
> output 0
> > [32:28:31.695980181] [4979] DEBUG DeviceEnumerator
> device_enumerator_udev.cpp:330 All dependencies for media device
> /dev/media3 found
> > [32:28:31.696112512] [4979] DEBUG DeviceEnumerator
> device_enumerator.cpp:270 Added device /dev/media3: xilinx-video
> > [32:28:31.697657323] [4979] DEBUG Camera camera_manager.cpp:140 Found
> registered pipeline handler 'xisp'
> > ...
>
> See the messages after: "All dependencies for media device /dev/mediaX
> found".
> So it seems to me that everything is in order, four media devices
> successfully discovered.
>
>
> >
> > I also get these unexpected messages:
> >
> > ...
> > [32:28:31.709995969] [4979] DEBUG CameraSensor camera_sensor_raw.cpp:210
> imx219 1-0010: unsupported number of sinks (0) or sources (2)
> > ...
> > [32:28:31.728432108] [4979] DEBUG CameraSensor camera_sensor_raw.cpp:210
> imx708: unsupported number of sinks (0) or sources (2)
> > ...
> > [32:28:31.746255953] [4979] DEBUG CameraSensor camera_sensor_raw.cpp:210
> imx500 3-001a: unsupported number of sinks (0) or sources (2)
> > [32:28:31.750244771] [4979] WARN CameraSensorProperties
> camera_sensor_properties.cpp:463 No static properties available for 'imx500'
> > [32:28:31.750356602] [4979] WARN CameraSensorProperties
> camera_sensor_properties.cpp:465 Please consider updating the camera sensor
> properties database
> > ...
> > [32:28:31.762326825] [4979] DEBUG CameraSensor camera_sensor_raw.cpp:210
> imx477 4-001a: unsupported number of sinks (0) or sources (2)
> > ...
> >
> >
> > I need to know if the previous messages are normal, or need to be
> resolved.
> >
> >
> > When I try to capture images, I can see the "generateConfigration" and
> "validate" phases being run, but get a "Failed to Acquire" error before the
> "configure" phase ...
> >
> > # LIBCAMERA_LOG_LEVELS=XISP:0 cam --camera /base/pl-bus/i2c at 80040000
> /sensor at 10 -c1
> > ...
> > [43:26:44.685016474] [5673] DEBUG XISP xisp.cpp:1153
> [PipelineHandlerXISP::match] Looking for capture pipeline
> > ...
> > [43:26:44.723683858] [5672] DEBUG XISP xisp.cpp:949
> [PipelineHandlerXISP::generateConfiguration] Generate Configuration
> > [43:26:44.723784408] [5672] DEBUG XISP xisp.cpp:634
> [PipelineHandlerXISP::validate] Validating Configuration
> > [43:26:44.723851918] [5672] DEBUG XISP xisp.cpp:678 Computed sensor
> configuration: 1920x1080-UYVY8_1X16
> > [43:26:44.723925599] [5672] DEBUG XISP xisp.cpp:730 Selected sensor
> format: 1920x1080-UYVY8_1X16
> > [43:26:44.724095059] [5672] DEBUG XISP xisp.cpp:634
> [PipelineHandlerXISP::validate] Validating Configuration
> > [43:26:44.724171159] [5672] DEBUG XISP xisp.cpp:678 Computed sensor
> configuration: 1920x1080-UYVY8_1X16
> > [43:26:44.724235059] [5672] DEBUG XISP xisp.cpp:730 Selected sensor
> format: 1920x1080-UYVY8_1X16
> > Using camera /base/pl-bus/i2c at 80040000/sensor at 10 as cam0
> > [43:26:44.724404510] [5672] ERROR Camera camera.cpp:690 Camera in
> Acquired state trying acquire() requiring state Available
> > Failed to acquire camera 1
> > Failed to create camera session
>
> The camera "--camera /base/pl-bus/i2c at 80040000/sensor at 10" and "-c1"
> arguments designate
> the same camera. As the error says "Camera in Acquired state" because it
> has
> been already acquired: "Using camera /base/pl-bus/i2c at 80040000/sensor at 10
> as cam0".
> What exactly are you trying to achieve here?
>
>
> Regards,
> Barnabás Pőcze
>
>
> >
> >
> > I have not modified the source code for the cam application.
> > I built the libcamera repo as follows:
> >
> > # meson setup build --buildtype=release -Dpipelines=xisp -Dv4l2=true
> -Dgstreamer=enabled -Dtest=false -Dlc-compliance=disabled -Dcam=enabled
> -Dqcam=disabled -Ddocumentation=disabled -Dpycamera=enabled
> > # ninja -C build install
> >
> > Then launch the application as follows:
> >
> > # meson devenv -C build
> > # cam ...
> > # LIBCAMERA_LOG_LEVELS=*:0 cam ...
> >
> > Any help and/or collaboration would be greatly appreciated :)
> >
> > Cheers !
> >
> > Mario Bergeron (AlbertaBeef)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.libcamera.org/pipermail/libcamera-devel/attachments/20250114/2cc87ae0/attachment.htm>