[libcamera-devel] How to detect camera after power on the raspberryPi.

Tue Oct 20 13:48:42 CEST 2020

Dear Sirs

Maybe I understand less than 50% of your conversation below, but if you need MIPI 4 lane camera board
which can be connected to Raspi CM3+, I can provide you the IMX219 4lane camera board.

And if you are interested in controlling the focus, I can also send you the AF type module of IMX219.
The other option, IMX378 which is MX477 compatible sensor, with focus driver is also available.
https://soho-enterprise.com/2020/08/31/raspi-libcameraqcam-for-imx378/

I already tested them on Raspi 3 and 4 with qcam and they works well.
I tested the 4 lane mode of IMX219 module with Tinker Edge R, this means you can get full pixel 30fps image.
https://soho-enterprise.com/2020/02/27/post-681/
I captured high speed RAW image in 4 lane mode as below article.
https://soho-enterprise.com/2020/04/10/high-frame-rate-high-speed-camera-on-tinker-edge-r-testing-mipi-4-lane-feature/

And if you could help us to operate our SE-CXB01 Tx/Rx, the FPD LINK III camera extension board, 
I also can provide the one.
(Let me try the method using "dtoverlay" first.)
https://soho-enterprise.com/2020/10/19/dual-camera-system-under-development-se-cxb01-tx-rx/

I hope my offer will be your help.

Best Regards,

NOMURA

-----Original Message-----
From: Kieran Bingham <kieran.bingham at ideasonboard.com> 
Sent: Tuesday, October 20, 2020 7:50 PM
To: Dave Stevenson <dave.stevenson at raspberrypi.com>
Cc: Laurent Pinchart <laurent.pinchart at ideasonboard.com>; libcamera-devel at lists.libcamera.org; tetsuya.nomura at soho-enterprise.com; Niklas Söderlund <niklas.soderlund at ragnatech.se>
Subject: Re: [libcamera-devel] How to detect camera after power on the raspberryPi.

Hi Dave,

On 20/10/2020 11:32, Dave Stevenson wrote:
> Hi Kieran
> 
> On Tue, 20 Oct 2020 at 10:36, Kieran Bingham 
> <kieran.bingham at ideasonboard.com> wrote:
>>
>> Hi Dave,
>>
>> On 19/10/2020 16:53, Dave Stevenson wrote:
>>> Hi Laurent
>>>
>>> On Mon, 19 Oct 2020 at 13:50, Laurent Pinchart 
>>> <laurent.pinchart at ideasonboard.com> wrote:
>>>>
>>>> Hi Dave,
>>>>
>>>> On Mon, Oct 19, 2020 at 11:10:55AM +0100, Dave Stevenson wrote:
>>>>> On Mon, 19 Oct 2020 at 10:38, <tetsuya.nomura at soho-enterprise.com> wrote:
>>>>>>
>>>>>> Dear Sirs.
>>>>>>
>>>>>> May I have your favor about initialization of the RaspberryPi and the camera?
>>>>>>
>>>>>> I'm now evaluating the camera extension board of FPD LINK III technology to extend the camera cable for 2~3m from the RaspberryPi.
>>>>>> https://soho-enterprise.com/2020/10/11/post-866/
>>>>>>
>>>>>> This board can pass the I2C communication to IMX219 after 
>>>>>> configuring the transceiver chip, However, after configuration 
>>>>>> and run the qcam, it seems the qcam program does not detect the IMX219 And I cannot start the camera even I designate the image sensor name.
>>>>>
>>>>> The Pi kernel driver for imx219 probes the sensor driver as the 
>>>>> module loads, which is only a few seconds into boot if you've 
>>>>> added "dtoverlay=imx219" into config.txt. No or incorrect response 
>>>>> from the sensor and it won't create the /dev/video0 device nodes. 
>>>>> If your transceivers need some form of configuration then I 
>>>>> suspect this has not happened by this point.
>>>>>
>>>>> The correct approach is probably to have a driver for your 
>>>>> transceiver in the kernel driver chain, ie imx219 -> transceiver 
>>>>> -> Unicam, however that opens a big can of worms over 
>>>>> configuration as it forces the use of the Media Controller API to 
>>>>> configure each link independently.
>>>>
>>>> A very open question: assuming we can retain control through the 
>>>> video node as implemented today for the existing use cases, and 
>>>> implement MC-based pipeline control in parallel in the driver 
>>>> (possibly selecting one of the options through a module parameter) 
>>>> for use cases that would be tricky to support otherwise, would you be OK with that ?
>>>
>>> I'm totally OK with it!
>>> I've had a couple of other threads in the last few weeks that kicked 
>>> me to look into MC and what is required.
>>>
>>> The first was trying to support Analog Devices ADV7482 which Kieran 
>>> will know and "love". That showed up a couple of oddities around
>>
>> Ahem, cough cough. Indeed :-)
>>
>> Is there a board available that can connect this to the RPi?
>> If so I'd be interested in buying one.
> 
> Not that I'm aware of.
> The forum contributor's company had brought up a board with ADV7282-M 
> (analogue video to CSI2) on. Based on that they ploughed ahead in 
> building a prototype Compute Module carrier board for ADV7482 (and DPI 
> to VGA as done on the VGA666 board). That's the point that he posted 
> on the forums as things didn't connect up nice and easily.
> He has been kind enough to ship one to me from Chile(!), but I haven't 
> had time to power it up as yet. Looking at the PCB it appears to only 
> route 2 CSI-2 data lanes which is a bit of a shame - oh well.

Oh indeed, would have been nice to route all 4 lanes for a compute-module. Perhaps it's not too late for their next board revision?

Well if you hit any blocker, or need anything let us know.

> 
>>> linking to async_notifier and incorrect behaviour with our use of 
>>> pad indexes. Those two bits are easy to rectify, although having 
>>> made the async_notifier stuff match other platforms it no longer 
>>> probed the simple modules correctly :-( There must be a solution 
>>> there somewhere, it's only finding the correct runes.
>>
>> The ADV748x requires endpoint matching, and it complicates things indeed.
>>
>> For a long time, I think the only receiver that supported this driver 
>> was the Renesas RCar-CSI2/VIN.
>>
>> There is a series here:
>>
>> https://lore.kernel.org/linux-media/20200701062140.12953-1-laurent.pi
>> nchart+renesas at ideasonboard.com/
>>
>> Which should finally ease things and make endpoint matching 
>> compatible with all receivers, which I believe has landed in v5.9.
> 
> I thought I'd been running on our 5.9 branch, but looking at the 
> branch I'd pushed for their reference[1] it looks to be 5.4.
> I'll try rebasing and see what I get.
> 
> [1] https://github.com/6by9/linux/tree/adv748x
> 
>>> The second use case was Toshiba's TC358746/TC358748 parallel to CSI 
>>> bridges, where similar to this case it's chaining sensor -> 
>>> TC35874[6|8] -> Unicam.
>>>
>>> With regard switching behaviours, it's relatively simple to look at 
>>> the immediately upstream node and see if it has any sink pads. If 
>>> yes we go MC, otherwise video-device. Doing so dynamically at 
>>> runtime is nicer than a module parameter.
>>
>> +Niklas,
>>
>> I sort of agree here, as it should be easier for end users. Niklas 
>> may have different opinions as he has felt pain of supporting both MC 
>> and non-MC interfaces in the RCar-VIN.
> 
> Felt as in past tense and you've dropped non-MC?
> I'll have a look at the RCar-VIN driver to see if I can work through 
> the differences.
> ...
> It appears MC is enabled dependent on the platform - that gives me 
> something to follow.
> I'm a little confused though as it still seems to call 
> v4l2_subdev_call(sd, pad, set_fmt, pad_cfg, &format); from 
> rvin_s_fmt_vid_cap to set the format on the upstream subdevice sink 
> pad. Unless I'm missing something then I thought it wasn't allowed in 
> MC to propagate formats across links.
> 
>>> That's the easy bit. The harder bit is working out which bits of 
>>> functionality that then requires dropping when in MC mode. At the 
>>> moment I'm not totally clear over the correct approach for 
>>> configuring the actual Unicam hardware - S_FMT on /dev/videoN, or MC 
>>> API setting the sink pad format of Unicam. The mismash of 
>>> MEDIA_BUS_FMT_xxx vs V4L2_PIX_FMT_xxx formats surfaces again, how do 
>>> you avoid format mismatches (and unpacking to 16bit), and which do 
>>> you use to configure CSI data type filtering.
>>> I'm assuming all the subdev API calls for EDIDs, timings (DV and 
>>> standards), parm, and enumerations disappear with MC too, as you 
>>> open the subdev node to configure that lot.
>>
>> Indeed, those can then be operated on the subdev node.
> 
> OK, so lots of disable_ioctl calls but those are easy enough and 
> already half in place for when the subdev doesn't support them.
> 
> So just the step of understanding what configures the hardware 
> format/width/height/stride/buffer size.

Userspace ? I think that's where all the format propagation comes in.

I'm very much aware that the TC358746/TC358748 is not a camera sensor, but libcamera has so much of this 'done' that I do wonder how non-camera
(media-controller) type devices should be used in the future. Perhaps there is some core parts of libcamera that should be factored out to facilitate non-camera type devices that use the same pipelines, or perhaps libcamera would grow to support them. It feels like a bit of a shame that they are out of scope for us at the moment.

Same with the ADV748x HDMI input ... that's not a camera, but libcamera would make it much easier to configure/capture from - though technically, the 8 analogue inputs could be ... ;-)

--
Kieran

>   Dave
> 
>>>
>>> Thanks
>>>   Dave
>>>
>>>>> One workaround would be to remove "dtoverlay=imx219" from 
>>>>> config.txt and use dynamic device tree overlay loading to load the 
>>>>> relevant modules later in boot. "sudo dtoverlay imx219" should do that for you.
>>>>> The niggle is that when done from config.txt the firmware fixes up 
>>>>> the camera shutdown GPIO to match the platform (it moves GPIOs 
>>>>> between different variants of the Pi). If loading dynamically then 
>>>>> this can't happen, and you'll need to fix up the regulator 
>>>>> shutdown line manually [1] (assuming that it is actually 
>>>>> controllable at the other end of the FPD Link)
>>>>>
>>>>> [1] 
>>>>> (https://github.com/raspberrypi/linux/blob/rpi-5.4.y/arch/arm/boot
>>>>> /dts/overlays/imx219-overlay.dts#L77)
>>>>>
>>>>>> I can start the IMX219, get the RAW image by our own program and see the image.
>>>>>> Then if qcam can send the necessary command to IMX219 when start 
>>>>>> running the qcam, I think I can see the image through qcam program.
>>>>>>
>>>>>> It would be great if you could give us an advice to solve it.
>>>>>> I appreciate your kind help in advance.
>>>>>>
>>>>>> Best Regards,
>>>>>>
>>>>>> NOMURA
>>>>
>>>> --
>>>> Regards,
>>>>
>>>> Laurent Pinchart
>>> _______________________________________________
>>> libcamera-devel mailing list
>>> libcamera-devel at lists.libcamera.org
>>> https://lists.libcamera.org/listinfo/libcamera-devel
>>>
>>
>> --
>> Regards
>> --
>> Kieran

--
Regards
--
Kieran