[libcamera-devel] qcam display scale bug

hermann at stamm-wilbrandt.de hermann at stamm-wilbrandt.de
Tue Jul 7 20:43:31 CEST 2020 

Hi Naushir,

I identified the bug.
I did flash Raspberry Pi OS Desktop on new SD card.
Then I did all the updates, configured Camera, I2C and SSH to be used.
After raspistill did show a frame, I did "sudo rpi-update" and added
dtoverlay=imx477
core_freq_min=250
to /boot/config.txt and rebooted.
Then I did install and build libcamera per:
https://github.com/raspberrypi/documentation/blob/master/linux/software/libcamera/README.md

Starting build/src/qcam/qcam I got nice 800x600 frame displayed in qcam:
https://stamm-wilbrandt.de/en/libcamera/2020-07-07-200815_1920x1080_scrot.png

But as the screenshot shows, X11 runs in 1920x1080 mode.
My HDMI monitor has 1360x768 native resolution.
So I added ~/.xsessionrc to make X11 show 1360x768:
pi at raspberrypi:~ $ cat .xsessionrc
xrandr --output HDMI-1 --mode 1360x768
pi at raspberrypi:~ $

After restarting X11 I get 1360x768 display, but now qcam frame is only 
800x472:
https://stamm-wilbrandt.de/en/libcamera/2020-07-07-202453_1360x768_scrot.png

So the bug is that qcam cannot deal with the (correct) settings of 
xrandr.
I think bringing X11 to HDMI monitor native resolution is valid.
So something needs to be changed in qcam, other applications like xclock 
do
not have this problem.

Thanks,
Hermann.

On 2020-07-07 12:37, Naushir Patuck wrote:
> Hi Hermann,
> 
> On Tue, 7 Jul 2020 at 11:09, <hermann at stamm-wilbrandt.de> wrote:
>> 
>> Hi Naushir,
>> 
>> I could do the experiment you asked for.
>> 
>> But I seem accidentially having uncovered that it is a Raspberry X11
>> only issue.
>> I did run
>> $ LIBCAMERA_LOG_LEVELS=RPiFocus:DEBUG build/src/qcam/qcam
>> in an ssh session with X11 forwarding
>> $ ssh -X pi at 192.168.178.44
>> 
>> That qcam window that I saw looked better, and I took a screenshot:
>> https://stamm-wilbrandt.de/en/libcamera/qcam.ssh_X.screenshot.png
>> 
>> The top left coordinate is 10,84, bottom right is 809,684.
>> So when qcam app is displayed on my RHEL 7.8, I get 800x600 as
>> requested.
>> Only when started on Raspberry, I only get 800x472.
>> I get same 800x472 frames on X11 HDMI and X11 DPMI display -- its same
>> X11.
>> So it seems to be a Raspberry X11 only qcam display scale issue.
>> 
>> Can you please start qcam like I did on Raspberry, take a screenshot 
>> and
>> measure top left and bottom right coordinates of displayed frame?
>> And see whether you get 800x600 or 800x472 or something else.
>> "dtoverlay=vc4-fkms-v3d" is commented out in my /boot/config.txt
> 
> I seem to get 800x600 as expected.  "dtoverlay=vc4-fkms-v3d" is
> enabled in my build, not sure how (if) that would make any difference.
> 
> Thanks,
> Naush
> 
>> 
>> Thanks,
>> 
>> Hermann.
>> 
>> On 2020-07-07 10:28, Naushir Patuck wrote:
>> > Hi Hermann,
>> >
>> > On Tue, 7 Jul 2020 at 08:02, <hermann at stamm-wilbrandt.de> wrote:
>> >>
>> >> Hi Laurent,
>> >>
>> >> > I assume you're using the new HQ camera, right ?
>> >> >
>> >> yes, but from looking into the code that should not make a difference.
>> >>
>> >> > I'd like to clarify this first, to make sure I understand the problem
>> >> > correctly. Do you mean that qcam display a captured frame of 800x472
>> >> > pixels in a 800x600 screen region, scaling it in a way that doesn't
>> >> > preserve square pixels ? Do you just start the qcam application without
>> >> > passing it any parameter or resizing the window, or do you perform any
>> >> > specific operation ?
>> >> >
>> >> Yes, I am calling qcam without parameters.
>> >> qcam application requests 800x600 frames (see "size"):
>> >>
>> >> libcamera/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
>> >>                  case StreamRole::Viewfinder:
>> >>                          fmts =
>> >> data->isp_[Isp::Output0].dev()->formats();
>> >>                          pixelFormat = formats::ARGB8888;
>> >>                          size = { 800, 600 };
>> >>                          bufferCount = 4;
>> >>                          outCount++;
>> >>                          break;
>> >>
>> >> libcamera determines "2028x1520 pbCC" as best fit per log output.
>> >> The 2028x1520 frames captured get scaled into the display frame.
>> >> If the display frame would be 800x600 as requested, all would be fine.
>> >> But display frame being 800x472 compresses captured frame vertically
>> >> by factor 600/472=1.27.
>> >
>> > The 2028x1520 frame actually gets cropped in the ISP to correct for
>> > the requested aspect ratio before getting resized to the 800x600
>> > resolution.   So, the flow is actually as follows:
>> >
>> > 2028x1520 (sensor) -> 2026x1520 (aspect ratio crop) -> 800x600 (isp
>> > resize).
>> >
>> > The reason for the crop in the middle is that the 800/600 aspect ratio
>> > is exactly 4/3, but 2028/1520 is not.  This is why there will be a
>> > small amount of mismatch.  Having said that, you seem to be seeing a
>> > much bigger variation that does not correspond to the calculations
>> > that take place.
>> >
>> >
>> >>
>> >> I verified that by picture of micrometer:
>> >> https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=278268&p=1690839#p1690839
>> >> The longest divisions are of length 300µm vertically, but show only
>> >> 221px in display. Horizontally I measured 281px for 300µm.
>> >>
>> >
>> > Could you try getting qcam to output 2028x1520 so that no crop/resize
>> > takes place just to confirm your measurements are correct?
>> >
>> > Thanks,
>> > Naush
>> >
>> >> I just see that the 800x600 is specified in raspberrypi.cpp.
>> >> Can you please start qcam on some non-Raspberry system passing
>> >> width=800
>> >> and height=600, and then take screenshot and use image processing (I
>> >> use
>> >> gimp) to measure size of displayed frame?
>> >>
>> >>
>> >> On 2020-07-07 01:30, Laurent Pinchart wrote:
>> >> > Hi Hermann,
>> >> >
>> >> > (CC'ing Naush)
>> >> >
>> >> > On Sun, Jul 05, 2020 at 09:44:15PM +0200, hermann at stamm-wilbrandt.de
>> >> > wrote:
>> >> >> I started working with libcamera on Raspberry Pi.
>> >> >
>> >> > Thank you for your interest in the project :-)
>> >> >
>> >> >> I cannot tell whether this scale bug exists for raspberrypi only, or
>> >> >> in
>> >> >> general.
>> >> >>
>> >> >> qcam app requests a 800x600 window, but displayed window is 800x472
>> >> >> only.
>> >> >> This is not acceptable when using qcam as microscope app:
>> >> >> https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=278268&p=1690839#p1690839
>> >> >
>> >> > I assume you're using the new HQ camera, right ?
>> >> >
>> >> >> The best mode selected by libcamera is 2028x1520 which is 4:3.
>> >> >> So 800x600 (also 4:3) is needed for display.
>> >> >>
>> >> >> It would be helpful to know whether qcam display is also 800x472
>> >> >> instead
>> >> >> of 800x600 when not running on Rasberry Pi.
>> >> >
>> >> > I'd like to clarify this first, to make sure I understand the problem
>> >> > correctly. Do you mean that qcam display a captured frame of 800x472
>> >> > pixels in a 800x600 screen region, scaling it in a way that doesn't
>> >> > preserve square pixels ? Do you just start the qcam application without
>> >> > passing it any parameter or resizing the window, or do you perform any
>> >> > specific operation ?

More information about the libcamera-devel mailing list