[libcamera-devel] qcam display scale bug

[hermann at stamm-wilbrandt.de]

Hi Naushir,

     I seem to get 800x600 as expected.  "dtoverlay=vc4-fkms-v3d" is
     enabled in my build, not sure how (if) that would make any 
difference.

I commented in "dtoverlay=vc4-fkms-v3d" and rebooted, no change.

So I did "sudo apt-get update", then "sudo apt-get upgrade" and finally
"sudo rpi-update". Then I removed libcamer directory, did clone and
build it newly. Still 800x472 in HDMI monitor snapshot:
https://stamm-wilbrandt.de/en/libcamera/snapshot.png

Just for completeness, this is /boot/config.txt I use:
https://stamm-wilbrandt.de/en/libcamera/config.txt

As I said Raspberry qcam application shows 800x600 over ssh with
X11 forwarding, so qcam seems to be fine.
And you see 800x600 in your qcam test as well.
What can be responsible for the scaling of 800x600 to 800x472?

Thanks,
Hermann.

On 2020-07-07 12:37, Naushir Patuck wrote:
> Hi Hermann,
> 
> On Tue, 7 Jul 2020 at 11:09, <hermann at stamm-wilbrandt.de> wrote:
>> 
>> Hi Naushir,
>> 
>> I could do the experiment you asked for.
>> 
>> But I seem accidentially having uncovered that it is a Raspberry X11
>> only issue.
>> I did run
>> $ LIBCAMERA_LOG_LEVELS=RPiFocus:DEBUG build/src/qcam/qcam
>> in an ssh session with X11 forwarding
>> $ ssh -X pi at 192.168.178.44
>> 
>> That qcam window that I saw looked better, and I took a screenshot:
>> https://stamm-wilbrandt.de/en/libcamera/qcam.ssh_X.screenshot.png
>> 
>> The top left coordinate is 10,84, bottom right is 809,684.
>> So when qcam app is displayed on my RHEL 7.8, I get 800x600 as
>> requested.
>> Only when started on Raspberry, I only get 800x472.
>> I get same 800x472 frames on X11 HDMI and X11 DPMI display -- its same
>> X11.
>> So it seems to be a Raspberry X11 only qcam display scale issue.
>> 
>> Can you please start qcam like I did on Raspberry, take a screenshot 
>> and
>> measure top left and bottom right coordinates of displayed frame?
>> And see whether you get 800x600 or 800x472 or something else.
>> "dtoverlay=vc4-fkms-v3d" is commented out in my /boot/config.txt
> 
> I seem to get 800x600 as expected.  "dtoverlay=vc4-fkms-v3d" is
> enabled in my build, not sure how (if) that would make any difference.
> 
> Thanks,
> Naush
> 
>> 
>> Thanks,
>> 
>> Hermann.
>> 
>> On 2020-07-07 10:28, Naushir Patuck wrote:
>> > Hi Hermann,
>> >
>> > On Tue, 7 Jul 2020 at 08:02, <hermann at stamm-wilbrandt.de> wrote:
>> >>
>> >> Hi Laurent,
>> >>
>> >> > I assume you're using the new HQ camera, right ?
>> >> >
>> >> yes, but from looking into the code that should not make a difference.
>> >>
>> >> > I'd like to clarify this first, to make sure I understand the problem
>> >> > correctly. Do you mean that qcam display a captured frame of 800x472
>> >> > pixels in a 800x600 screen region, scaling it in a way that doesn't
>> >> > preserve square pixels ? Do you just start the qcam application without
>> >> > passing it any parameter or resizing the window, or do you perform any
>> >> > specific operation ?
>> >> >
>> >> Yes, I am calling qcam without parameters.
>> >> qcam application requests 800x600 frames (see "size"):
>> >>
>> >> libcamera/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
>> >>                  case StreamRole::Viewfinder:
>> >>                          fmts =
>> >> data->isp_[Isp::Output0].dev()->formats();
>> >>                          pixelFormat = formats::ARGB8888;
>> >>                          size = { 800, 600 };
>> >>                          bufferCount = 4;
>> >>                          outCount++;
>> >>                          break;
>> >>
>> >> libcamera determines "2028x1520 pbCC" as best fit per log output.
>> >> The 2028x1520 frames captured get scaled into the display frame.
>> >> If the display frame would be 800x600 as requested, all would be fine.
>> >> But display frame being 800x472 compresses captured frame vertically
>> >> by factor 600/472=1.27.
>> >
>> > The 2028x1520 frame actually gets cropped in the ISP to correct for
>> > the requested aspect ratio before getting resized to the 800x600
>> > resolution.   So, the flow is actually as follows:
>> >
>> > 2028x1520 (sensor) -> 2026x1520 (aspect ratio crop) -> 800x600 (isp
>> > resize).
>> >
>> > The reason for the crop in the middle is that the 800/600 aspect ratio
>> > is exactly 4/3, but 2028/1520 is not.  This is why there will be a
>> > small amount of mismatch.  Having said that, you seem to be seeing a
>> > much bigger variation that does not correspond to the calculations
>> > that take place.
>> >
>> >
>> >>
>> >> I verified that by picture of micrometer:
>> >> https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=278268&p=1690839#p1690839
>> >> The longest divisions are of length 300µm vertically, but show only
>> >> 221px in display. Horizontally I measured 281px for 300µm.
>> >>
>> >
>> > Could you try getting qcam to output 2028x1520 so that no crop/resize
>> > takes place just to confirm your measurements are correct?
>> >
>> > Thanks,
>> > Naush
>> >
>> >> I just see that the 800x600 is specified in raspberrypi.cpp.
>> >> Can you please start qcam on some non-Raspberry system passing
>> >> width=800
>> >> and height=600, and then take screenshot and use image processing (I
>> >> use
>> >> gimp) to measure size of displayed frame?
>> >>
>> >>
>> >> On 2020-07-07 01:30, Laurent Pinchart wrote:
>> >> > Hi Hermann,
>> >> >
>> >> > (CC'ing Naush)
>> >> >
>> >> > On Sun, Jul 05, 2020 at 09:44:15PM +0200, hermann at stamm-wilbrandt.de
>> >> > wrote:
>> >> >> I started working with libcamera on Raspberry Pi.
>> >> >
>> >> > Thank you for your interest in the project :-)
>> >> >
>> >> >> I cannot tell whether this scale bug exists for raspberrypi only, or
>> >> >> in
>> >> >> general.
>> >> >>
>> >> >> qcam app requests a 800x600 window, but displayed window is 800x472
>> >> >> only.
>> >> >> This is not acceptable when using qcam as microscope app:
>> >> >> https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=278268&p=1690839#p1690839
>> >> >
>> >> > I assume you're using the new HQ camera, right ?
>> >> >
>> >> >> The best mode selected by libcamera is 2028x1520 which is 4:3.
>> >> >> So 800x600 (also 4:3) is needed for display.
>> >> >>
>> >> >> It would be helpful to know whether qcam display is also 800x472
>> >> >> instead
>> >> >> of 800x600 when not running on Rasberry Pi.
>> >> >
>> >> > I'd like to clarify this first, to make sure I understand the problem
>> >> > correctly. Do you mean that qcam display a captured frame of 800x472
>> >> > pixels in a 800x600 screen region, scaling it in a way that doesn't
>> >> > preserve square pixels ? Do you just start the qcam application without
>> >> > passing it any parameter or resizing the window, or do you perform any
>> >> > specific operation ?