Browse Source

Handle pixel formats with odd shift values

Our fast paths assume that each channel fits in to a separate byte.
That means the shift needs to be a multiple of 8. Start actually
checking this so that a client cannot trip us up and possibly cause
incorrect code exection.

Issue found by Pavel Cheremushkin from Kaspersky Lab.

(cherry picked from commit 05e2849087)
tags/v1.10.1
Pierre Ossman 4 years ago
parent
commit
6c47340e09
2 changed files with 12 additions and 0 deletions
  1. 6
    0
      common/rfb/PixelFormat.cxx
  2. 6
    0
      tests/unit/pixelformat.cxx

+ 6
- 0
common/rfb/PixelFormat.cxx View File

@@ -206,6 +206,12 @@ bool PixelFormat::is888(void) const
return false;
if (blueMax != 255)
return false;
if ((redShift & 0x7) != 0)
return false;
if ((greenShift & 0x7) != 0)
return false;
if ((blueShift & 0x7) != 0)
return false;

return true;
}

+ 6
- 0
tests/unit/pixelformat.cxx View File

@@ -170,6 +170,12 @@ void is888Tests()

do888Test(false, 8, 8, false, false, 0, 0, 0, 0, 0, 0);

/* Odd shifts */

do888Test(false, 32, 24, false, true, 255, 255, 255, 0, 8, 18);
do888Test(false, 32, 24, false, true, 255, 255, 255, 0, 11, 24);
do888Test(false, 32, 24, false, true, 255, 255, 255, 4, 16, 24);

printf("\n");
}


Loading…
Cancel
Save