Restrict PixelBuffer dimensions to safe values

Restrict PixelBuffer dimensions to safe values

We do a lot of calculations based on pixel coordinates and we need
to make sure they do not overflow. Restrict the maximum dimensions
we support rather than try to switch over all calculations to use
64 bit integers.

This prevents attackers from from injecting code by specifying a
huge framebuffer size and relying on the values overflowing to
access invalid areas of the heap.

This primarily affects the client which gets both the screen
dimensions and the pixel contents from the remote side. But the
server might also be affected as a client can adjust the screen
dimensions, as can applications inside the session.

Issue found by Pavel Cheremushkin from Kaspersky Lab.