Enhanced ComparingUpdateTracker to crop changed blocks
64x64 changed block can be large for fine changes such as cursor
movement and typing in terminal windows, or an update to a clock.
If the block can be efficiently cropped, this will reduce latency
and bandwidth. Every pixel cropped is a pixel less to analyze, encode,
transmit, and decode.
The previous code already detected the top of the change in order
to determine if the block had changed. However, it did not use
this information to reduce the size of the change rectangle, nor
did it calculate any of the other edges.
The new code introduces detection of the other edges, and uses
the information to build a reduced area change rectangle. This
has the additional effect of reducing the number of discrete pixel
values in the change block which may allow a more efficient
encoding algorithm to be selected.
As this section of code is performance sensitive, the method
of detecting the edges has been optimized to quickly fall back
to pessimistic values as soon as a single comparison fails on
each edge. In the case that full 64x64 block are changing,
there will be three extra comparisons per block.
In cases where the change rectangle can be reduced from 64x64,
the reduced size of the change rectangle represents reduced
effort to encode, transfer, and decode the contained pixels.
In the case of images with high frequency changes, which
specifically includes text, the lossy JPEG encoding can be
highly distorted, especially with JPEG level 6 or less. The
quick flash from a distorted JPEG to a lossless JPEG can
appear as a flickering to some people. This effect was more
obvious when the surrounding area is not expected to change,
but is being distorted anyways due to being part of the 64x64
blocking algorithm.
In the case of a user typing in a terminal window, this change
may commonly reduce the number of pixels updated with every
character typed from 4096 pixels (64x64) to 640 pixels (32x20)
or less.
4 jaren geleden Enhanced ComparingUpdateTracker to crop changed blocks
64x64 changed block can be large for fine changes such as cursor
movement and typing in terminal windows, or an update to a clock.
If the block can be efficiently cropped, this will reduce latency
and bandwidth. Every pixel cropped is a pixel less to analyze, encode,
transmit, and decode.
The previous code already detected the top of the change in order
to determine if the block had changed. However, it did not use
this information to reduce the size of the change rectangle, nor
did it calculate any of the other edges.
The new code introduces detection of the other edges, and uses
the information to build a reduced area change rectangle. This
has the additional effect of reducing the number of discrete pixel
values in the change block which may allow a more efficient
encoding algorithm to be selected.
As this section of code is performance sensitive, the method
of detecting the edges has been optimized to quickly fall back
to pessimistic values as soon as a single comparison fails on
each edge. In the case that full 64x64 block are changing,
there will be three extra comparisons per block.
In cases where the change rectangle can be reduced from 64x64,
the reduced size of the change rectangle represents reduced
effort to encode, transfer, and decode the contained pixels.
In the case of images with high frequency changes, which
specifically includes text, the lossy JPEG encoding can be
highly distorted, especially with JPEG level 6 or less. The
quick flash from a distorted JPEG to a lossless JPEG can
appear as a flickering to some people. This effect was more
obvious when the surrounding area is not expected to change,
but is being distorted anyways due to being part of the 64x64
blocking algorithm.
In the case of a user typing in a terminal window, this change
may commonly reduce the number of pixels updated with every
character typed from 4096 pixels (64x64) to 640 pixels (32x20)
or less.
4 jaren geleden Enhanced ComparingUpdateTracker to crop changed blocks
64x64 changed block can be large for fine changes such as cursor
movement and typing in terminal windows, or an update to a clock.
If the block can be efficiently cropped, this will reduce latency
and bandwidth. Every pixel cropped is a pixel less to analyze, encode,
transmit, and decode.
The previous code already detected the top of the change in order
to determine if the block had changed. However, it did not use
this information to reduce the size of the change rectangle, nor
did it calculate any of the other edges.
The new code introduces detection of the other edges, and uses
the information to build a reduced area change rectangle. This
has the additional effect of reducing the number of discrete pixel
values in the change block which may allow a more efficient
encoding algorithm to be selected.
As this section of code is performance sensitive, the method
of detecting the edges has been optimized to quickly fall back
to pessimistic values as soon as a single comparison fails on
each edge. In the case that full 64x64 block are changing,
there will be three extra comparisons per block.
In cases where the change rectangle can be reduced from 64x64,
the reduced size of the change rectangle represents reduced
effort to encode, transfer, and decode the contained pixels.
In the case of images with high frequency changes, which
specifically includes text, the lossy JPEG encoding can be
highly distorted, especially with JPEG level 6 or less. The
quick flash from a distorted JPEG to a lossless JPEG can
appear as a flickering to some people. This effect was more
obvious when the surrounding area is not expected to change,
but is being distorted anyways due to being part of the 64x64
blocking algorithm.
In the case of a user typing in a terminal window, this change
may commonly reduce the number of pixels updated with every
character typed from 4096 pixels (64x64) to 640 pixels (32x20)
or less.
4 jaren geleden Enhanced ComparingUpdateTracker to crop changed blocks
64x64 changed block can be large for fine changes such as cursor
movement and typing in terminal windows, or an update to a clock.
If the block can be efficiently cropped, this will reduce latency
and bandwidth. Every pixel cropped is a pixel less to analyze, encode,
transmit, and decode.
The previous code already detected the top of the change in order
to determine if the block had changed. However, it did not use
this information to reduce the size of the change rectangle, nor
did it calculate any of the other edges.
The new code introduces detection of the other edges, and uses
the information to build a reduced area change rectangle. This
has the additional effect of reducing the number of discrete pixel
values in the change block which may allow a more efficient
encoding algorithm to be selected.
As this section of code is performance sensitive, the method
of detecting the edges has been optimized to quickly fall back
to pessimistic values as soon as a single comparison fails on
each edge. In the case that full 64x64 block are changing,
there will be three extra comparisons per block.
In cases where the change rectangle can be reduced from 64x64,
the reduced size of the change rectangle represents reduced
effort to encode, transfer, and decode the contained pixels.
In the case of images with high frequency changes, which
specifically includes text, the lossy JPEG encoding can be
highly distorted, especially with JPEG level 6 or less. The
quick flash from a distorted JPEG to a lossless JPEG can
appear as a flickering to some people. This effect was more
obvious when the surrounding area is not expected to change,
but is being distorted anyways due to being part of the 64x64
blocking algorithm.
In the case of a user typing in a terminal window, this change
may commonly reduce the number of pixels updated with every
character typed from 4096 pixels (64x64) to 640 pixels (32x20)
or less.
4 jaren geleden Enhanced ComparingUpdateTracker to crop changed blocks
64x64 changed block can be large for fine changes such as cursor
movement and typing in terminal windows, or an update to a clock.
If the block can be efficiently cropped, this will reduce latency
and bandwidth. Every pixel cropped is a pixel less to analyze, encode,
transmit, and decode.
The previous code already detected the top of the change in order
to determine if the block had changed. However, it did not use
this information to reduce the size of the change rectangle, nor
did it calculate any of the other edges.
The new code introduces detection of the other edges, and uses
the information to build a reduced area change rectangle. This
has the additional effect of reducing the number of discrete pixel
values in the change block which may allow a more efficient
encoding algorithm to be selected.
As this section of code is performance sensitive, the method
of detecting the edges has been optimized to quickly fall back
to pessimistic values as soon as a single comparison fails on
each edge. In the case that full 64x64 block are changing,
there will be three extra comparisons per block.
In cases where the change rectangle can be reduced from 64x64,
the reduced size of the change rectangle represents reduced
effort to encode, transfer, and decode the contained pixels.
In the case of images with high frequency changes, which
specifically includes text, the lossy JPEG encoding can be
highly distorted, especially with JPEG level 6 or less. The
quick flash from a distorted JPEG to a lossless JPEG can
appear as a flickering to some people. This effect was more
obvious when the surrounding area is not expected to change,
but is being distorted anyways due to being part of the 64x64
blocking algorithm.
In the case of a user typing in a terminal window, this change
may commonly reduce the number of pixels updated with every
character typed from 4096 pixels (64x64) to 640 pixels (32x20)
or less.
4 jaren geleden Enhanced ComparingUpdateTracker to crop changed blocks
64x64 changed block can be large for fine changes such as cursor
movement and typing in terminal windows, or an update to a clock.
If the block can be efficiently cropped, this will reduce latency
and bandwidth. Every pixel cropped is a pixel less to analyze, encode,
transmit, and decode.
The previous code already detected the top of the change in order
to determine if the block had changed. However, it did not use
this information to reduce the size of the change rectangle, nor
did it calculate any of the other edges.
The new code introduces detection of the other edges, and uses
the information to build a reduced area change rectangle. This
has the additional effect of reducing the number of discrete pixel
values in the change block which may allow a more efficient
encoding algorithm to be selected.
As this section of code is performance sensitive, the method
of detecting the edges has been optimized to quickly fall back
to pessimistic values as soon as a single comparison fails on
each edge. In the case that full 64x64 block are changing,
there will be three extra comparisons per block.
In cases where the change rectangle can be reduced from 64x64,
the reduced size of the change rectangle represents reduced
effort to encode, transfer, and decode the contained pixels.
In the case of images with high frequency changes, which
specifically includes text, the lossy JPEG encoding can be
highly distorted, especially with JPEG level 6 or less. The
quick flash from a distorted JPEG to a lossless JPEG can
appear as a flickering to some people. This effect was more
obvious when the surrounding area is not expected to change,
but is being distorted anyways due to being part of the 64x64
blocking algorithm.
In the case of a user typing in a terminal window, this change
may commonly reduce the number of pixels updated with every
character typed from 4096 pixels (64x64) to 640 pixels (32x20)
or less.
4 jaren geleden Enhanced ComparingUpdateTracker to crop changed blocks
64x64 changed block can be large for fine changes such as cursor
movement and typing in terminal windows, or an update to a clock.
If the block can be efficiently cropped, this will reduce latency
and bandwidth. Every pixel cropped is a pixel less to analyze, encode,
transmit, and decode.
The previous code already detected the top of the change in order
to determine if the block had changed. However, it did not use
this information to reduce the size of the change rectangle, nor
did it calculate any of the other edges.
The new code introduces detection of the other edges, and uses
the information to build a reduced area change rectangle. This
has the additional effect of reducing the number of discrete pixel
values in the change block which may allow a more efficient
encoding algorithm to be selected.
As this section of code is performance sensitive, the method
of detecting the edges has been optimized to quickly fall back
to pessimistic values as soon as a single comparison fails on
each edge. In the case that full 64x64 block are changing,
there will be three extra comparisons per block.
In cases where the change rectangle can be reduced from 64x64,
the reduced size of the change rectangle represents reduced
effort to encode, transfer, and decode the contained pixels.
In the case of images with high frequency changes, which
specifically includes text, the lossy JPEG encoding can be
highly distorted, especially with JPEG level 6 or less. The
quick flash from a distorted JPEG to a lossless JPEG can
appear as a flickering to some people. This effect was more
obvious when the surrounding area is not expected to change,
but is being distorted anyways due to being part of the 64x64
blocking algorithm.
In the case of a user typing in a terminal window, this change
may commonly reduce the number of pixels updated with every
character typed from 4096 pixels (64x64) to 640 pixels (32x20)
or less.
4 jaren geleden Enhanced ComparingUpdateTracker to crop changed blocks
64x64 changed block can be large for fine changes such as cursor
movement and typing in terminal windows, or an update to a clock.
If the block can be efficiently cropped, this will reduce latency
and bandwidth. Every pixel cropped is a pixel less to analyze, encode,
transmit, and decode.
The previous code already detected the top of the change in order
to determine if the block had changed. However, it did not use
this information to reduce the size of the change rectangle, nor
did it calculate any of the other edges.
The new code introduces detection of the other edges, and uses
the information to build a reduced area change rectangle. This
has the additional effect of reducing the number of discrete pixel
values in the change block which may allow a more efficient
encoding algorithm to be selected.
As this section of code is performance sensitive, the method
of detecting the edges has been optimized to quickly fall back
to pessimistic values as soon as a single comparison fails on
each edge. In the case that full 64x64 block are changing,
there will be three extra comparisons per block.
In cases where the change rectangle can be reduced from 64x64,
the reduced size of the change rectangle represents reduced
effort to encode, transfer, and decode the contained pixels.
In the case of images with high frequency changes, which
specifically includes text, the lossy JPEG encoding can be
highly distorted, especially with JPEG level 6 or less. The
quick flash from a distorted JPEG to a lossless JPEG can
appear as a flickering to some people. This effect was more
obvious when the surrounding area is not expected to change,
but is being distorted anyways due to being part of the 64x64
blocking algorithm.
In the case of a user typing in a terminal window, this change
may commonly reduce the number of pixels updated with every
character typed from 4096 pixels (64x64) to 640 pixels (32x20)
or less.
4 jaren geleden Enhanced ComparingUpdateTracker to crop changed blocks
64x64 changed block can be large for fine changes such as cursor
movement and typing in terminal windows, or an update to a clock.
If the block can be efficiently cropped, this will reduce latency
and bandwidth. Every pixel cropped is a pixel less to analyze, encode,
transmit, and decode.
The previous code already detected the top of the change in order
to determine if the block had changed. However, it did not use
this information to reduce the size of the change rectangle, nor
did it calculate any of the other edges.
The new code introduces detection of the other edges, and uses
the information to build a reduced area change rectangle. This
has the additional effect of reducing the number of discrete pixel
values in the change block which may allow a more efficient
encoding algorithm to be selected.
As this section of code is performance sensitive, the method
of detecting the edges has been optimized to quickly fall back
to pessimistic values as soon as a single comparison fails on
each edge. In the case that full 64x64 block are changing,
there will be three extra comparisons per block.
In cases where the change rectangle can be reduced from 64x64,
the reduced size of the change rectangle represents reduced
effort to encode, transfer, and decode the contained pixels.
In the case of images with high frequency changes, which
specifically includes text, the lossy JPEG encoding can be
highly distorted, especially with JPEG level 6 or less. The
quick flash from a distorted JPEG to a lossless JPEG can
appear as a flickering to some people. This effect was more
obvious when the surrounding area is not expected to change,
but is being distorted anyways due to being part of the 64x64
blocking algorithm.
In the case of a user typing in a terminal window, this change
may commonly reduce the number of pixels updated with every
character typed from 4096 pixels (64x64) to 640 pixels (32x20)
or less.
4 jaren geleden |
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265 |
- /* Copyright (C) 2002-2005 RealVNC Ltd. All Rights Reserved.
- *
- * This is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or
- * (at your option) any later version.
- *
- * This software is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this software; if not, write to the Free Software
- * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307,
- * USA.
- */
-
- #ifdef HAVE_CONFIG_H
- #include <config.h>
- #endif
-
- #include <stdio.h>
- #include <string.h>
- #include <vector>
-
- #include <rfb/Exception.h>
- #include <rfb/LogWriter.h>
- #include <rfb/ComparingUpdateTracker.h>
-
- using namespace rfb;
-
- static LogWriter vlog("ComparingUpdateTracker");
-
- ComparingUpdateTracker::ComparingUpdateTracker(PixelBuffer* buffer)
- : fb(buffer), oldFb(fb->getPF(), 0, 0), firstCompare(true),
- enabled(true), totalPixels(0), missedPixels(0)
- {
- changed.assign_union(fb->getRect());
- }
-
- ComparingUpdateTracker::~ComparingUpdateTracker()
- {
- }
-
-
- #define BLOCK_SIZE 64
-
- bool ComparingUpdateTracker::compare()
- {
- std::vector<Rect> rects;
- std::vector<Rect>::iterator i;
-
- if (!enabled)
- return false;
-
- if (firstCompare) {
- // NB: We leave the change region untouched on this iteration,
- // since in effect the entire framebuffer has changed.
- oldFb.setSize(fb->width(), fb->height());
-
- for (int y=0; y<fb->height(); y+=BLOCK_SIZE) {
- Rect pos(0, y, fb->width(), __rfbmin(fb->height(), y+BLOCK_SIZE));
- int srcStride;
- const uint8_t* srcData = fb->getBuffer(pos, &srcStride);
- oldFb.imageRect(pos, srcData, srcStride);
- }
-
- firstCompare = false;
-
- return false;
- }
-
- copied.get_rects(&rects, copy_delta.x<=0, copy_delta.y<=0);
- for (i = rects.begin(); i != rects.end(); i++)
- oldFb.copyRect(*i, copy_delta);
-
- changed.get_rects(&rects);
-
- Region newChanged;
- for (i = rects.begin(); i != rects.end(); i++)
- compareRect(*i, &newChanged);
-
- changed.get_rects(&rects);
- for (i = rects.begin(); i != rects.end(); i++)
- totalPixels += i->area();
- newChanged.get_rects(&rects);
- for (i = rects.begin(); i != rects.end(); i++)
- missedPixels += i->area();
-
- if (changed.equals(newChanged))
- return false;
-
- changed = newChanged;
-
- return true;
- }
-
- void ComparingUpdateTracker::enable()
- {
- enabled = true;
- }
-
- void ComparingUpdateTracker::disable()
- {
- enabled = false;
-
- // Make sure we update the framebuffer next time we get enabled
- firstCompare = true;
- }
-
- void ComparingUpdateTracker::compareRect(const Rect& r, Region* newChanged)
- {
- if (!r.enclosed_by(fb->getRect())) {
- Rect safe;
- // Crop the rect and try again
- safe = r.intersect(fb->getRect());
- if (!safe.is_empty())
- compareRect(safe, newChanged);
- return;
- }
-
- int bytesPerPixel = fb->getPF().bpp/8;
- int oldStride;
- uint8_t* oldData = oldFb.getBufferRW(r, &oldStride);
- int oldStrideBytes = oldStride * bytesPerPixel;
-
- // Used to efficiently crop the left and right of the change rectangle
- int minCompareWidthInPixels = BLOCK_SIZE / 8;
- int minCompareWidthInBytes = minCompareWidthInPixels * bytesPerPixel;
-
- for (int blockTop = r.tl.y; blockTop < r.br.y; blockTop += BLOCK_SIZE)
- {
- // Get a strip of the source buffer
- Rect pos(r.tl.x, blockTop, r.br.x, __rfbmin(r.br.y, blockTop+BLOCK_SIZE));
- int fbStride;
- const uint8_t* newBlockPtr = fb->getBuffer(pos, &fbStride);
- int newStrideBytes = fbStride * bytesPerPixel;
-
- uint8_t* oldBlockPtr = oldData;
- int blockBottom = __rfbmin(blockTop+BLOCK_SIZE, r.br.y);
-
- for (int blockLeft = r.tl.x; blockLeft < r.br.x; blockLeft += BLOCK_SIZE)
- {
- const uint8_t* newPtr = newBlockPtr;
- uint8_t* oldPtr = oldBlockPtr;
-
- int blockRight = __rfbmin(blockLeft+BLOCK_SIZE, r.br.x);
- int blockWidthInBytes = (blockRight-blockLeft) * bytesPerPixel;
-
- // Scan the block top to bottom, to identify the first row of change
- for (int y = blockTop; y < blockBottom; y++)
- {
- if (memcmp(oldPtr, newPtr, blockWidthInBytes) != 0)
- {
- // Define the change rectangle using pessimistic values to start
- int changeHeight = blockBottom - y;
- int changeLeft = blockLeft;
- int changeRight = blockRight;
-
- // For every unchanged row at the bottom of the block, decrement change height
- {
- const uint8_t* newRowPtr = newPtr + ((changeHeight - 1) * newStrideBytes);
- const uint8_t* oldRowPtr = oldPtr + ((changeHeight - 1) * oldStrideBytes);
- while (changeHeight > 1 && memcmp(oldRowPtr, newRowPtr, blockWidthInBytes) == 0)
- {
- newRowPtr -= newStrideBytes;
- oldRowPtr -= oldStrideBytes;
-
- changeHeight--;
- }
- }
-
- // For every unchanged column at the left of the block, increment change left
- {
- const uint8_t* newColumnPtr = newPtr;
- const uint8_t* oldColumnPtr = oldPtr;
- while (changeLeft + minCompareWidthInPixels < changeRight)
- {
- const uint8_t* newRowPtr = newColumnPtr;
- const uint8_t* oldRowPtr = oldColumnPtr;
- for (int row = 0; row < changeHeight; row++)
- {
- if (memcmp(oldRowPtr, newRowPtr, minCompareWidthInBytes) != 0)
- goto endOfChangeLeft;
-
- newRowPtr += newStrideBytes;
- oldRowPtr += oldStrideBytes;
- }
-
- newColumnPtr += minCompareWidthInBytes;
- oldColumnPtr += minCompareWidthInBytes;
-
- changeLeft += minCompareWidthInPixels;
- }
- }
- endOfChangeLeft:
-
- // For every unchanged column at the right of the block, decrement change right
- {
- const uint8_t* newColumnPtr = newPtr + blockWidthInBytes;
- const uint8_t* oldColumnPtr = oldPtr + blockWidthInBytes;
- while (changeLeft + minCompareWidthInPixels < changeRight)
- {
- newColumnPtr -= minCompareWidthInBytes;
- oldColumnPtr -= minCompareWidthInBytes;
-
- const uint8_t* newRowPtr = newColumnPtr;
- const uint8_t* oldRowPtr = oldColumnPtr;
- for (int row = 0; row < changeHeight; row++)
- {
- if (memcmp(oldRowPtr, newRowPtr, minCompareWidthInBytes) != 0)
- goto endOfChangeRight;
-
- newRowPtr += newStrideBytes;
- oldRowPtr += oldStrideBytes;
- }
-
- changeRight -= minCompareWidthInPixels;
- }
- }
- endOfChangeRight:
-
- // Block change extends from (changeLeft, y) to (changeRight, y + changeHeight)
- newChanged->assign_union(Region(Rect(changeLeft, y, changeRight, y + changeHeight)));
-
- // Copy the change from fb to oldFb to allow future changes to be identified
- for (int row = 0; row < changeHeight; row++)
- {
- memcpy(oldPtr, newPtr, blockWidthInBytes);
- newPtr += newStrideBytes;
- oldPtr += oldStrideBytes;
- }
-
- // No further processing is required for this block
- break;
- }
-
- newPtr += newStrideBytes;
- oldPtr += oldStrideBytes;
- }
-
- oldBlockPtr += blockWidthInBytes;
- newBlockPtr += blockWidthInBytes;
- }
-
- oldData += oldStrideBytes * BLOCK_SIZE;
- }
-
- oldFb.commitBufferRW(r);
- }
-
- void ComparingUpdateTracker::logStats()
- {
- double ratio;
-
- ratio = (double)totalPixels / missedPixels;
-
- vlog.info("%s in / %s out",
- siPrefix(totalPixels, "pixels").c_str(),
- siPrefix(missedPixels, "pixels").c_str());
- vlog.info("(1:%g ratio)", ratio);
-
- totalPixels = missedPixels = 0;
- }
|