You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

ComparingUpdateTracker.cxx 7.9KB

Enhanced ComparingUpdateTracker to crop changed blocks 64x64 changed block can be large for fine changes such as cursor movement and typing in terminal windows, or an update to a clock. If the block can be efficiently cropped, this will reduce latency and bandwidth. Every pixel cropped is a pixel less to analyze, encode, transmit, and decode. The previous code already detected the top of the change in order to determine if the block had changed. However, it did not use this information to reduce the size of the change rectangle, nor did it calculate any of the other edges. The new code introduces detection of the other edges, and uses the information to build a reduced area change rectangle. This has the additional effect of reducing the number of discrete pixel values in the change block which may allow a more efficient encoding algorithm to be selected. As this section of code is performance sensitive, the method of detecting the edges has been optimized to quickly fall back to pessimistic values as soon as a single comparison fails on each edge. In the case that full 64x64 block are changing, there will be three extra comparisons per block. In cases where the change rectangle can be reduced from 64x64, the reduced size of the change rectangle represents reduced effort to encode, transfer, and decode the contained pixels. In the case of images with high frequency changes, which specifically includes text, the lossy JPEG encoding can be highly distorted, especially with JPEG level 6 or less. The quick flash from a distorted JPEG to a lossless JPEG can appear as a flickering to some people. This effect was more obvious when the surrounding area is not expected to change, but is being distorted anyways due to being part of the 64x64 blocking algorithm. In the case of a user typing in a terminal window, this change may commonly reduce the number of pixels updated with every character typed from 4096 pixels (64x64) to 640 pixels (32x20) or less.
3 년 전
Enhanced ComparingUpdateTracker to crop changed blocks 64x64 changed block can be large for fine changes such as cursor movement and typing in terminal windows, or an update to a clock. If the block can be efficiently cropped, this will reduce latency and bandwidth. Every pixel cropped is a pixel less to analyze, encode, transmit, and decode. The previous code already detected the top of the change in order to determine if the block had changed. However, it did not use this information to reduce the size of the change rectangle, nor did it calculate any of the other edges. The new code introduces detection of the other edges, and uses the information to build a reduced area change rectangle. This has the additional effect of reducing the number of discrete pixel values in the change block which may allow a more efficient encoding algorithm to be selected. As this section of code is performance sensitive, the method of detecting the edges has been optimized to quickly fall back to pessimistic values as soon as a single comparison fails on each edge. In the case that full 64x64 block are changing, there will be three extra comparisons per block. In cases where the change rectangle can be reduced from 64x64, the reduced size of the change rectangle represents reduced effort to encode, transfer, and decode the contained pixels. In the case of images with high frequency changes, which specifically includes text, the lossy JPEG encoding can be highly distorted, especially with JPEG level 6 or less. The quick flash from a distorted JPEG to a lossless JPEG can appear as a flickering to some people. This effect was more obvious when the surrounding area is not expected to change, but is being distorted anyways due to being part of the 64x64 blocking algorithm. In the case of a user typing in a terminal window, this change may commonly reduce the number of pixels updated with every character typed from 4096 pixels (64x64) to 640 pixels (32x20) or less.
3 년 전
Enhanced ComparingUpdateTracker to crop changed blocks 64x64 changed block can be large for fine changes such as cursor movement and typing in terminal windows, or an update to a clock. If the block can be efficiently cropped, this will reduce latency and bandwidth. Every pixel cropped is a pixel less to analyze, encode, transmit, and decode. The previous code already detected the top of the change in order to determine if the block had changed. However, it did not use this information to reduce the size of the change rectangle, nor did it calculate any of the other edges. The new code introduces detection of the other edges, and uses the information to build a reduced area change rectangle. This has the additional effect of reducing the number of discrete pixel values in the change block which may allow a more efficient encoding algorithm to be selected. As this section of code is performance sensitive, the method of detecting the edges has been optimized to quickly fall back to pessimistic values as soon as a single comparison fails on each edge. In the case that full 64x64 block are changing, there will be three extra comparisons per block. In cases where the change rectangle can be reduced from 64x64, the reduced size of the change rectangle represents reduced effort to encode, transfer, and decode the contained pixels. In the case of images with high frequency changes, which specifically includes text, the lossy JPEG encoding can be highly distorted, especially with JPEG level 6 or less. The quick flash from a distorted JPEG to a lossless JPEG can appear as a flickering to some people. This effect was more obvious when the surrounding area is not expected to change, but is being distorted anyways due to being part of the 64x64 blocking algorithm. In the case of a user typing in a terminal window, this change may commonly reduce the number of pixels updated with every character typed from 4096 pixels (64x64) to 640 pixels (32x20) or less.
3 년 전
Enhanced ComparingUpdateTracker to crop changed blocks 64x64 changed block can be large for fine changes such as cursor movement and typing in terminal windows, or an update to a clock. If the block can be efficiently cropped, this will reduce latency and bandwidth. Every pixel cropped is a pixel less to analyze, encode, transmit, and decode. The previous code already detected the top of the change in order to determine if the block had changed. However, it did not use this information to reduce the size of the change rectangle, nor did it calculate any of the other edges. The new code introduces detection of the other edges, and uses the information to build a reduced area change rectangle. This has the additional effect of reducing the number of discrete pixel values in the change block which may allow a more efficient encoding algorithm to be selected. As this section of code is performance sensitive, the method of detecting the edges has been optimized to quickly fall back to pessimistic values as soon as a single comparison fails on each edge. In the case that full 64x64 block are changing, there will be three extra comparisons per block. In cases where the change rectangle can be reduced from 64x64, the reduced size of the change rectangle represents reduced effort to encode, transfer, and decode the contained pixels. In the case of images with high frequency changes, which specifically includes text, the lossy JPEG encoding can be highly distorted, especially with JPEG level 6 or less. The quick flash from a distorted JPEG to a lossless JPEG can appear as a flickering to some people. This effect was more obvious when the surrounding area is not expected to change, but is being distorted anyways due to being part of the 64x64 blocking algorithm. In the case of a user typing in a terminal window, this change may commonly reduce the number of pixels updated with every character typed from 4096 pixels (64x64) to 640 pixels (32x20) or less.
3 년 전
Enhanced ComparingUpdateTracker to crop changed blocks 64x64 changed block can be large for fine changes such as cursor movement and typing in terminal windows, or an update to a clock. If the block can be efficiently cropped, this will reduce latency and bandwidth. Every pixel cropped is a pixel less to analyze, encode, transmit, and decode. The previous code already detected the top of the change in order to determine if the block had changed. However, it did not use this information to reduce the size of the change rectangle, nor did it calculate any of the other edges. The new code introduces detection of the other edges, and uses the information to build a reduced area change rectangle. This has the additional effect of reducing the number of discrete pixel values in the change block which may allow a more efficient encoding algorithm to be selected. As this section of code is performance sensitive, the method of detecting the edges has been optimized to quickly fall back to pessimistic values as soon as a single comparison fails on each edge. In the case that full 64x64 block are changing, there will be three extra comparisons per block. In cases where the change rectangle can be reduced from 64x64, the reduced size of the change rectangle represents reduced effort to encode, transfer, and decode the contained pixels. In the case of images with high frequency changes, which specifically includes text, the lossy JPEG encoding can be highly distorted, especially with JPEG level 6 or less. The quick flash from a distorted JPEG to a lossless JPEG can appear as a flickering to some people. This effect was more obvious when the surrounding area is not expected to change, but is being distorted anyways due to being part of the 64x64 blocking algorithm. In the case of a user typing in a terminal window, this change may commonly reduce the number of pixels updated with every character typed from 4096 pixels (64x64) to 640 pixels (32x20) or less.
3 년 전
Enhanced ComparingUpdateTracker to crop changed blocks 64x64 changed block can be large for fine changes such as cursor movement and typing in terminal windows, or an update to a clock. If the block can be efficiently cropped, this will reduce latency and bandwidth. Every pixel cropped is a pixel less to analyze, encode, transmit, and decode. The previous code already detected the top of the change in order to determine if the block had changed. However, it did not use this information to reduce the size of the change rectangle, nor did it calculate any of the other edges. The new code introduces detection of the other edges, and uses the information to build a reduced area change rectangle. This has the additional effect of reducing the number of discrete pixel values in the change block which may allow a more efficient encoding algorithm to be selected. As this section of code is performance sensitive, the method of detecting the edges has been optimized to quickly fall back to pessimistic values as soon as a single comparison fails on each edge. In the case that full 64x64 block are changing, there will be three extra comparisons per block. In cases where the change rectangle can be reduced from 64x64, the reduced size of the change rectangle represents reduced effort to encode, transfer, and decode the contained pixels. In the case of images with high frequency changes, which specifically includes text, the lossy JPEG encoding can be highly distorted, especially with JPEG level 6 or less. The quick flash from a distorted JPEG to a lossless JPEG can appear as a flickering to some people. This effect was more obvious when the surrounding area is not expected to change, but is being distorted anyways due to being part of the 64x64 blocking algorithm. In the case of a user typing in a terminal window, this change may commonly reduce the number of pixels updated with every character typed from 4096 pixels (64x64) to 640 pixels (32x20) or less.
3 년 전
Enhanced ComparingUpdateTracker to crop changed blocks 64x64 changed block can be large for fine changes such as cursor movement and typing in terminal windows, or an update to a clock. If the block can be efficiently cropped, this will reduce latency and bandwidth. Every pixel cropped is a pixel less to analyze, encode, transmit, and decode. The previous code already detected the top of the change in order to determine if the block had changed. However, it did not use this information to reduce the size of the change rectangle, nor did it calculate any of the other edges. The new code introduces detection of the other edges, and uses the information to build a reduced area change rectangle. This has the additional effect of reducing the number of discrete pixel values in the change block which may allow a more efficient encoding algorithm to be selected. As this section of code is performance sensitive, the method of detecting the edges has been optimized to quickly fall back to pessimistic values as soon as a single comparison fails on each edge. In the case that full 64x64 block are changing, there will be three extra comparisons per block. In cases where the change rectangle can be reduced from 64x64, the reduced size of the change rectangle represents reduced effort to encode, transfer, and decode the contained pixels. In the case of images with high frequency changes, which specifically includes text, the lossy JPEG encoding can be highly distorted, especially with JPEG level 6 or less. The quick flash from a distorted JPEG to a lossless JPEG can appear as a flickering to some people. This effect was more obvious when the surrounding area is not expected to change, but is being distorted anyways due to being part of the 64x64 blocking algorithm. In the case of a user typing in a terminal window, this change may commonly reduce the number of pixels updated with every character typed from 4096 pixels (64x64) to 640 pixels (32x20) or less.
3 년 전
Enhanced ComparingUpdateTracker to crop changed blocks 64x64 changed block can be large for fine changes such as cursor movement and typing in terminal windows, or an update to a clock. If the block can be efficiently cropped, this will reduce latency and bandwidth. Every pixel cropped is a pixel less to analyze, encode, transmit, and decode. The previous code already detected the top of the change in order to determine if the block had changed. However, it did not use this information to reduce the size of the change rectangle, nor did it calculate any of the other edges. The new code introduces detection of the other edges, and uses the information to build a reduced area change rectangle. This has the additional effect of reducing the number of discrete pixel values in the change block which may allow a more efficient encoding algorithm to be selected. As this section of code is performance sensitive, the method of detecting the edges has been optimized to quickly fall back to pessimistic values as soon as a single comparison fails on each edge. In the case that full 64x64 block are changing, there will be three extra comparisons per block. In cases where the change rectangle can be reduced from 64x64, the reduced size of the change rectangle represents reduced effort to encode, transfer, and decode the contained pixels. In the case of images with high frequency changes, which specifically includes text, the lossy JPEG encoding can be highly distorted, especially with JPEG level 6 or less. The quick flash from a distorted JPEG to a lossless JPEG can appear as a flickering to some people. This effect was more obvious when the surrounding area is not expected to change, but is being distorted anyways due to being part of the 64x64 blocking algorithm. In the case of a user typing in a terminal window, this change may commonly reduce the number of pixels updated with every character typed from 4096 pixels (64x64) to 640 pixels (32x20) or less.
3 년 전
Enhanced ComparingUpdateTracker to crop changed blocks 64x64 changed block can be large for fine changes such as cursor movement and typing in terminal windows, or an update to a clock. If the block can be efficiently cropped, this will reduce latency and bandwidth. Every pixel cropped is a pixel less to analyze, encode, transmit, and decode. The previous code already detected the top of the change in order to determine if the block had changed. However, it did not use this information to reduce the size of the change rectangle, nor did it calculate any of the other edges. The new code introduces detection of the other edges, and uses the information to build a reduced area change rectangle. This has the additional effect of reducing the number of discrete pixel values in the change block which may allow a more efficient encoding algorithm to be selected. As this section of code is performance sensitive, the method of detecting the edges has been optimized to quickly fall back to pessimistic values as soon as a single comparison fails on each edge. In the case that full 64x64 block are changing, there will be three extra comparisons per block. In cases where the change rectangle can be reduced from 64x64, the reduced size of the change rectangle represents reduced effort to encode, transfer, and decode the contained pixels. In the case of images with high frequency changes, which specifically includes text, the lossy JPEG encoding can be highly distorted, especially with JPEG level 6 or less. The quick flash from a distorted JPEG to a lossless JPEG can appear as a flickering to some people. This effect was more obvious when the surrounding area is not expected to change, but is being distorted anyways due to being part of the 64x64 blocking algorithm. In the case of a user typing in a terminal window, this change may commonly reduce the number of pixels updated with every character typed from 4096 pixels (64x64) to 640 pixels (32x20) or less.
3 년 전
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265
  1. /* Copyright (C) 2002-2005 RealVNC Ltd. All Rights Reserved.
  2. *
  3. * This is free software; you can redistribute it and/or modify
  4. * it under the terms of the GNU General Public License as published by
  5. * the Free Software Foundation; either version 2 of the License, or
  6. * (at your option) any later version.
  7. *
  8. * This software is distributed in the hope that it will be useful,
  9. * but WITHOUT ANY WARRANTY; without even the implied warranty of
  10. * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
  11. * GNU General Public License for more details.
  12. *
  13. * You should have received a copy of the GNU General Public License
  14. * along with this software; if not, write to the Free Software
  15. * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307,
  16. * USA.
  17. */
  18. #ifdef HAVE_CONFIG_H
  19. #include <config.h>
  20. #endif
  21. #include <stdio.h>
  22. #include <string.h>
  23. #include <vector>
  24. #include <rfb/Exception.h>
  25. #include <rfb/LogWriter.h>
  26. #include <rfb/ComparingUpdateTracker.h>
  27. using namespace rfb;
  28. static LogWriter vlog("ComparingUpdateTracker");
  29. ComparingUpdateTracker::ComparingUpdateTracker(PixelBuffer* buffer)
  30. : fb(buffer), oldFb(fb->getPF(), 0, 0), firstCompare(true),
  31. enabled(true), totalPixels(0), missedPixels(0)
  32. {
  33. changed.assign_union(fb->getRect());
  34. }
  35. ComparingUpdateTracker::~ComparingUpdateTracker()
  36. {
  37. }
  38. #define BLOCK_SIZE 64
  39. bool ComparingUpdateTracker::compare()
  40. {
  41. std::vector<Rect> rects;
  42. std::vector<Rect>::iterator i;
  43. if (!enabled)
  44. return false;
  45. if (firstCompare) {
  46. // NB: We leave the change region untouched on this iteration,
  47. // since in effect the entire framebuffer has changed.
  48. oldFb.setSize(fb->width(), fb->height());
  49. for (int y=0; y<fb->height(); y+=BLOCK_SIZE) {
  50. Rect pos(0, y, fb->width(), __rfbmin(fb->height(), y+BLOCK_SIZE));
  51. int srcStride;
  52. const uint8_t* srcData = fb->getBuffer(pos, &srcStride);
  53. oldFb.imageRect(pos, srcData, srcStride);
  54. }
  55. firstCompare = false;
  56. return false;
  57. }
  58. copied.get_rects(&rects, copy_delta.x<=0, copy_delta.y<=0);
  59. for (i = rects.begin(); i != rects.end(); i++)
  60. oldFb.copyRect(*i, copy_delta);
  61. changed.get_rects(&rects);
  62. Region newChanged;
  63. for (i = rects.begin(); i != rects.end(); i++)
  64. compareRect(*i, &newChanged);
  65. changed.get_rects(&rects);
  66. for (i = rects.begin(); i != rects.end(); i++)
  67. totalPixels += i->area();
  68. newChanged.get_rects(&rects);
  69. for (i = rects.begin(); i != rects.end(); i++)
  70. missedPixels += i->area();
  71. if (changed.equals(newChanged))
  72. return false;
  73. changed = newChanged;
  74. return true;
  75. }
  76. void ComparingUpdateTracker::enable()
  77. {
  78. enabled = true;
  79. }
  80. void ComparingUpdateTracker::disable()
  81. {
  82. enabled = false;
  83. // Make sure we update the framebuffer next time we get enabled
  84. firstCompare = true;
  85. }
  86. void ComparingUpdateTracker::compareRect(const Rect& r, Region* newChanged)
  87. {
  88. if (!r.enclosed_by(fb->getRect())) {
  89. Rect safe;
  90. // Crop the rect and try again
  91. safe = r.intersect(fb->getRect());
  92. if (!safe.is_empty())
  93. compareRect(safe, newChanged);
  94. return;
  95. }
  96. int bytesPerPixel = fb->getPF().bpp/8;
  97. int oldStride;
  98. uint8_t* oldData = oldFb.getBufferRW(r, &oldStride);
  99. int oldStrideBytes = oldStride * bytesPerPixel;
  100. // Used to efficiently crop the left and right of the change rectangle
  101. int minCompareWidthInPixels = BLOCK_SIZE / 8;
  102. int minCompareWidthInBytes = minCompareWidthInPixels * bytesPerPixel;
  103. for (int blockTop = r.tl.y; blockTop < r.br.y; blockTop += BLOCK_SIZE)
  104. {
  105. // Get a strip of the source buffer
  106. Rect pos(r.tl.x, blockTop, r.br.x, __rfbmin(r.br.y, blockTop+BLOCK_SIZE));
  107. int fbStride;
  108. const uint8_t* newBlockPtr = fb->getBuffer(pos, &fbStride);
  109. int newStrideBytes = fbStride * bytesPerPixel;
  110. uint8_t* oldBlockPtr = oldData;
  111. int blockBottom = __rfbmin(blockTop+BLOCK_SIZE, r.br.y);
  112. for (int blockLeft = r.tl.x; blockLeft < r.br.x; blockLeft += BLOCK_SIZE)
  113. {
  114. const uint8_t* newPtr = newBlockPtr;
  115. uint8_t* oldPtr = oldBlockPtr;
  116. int blockRight = __rfbmin(blockLeft+BLOCK_SIZE, r.br.x);
  117. int blockWidthInBytes = (blockRight-blockLeft) * bytesPerPixel;
  118. // Scan the block top to bottom, to identify the first row of change
  119. for (int y = blockTop; y < blockBottom; y++)
  120. {
  121. if (memcmp(oldPtr, newPtr, blockWidthInBytes) != 0)
  122. {
  123. // Define the change rectangle using pessimistic values to start
  124. int changeHeight = blockBottom - y;
  125. int changeLeft = blockLeft;
  126. int changeRight = blockRight;
  127. // For every unchanged row at the bottom of the block, decrement change height
  128. {
  129. const uint8_t* newRowPtr = newPtr + ((changeHeight - 1) * newStrideBytes);
  130. const uint8_t* oldRowPtr = oldPtr + ((changeHeight - 1) * oldStrideBytes);
  131. while (changeHeight > 1 && memcmp(oldRowPtr, newRowPtr, blockWidthInBytes) == 0)
  132. {
  133. newRowPtr -= newStrideBytes;
  134. oldRowPtr -= oldStrideBytes;
  135. changeHeight--;
  136. }
  137. }
  138. // For every unchanged column at the left of the block, increment change left
  139. {
  140. const uint8_t* newColumnPtr = newPtr;
  141. const uint8_t* oldColumnPtr = oldPtr;
  142. while (changeLeft + minCompareWidthInPixels < changeRight)
  143. {
  144. const uint8_t* newRowPtr = newColumnPtr;
  145. const uint8_t* oldRowPtr = oldColumnPtr;
  146. for (int row = 0; row < changeHeight; row++)
  147. {
  148. if (memcmp(oldRowPtr, newRowPtr, minCompareWidthInBytes) != 0)
  149. goto endOfChangeLeft;
  150. newRowPtr += newStrideBytes;
  151. oldRowPtr += oldStrideBytes;
  152. }
  153. newColumnPtr += minCompareWidthInBytes;
  154. oldColumnPtr += minCompareWidthInBytes;
  155. changeLeft += minCompareWidthInPixels;
  156. }
  157. }
  158. endOfChangeLeft:
  159. // For every unchanged column at the right of the block, decrement change right
  160. {
  161. const uint8_t* newColumnPtr = newPtr + blockWidthInBytes;
  162. const uint8_t* oldColumnPtr = oldPtr + blockWidthInBytes;
  163. while (changeLeft + minCompareWidthInPixels < changeRight)
  164. {
  165. newColumnPtr -= minCompareWidthInBytes;
  166. oldColumnPtr -= minCompareWidthInBytes;
  167. const uint8_t* newRowPtr = newColumnPtr;
  168. const uint8_t* oldRowPtr = oldColumnPtr;
  169. for (int row = 0; row < changeHeight; row++)
  170. {
  171. if (memcmp(oldRowPtr, newRowPtr, minCompareWidthInBytes) != 0)
  172. goto endOfChangeRight;
  173. newRowPtr += newStrideBytes;
  174. oldRowPtr += oldStrideBytes;
  175. }
  176. changeRight -= minCompareWidthInPixels;
  177. }
  178. }
  179. endOfChangeRight:
  180. // Block change extends from (changeLeft, y) to (changeRight, y + changeHeight)
  181. newChanged->assign_union(Region(Rect(changeLeft, y, changeRight, y + changeHeight)));
  182. // Copy the change from fb to oldFb to allow future changes to be identified
  183. for (int row = 0; row < changeHeight; row++)
  184. {
  185. memcpy(oldPtr, newPtr, blockWidthInBytes);
  186. newPtr += newStrideBytes;
  187. oldPtr += oldStrideBytes;
  188. }
  189. // No further processing is required for this block
  190. break;
  191. }
  192. newPtr += newStrideBytes;
  193. oldPtr += oldStrideBytes;
  194. }
  195. oldBlockPtr += blockWidthInBytes;
  196. newBlockPtr += blockWidthInBytes;
  197. }
  198. oldData += oldStrideBytes * BLOCK_SIZE;
  199. }
  200. oldFb.commitBufferRW(r);
  201. }
  202. void ComparingUpdateTracker::logStats()
  203. {
  204. double ratio;
  205. ratio = (double)totalPixels / missedPixels;
  206. vlog.info("%s in / %s out",
  207. siPrefix(totalPixels, "pixels").c_str(),
  208. siPrefix(missedPixels, "pixels").c_str());
  209. vlog.info("(1:%g ratio)", ratio);
  210. totalPixels = missedPixels = 0;
  211. }