Merged revisions 627779-634630 via svnmerge from

https://svn.apache.org/repos/asf/poi/trunk

........
  r627779 | nick | 2008-02-14 16:32:49 +0100 (Thu, 14 Feb 2008) | 1 line
  
  In the interests of sanity, stop having hssf test data files in scratchpad and main, go to just having them in main
........
  r627788 | nick | 2008-02-14 17:01:10 +0100 (Thu, 14 Feb 2008) | 1 line
  
  Big formula update from Josh from bug #44364 - support for Match, NA and SumProduct functions, and initial error support in functions
........
  r627999 | nick | 2008-02-15 11:30:10 +0100 (Fri, 15 Feb 2008) | 1 line
  
  To avoid confusion and repeated changes in svn, update the TestDataValidation test to output its file (that needs opening in excel to check to output) into the system tmp directory
........
  r628027 | nick | 2008-02-15 12:45:13 +0100 (Fri, 15 Feb 2008) | 1 line
  
  Fix for bug #44403 - Have mid use the third argument properly, and test
........
  r628029 | nick | 2008-02-15 12:53:25 +0100 (Fri, 15 Feb 2008) | 1 line
  
  Fix for bug #44413 from Josh - Fix for circular references in INDEX, OFFSET, VLOOKUP formulas, where a cell is actually allowed to reference itself
........
  r628033 | nick | 2008-02-15 13:04:42 +0100 (Fri, 15 Feb 2008) | 1 line
  
  Fix from Josh from bug #44417 - Improved handling of references for the need to quote the sheet name for some formulas, but not when fetching a sheet by name
........
  r628035 | nick | 2008-02-15 13:13:25 +0100 (Fri, 15 Feb 2008) | 1 line
  
  Fix from Josh from bug #44421 - Update Match function to properly support Area references
........
  r628044 | nick | 2008-02-15 13:59:40 +0100 (Fri, 15 Feb 2008) | 1 line
  
  Partial fix for bug #44410 - support whole column ranges such as C:C in the formula evaluator (so SUM(D:D) will now work). However, the formula string will still be displayed wrong
........
  r628065 | nick | 2008-02-15 14:50:38 +0100 (Fri, 15 Feb 2008) | 1 line
  
  Further support for whole-column references, including formula strings and the evaluator. Also has some new tests for it
........
  r628714 | nick | 2008-02-18 14:08:16 +0100 (Mon, 18 Feb 2008) | 1 line
  
  Update notice for latest guidance on ooxml xsd licence, and update getting involved to link to the newly released binary file format docs
........
  r629552 | nick | 2008-02-20 19:14:30 +0100 (Wed, 20 Feb 2008) | 1 line
  
  Patch from Josh from bug #44403 - Further support for unusual, but valid, arguments to the Mid function
........
  r629738 | nick | 2008-02-21 11:36:08 +0100 (Thu, 21 Feb 2008) | 1 line
  
  Fix from Josh from bug #44456 - Update contrib SViewer to not fail if a HSSFRow is null
........
  r629742 | nick | 2008-02-21 11:49:25 +0100 (Thu, 21 Feb 2008) | 1 line
  
  Use the right way to figure out how many rows on a sheet, so we display the row number for all of them on the left hand side. Also, tidy up some imports
........
  r629755 | nick | 2008-02-21 12:34:25 +0100 (Thu, 21 Feb 2008) | 1 line
  
  Fix bug 38921, where HSSFPalette.findSimilar() wasn't working properly, and add tests for it
........
  r629821 | nick | 2008-02-21 16:08:44 +0100 (Thu, 21 Feb 2008) | 1 line
  
  Patch from Josh from bug #44371 - support for OFFSET function, and various tweaks to the formula evaluator to support this
........
  r629829 | nick | 2008-02-21 16:35:59 +0100 (Thu, 21 Feb 2008) | 1 line
  
  Patch from Josh from bug #44366 - InputStreams passed to POIFSFileSystem are now automatically closed. A warning is generated for people who might've relied on them not being closed before, and a wrapper to restore the old behaviour is supplied
........
  r629831 | nick | 2008-02-21 16:40:34 +0100 (Thu, 21 Feb 2008) | 1 line
  
  Patch from Josh from bug #44437 - improved unit test for poifs
........
  r629832 | nick | 2008-02-21 16:42:06 +0100 (Thu, 21 Feb 2008) | 1 line
  
  Patch from Josh from bug #44437 - improved unit test for poifs
........
  r629837 | nick | 2008-02-21 16:48:52 +0100 (Thu, 21 Feb 2008) | 1 line
  
  Patch from Josh from bug #44449 - Handle SharedFormulas better, for where there are formulas for the same area on two sheets, and when the shared formula flag is set incorrectly
........
  r629849 | nick | 2008-02-21 17:22:18 +0100 (Thu, 21 Feb 2008) | 1 line
  
  Add a disabled test for a file with whacky StyleRecords that trigger an AIOOB
........
  r629865 | nick | 2008-02-21 17:44:46 +0100 (Thu, 21 Feb 2008) | 1 line
  
  At the request of legal-discuss, shuffle the ooxml xsd licence details into LICENSE from NOTICE
........
  r630160 | nick | 2008-02-22 12:23:50 +0100 (Fri, 22 Feb 2008) | 1 line
  
  Patch from Josh from bug #44450 - VLookup and HLookup support, and improvements to Lookup and Offset
........
  r630164 | nick | 2008-02-22 12:40:00 +0100 (Fri, 22 Feb 2008) | 1 line
  
  Bug #44471 - Crystal Reports generates files with short StyleRecords, which isn't allowed in the spec. Work around this
........
  r633114 | nick | 2008-03-03 16:01:18 +0100 (Mon, 03 Mar 2008) | 1 line
  
  Patch from Paolo from bug #44481 - getVerticallyCenter shouldn't take a parameter, but leave the old version in as deprecated for now
........
  r633118 | nick | 2008-03-03 16:10:46 +0100 (Mon, 03 Mar 2008) | 1 line
  
  Fix from Yegor from bug #44491 - don't have the new style handy POIDocument property stuff break old style hpsf+hssf use
........
  r633126 | nick | 2008-03-03 16:26:38 +0100 (Mon, 03 Mar 2008) | 1 line
  
  Patch from Josh from bug #44495 - Handle named cell ranges in formulas that have lower case parts
........
  r633151 | nick | 2008-03-03 17:09:02 +0100 (Mon, 03 Mar 2008) | 1 line
  
  Patch from Josh from bug #44510 - Fix how DVALRecord works with dropdowns
........
  r633169 | nick | 2008-03-03 17:55:00 +0100 (Mon, 03 Mar 2008) | 1 line
  
  Patch from Josh from bug #44508 - Fix formula evaluation with evaluateInCell on boolean formulas
........
  r633205 | nick | 2008-03-03 18:47:36 +0100 (Mon, 03 Mar 2008) | 1 line
  
  Fix indent, add more documentation, and make the error message more helpful
........
  r633505 | nick | 2008-03-04 16:06:29 +0100 (Tue, 04 Mar 2008) | 1 line
  
  Problem files from bug #44501
........
  r633547 | nick | 2008-03-04 17:53:32 +0100 (Tue, 04 Mar 2008) | 1 line
  
  Big patch from Josh from bug #44504 - lots of formula parser improvements
........
  r633548 | nick | 2008-03-04 17:59:02 +0100 (Tue, 04 Mar 2008) | 1 line
  
  Changelog update for last patch
........
  r634318 | nick | 2008-03-06 16:54:06 +0100 (Thu, 06 Mar 2008) | 1 line
  
  Change the behaviour on short last blocks to be a warning not an exception, as some people seem to have "real" valid files that trigger this. Fixed bug #28231
........
  r634371 | nick | 2008-03-06 19:06:48 +0100 (Thu, 06 Mar 2008) | 1 line
  
  Embeded files from bug #44524
........
  r634372 | nick | 2008-03-06 19:13:47 +0100 (Thu, 06 Mar 2008) | 1 line
  
  Add broken test for bug #43901
........
  r634617 | nick | 2008-03-07 12:18:02 +0100 (Fri, 07 Mar 2008) | 1 line
  
  Patch from Josh from bug #43901 - Correctly update the internal last cell number when adding and removing cells (previously sometimes off-by-one)
........
  r634619 | nick | 2008-03-07 12:36:14 +0100 (Fri, 07 Mar 2008) | 1 line
  
  Improved support for read-only recommended workbooks, fixing bug #44536
........
  r634630 | nick | 2008-03-07 13:06:18 +0100 (Fri, 07 Mar 2008) | 1 line
  
  Patch largely from Josh from bug #44539 - Support for area references in formulas of rows >= 32768
........


git-svn-id: https://svn.apache.org/repos/asf/poi/branches/ooxml@634936 13f79535-47bb-0310-9956-ffa450edef68

2 files changed, 109 insertions, 19 deletions

diff --git a/src/java/org/apache/poi/poifs/filesystem/POIFSFileSystem.java b/src/java/org/apache/poi/poifs/filesystem/POIFSFileSystem.java
index 771db767be..ef9acfe60b 100644
--- a/src/java/org/apache/poi/poifs/filesystem/POIFSFileSystem.java
+++ b/src/java/org/apache/poi/poifs/filesystem/POIFSFileSystem.java
@@ -19,6 +19,7 @@
 
 package org.apache.poi.poifs.filesystem;
 
+import java.io.ByteArrayInputStream;
 import java.io.FileInputStream;
 import java.io.FileOutputStream;
 import java.io.IOException;
@@ -30,6 +31,8 @@ import java.util.Collections;
 import java.util.Iterator;
 import java.util.List;
 
+import org.apache.commons.logging.Log;
+import org.apache.commons.logging.LogFactory;
 import org.apache.poi.poifs.dev.POIFSViewable;
 import org.apache.poi.poifs.property.DirectoryProperty;
 import org.apache.poi.poifs.property.Property;
@@ -58,6 +61,33 @@ import org.apache.poi.util.LongField;
 public class POIFSFileSystem
     implements POIFSViewable
 {
+    private static final Log _logger = LogFactory.getLog(POIFSFileSystem.class);
+    
+    
+    private static final class CloseIgnoringInputStream extends InputStream {
+
+        private final InputStream _is;
+        public CloseIgnoringInputStream(InputStream is) {
+            _is = is;
+        }
+        public int read() throws IOException {
+            return _is.read();
+        }
+        public int read(byte[] b, int off, int len) throws IOException {
+            return _is.read(b, off, len);
+        }
+        public void close() {
+            // do nothing
+        }
+    }
+    
+    /**
+     * Convenience method for clients that want to avoid the auto-close behaviour of the constructor.
+     */
+    public static InputStream createNonClosingInputStream(InputStream is) {
+        return new CloseIgnoringInputStream(is);
+    }
+    
     private PropertyTable _property_table;
     private List          _documents;
     private DirectoryNode _root;
@@ -74,23 +104,52 @@ public class POIFSFileSystem
     }
 
     /**
-     * Create a POIFSFileSystem from an InputStream
+     * Create a POIFSFileSystem from an <tt>InputStream</tt>.  Normally the stream is read until
+     * EOF.  The stream is always closed.<p/>
+     * 
+     * Some streams are usable after reaching EOF (typically those that return <code>true</code> 
+     * for <tt>markSupported()</tt>).  In the unlikely case that the caller has such a stream 
+     * <i>and</i> needs to use it after this constructor completes, a work around is to wrap the
+     * stream in order to trap the <tt>close()</tt> call.  A convenience method (
+     * <tt>createNonClosingInputStream()</tt>) has been provided for this purpose:
+     * <pre>
+     * InputStream wrappedStream = POIFSFileSystem.createNonClosingInputStream(is);
+     * HSSFWorkbook wb = new HSSFWorkbook(wrappedStream);
+     * is.reset(); 
+     * doSomethingElse(is); 
+     * </pre>
+     * Note also the special case of <tt>ByteArrayInputStream</tt> for which the <tt>close()</tt>
+     * method does nothing. 
+     * <pre>
+     * ByteArrayInputStream bais = ...
+     * HSSFWorkbook wb = new HSSFWorkbook(bais); // calls bais.close() !
+     * bais.reset(); // no problem
+     * doSomethingElse(bais);
+     * </pre>
      *
      * @param stream the InputStream from which to read the data
      *
      * @exception IOException on errors reading, or on invalid data
      */
 
-    public POIFSFileSystem(final InputStream stream)
+    public POIFSFileSystem(InputStream stream)
         throws IOException
     {
         this();
+        boolean success = false;
 
         // read the header block from the stream
-        HeaderBlockReader header_block_reader = new HeaderBlockReader(stream);
-
+        HeaderBlockReader header_block_reader;
         // read the rest of the stream into blocks
-        RawDataBlockList  data_blocks         = new RawDataBlockList(stream);
+        RawDataBlockList data_blocks;
+        try {
+            header_block_reader = new HeaderBlockReader(stream);
+            data_blocks = new RawDataBlockList(stream);
+            success = true;
+        } finally {
+            closeInputStream(stream, success);
+        }
+        
 
         // set up the block allocation table (necessary for the
         // data_blocks to be manageable
@@ -112,7 +171,32 @@ public class POIFSFileSystem
                     .getSBATStart()), data_blocks, properties.getRoot()
                         .getChildren(), null);
     }
-    
+    /**
+     * @param stream the stream to be closed
+     * @param success <code>false</code> if an exception is currently being thrown in the calling method
+     */
+    private void closeInputStream(InputStream stream, boolean success) {
+        
+        if(stream.markSupported() && !(stream instanceof ByteArrayInputStream)) {
+            String msg = "POIFS is closing the supplied input stream of type (" 
+                    + stream.getClass().getName() + ") which supports mark/reset.  "
+                    + "This will be a problem for the caller if the stream will still be used.  "
+                    + "If that is the case the caller should wrap the input stream to avoid this close logic.  "
+                    + "This warning is only temporary and will not be present in future versions of POI.";
+            _logger.warn(msg);
+        }
+        try {
+            stream.close();
+        } catch (IOException e) {
+            if(success) {
+                throw new RuntimeException(e);
+            }
+            // else not success? Try block did not complete normally 
+            // just print stack trace and leave original ex to be thrown
+            e.printStackTrace();
+        }
+    }
+
     /**
      * Checks that the supplied InputStream (which MUST
      *  support mark and reset, or be a PushbackInputStream) 
@@ -123,23 +207,23 @@ public class POIFSFileSystem
      * @param inp An InputStream which supports either mark/reset, or is a PushbackInputStream 
      */
     public static boolean hasPOIFSHeader(InputStream inp) throws IOException {
-    	// We want to peek at the first 8 bytes 
-    	inp.mark(8);
+        // We want to peek at the first 8 bytes 
+        inp.mark(8);
 
-    	byte[] header = new byte[8];
-    	IOUtils.readFully(inp, header);
+        byte[] header = new byte[8];
+        IOUtils.readFully(inp, header);
         LongField signature = new LongField(HeaderBlockConstants._signature_offset, header);
 
         // Wind back those 8 bytes
         if(inp instanceof PushbackInputStream) {
-        	PushbackInputStream pin = (PushbackInputStream)inp;
-        	pin.unread(header);
+            PushbackInputStream pin = (PushbackInputStream)inp;
+            pin.unread(header);
         } else {
-        	inp.reset();
+            inp.reset();
         }
-    	
-    	// Did it match the signature?
-    	return (signature.get() == HeaderBlockConstants._signature);
+        
+        // Did it match the signature?
+        return (signature.get() == HeaderBlockConstants._signature);
     }
 
     /**
diff --git a/src/java/org/apache/poi/poifs/storage/RawDataBlock.java b/src/java/org/apache/poi/poifs/storage/RawDataBlock.java
index d554298400..472fd8b8b5 100644
--- a/src/java/org/apache/poi/poifs/storage/RawDataBlock.java
+++ b/src/java/org/apache/poi/poifs/storage/RawDataBlock.java
@@ -21,6 +21,8 @@ package org.apache.poi.poifs.storage;
 
 import org.apache.poi.poifs.common.POIFSConstants;
 import org.apache.poi.util.IOUtils;
+import org.apache.poi.util.POILogFactory;
+import org.apache.poi.util.POILogger;
 
 import java.io.*;
 
@@ -35,6 +37,7 @@ public class RawDataBlock
 {
     private byte[]  _data;
     private boolean _eof;
+    private static POILogger log = POILogFactory.getLogger(RawDataBlock.class);
 
     /**
      * Constructor RawDataBlock
@@ -75,9 +78,12 @@ public class RawDataBlock
             String type = " byte" + ((count == 1) ? ("")
                                                   : ("s"));
 
-            throw new IOException("Unable to read entire block; " + count
-                                  + type + " read before EOF; expected "
-                                  + blockSize + " bytes");
+            log.log(POILogger.ERROR,
+            		"Unable to read entire block; " + count
+                     + type + " read before EOF; expected "
+                     + blockSize + " bytes. Your document"
+                     + " has probably been truncated!"
+            );
         }
         else {
             _eof = false;