casacore
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Groups Pages
PagedArray.h
Go to the documentation of this file.
1 //# PagedArray.h: templated Lattice, paged from disk to memory on demand
2 //# Copyright (C) 1994,1995,1996,1997,1998,1999,2000,2001,2002,2003
3 //# Associated Universities, Inc. Washington DC, USA.
4 //#
5 //# This library is free software; you can redistribute it and/or modify it
6 //# under the terms of the GNU Library General Public License as published by
7 //# the Free Software Foundation; either version 2 of the License, or (at your
8 //# option) any later version.
9 //#
10 //# This library is distributed in the hope that it will be useful, but WITHOUT
11 //# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
12 //# FITNESS FOR A PARTICULAR PURPOSE. See the GNU Library General Public
13 //# License for more details.
14 //#
15 //# You should have received a copy of the GNU Library General Public License
16 //# along with this library; if not, write to the Free Software Foundation,
17 //# Inc., 675 Massachusetts Ave, Cambridge, MA 02139, USA.
18 //#
19 //# Correspondence concerning AIPS++ should be addressed as follows:
20 //# Internet email: aips2-request@nrao.edu.
21 //# Postal address: AIPS++ Project Office
22 //# National Radio Astronomy Observatory
23 //# 520 Edgemont Road
24 //# Charlottesville, VA 22903-2475 USA
25 //#
26 //# $Id$
27 
28 #ifndef LATTICES_PAGEDARRAY_H
29 #define LATTICES_PAGEDARRAY_H
30 
31 //# Includes
32 #include <casacore/casa/aips.h>
39 
40 
41 namespace casacore { //# NAMESPACE CASACORE - BEGIN
42 
43 // <summary>
44 // A Lattice that is read from or written to disk.
45 // </summary>
46 
47 // <use visibility=export>
48 
49 // <reviewed reviewer="Peter Barnes" date="1999/10/30" tests="tPagedArray.cc" demos="dPagedArray.cc">
50 // </reviewed>
51 
52 // <prerequisite>
53 // <li> <linkto class="Lattice">Lattice</linkto>
54 // <li> <linkto class="TiledShape">TiledShape</linkto>
55 // </prerequisite>
56 
57 // <etymology>
58 // "Demand paging" is a technique used to implement virtual memory in
59 // computer operating systems. In this scheme, code or data are read from
60 // disk to memory only as needed by a process, and are read in fixed-sized
61 // chunks called "pages". PagedArrays are somewhat the same -- though
62 // without the automatic features found in virtual memory demand paging.
63 // However PagedArrays do allow the user to access chunks of the disk in a
64 // flexible way, that can match the requirements of many algorithms.
65 // </etymology>
66 
67 // <synopsis>
68 // At the time of writing, typical scientific computers provide sufficient
69 // memory for storing and manipulating 2-dimensional astronomical images,
70 // which have average size of around 8 MBytes. Astronomy is increasingly
71 // using three or higher dimensional arrays, which can be larger by one or
72 // two orders of magnitude. PagedArrays provide a convenient way of
73 // accessing these large arrays without requiring all the data to be read
74 // into real or virtual memory.
75 // <p>
76 // When you construct a PagedArray you do not read any data into
77 // memory. Instead a disk file (ie. a Table) is created, in a place you
78 // specify, to hold the data. This means you need to have enough disk space
79 // to hold the array. Constructing a PagedArray is equivalent to opening a
80 // file.
81 // <p>
82 // Because the data is stored on disk it can be saved after the program,
83 // function, or task that created the PagedArray has finished. This saved
84 // array can then be read again at a later stage.
85 // <p>
86 // So there are two reasons for using a PagedArray:
87 // <ol>
88 // <li> To provide for arrays that are too large for the computer's memory.
89 // <li> To provide a way of saving arrays to disk for later access.
90 // </ol>
91 //
92 // To access the data in a PagedArray you can either:
93 // <ol>
94 // <li> Use a <linkto class=LatticeIterator>LatticeIterator</linkto>
95 // <li> Use the getSlice and putSlice member functions
96 // <li> Use the parenthesis operator or getAt and putAt functions
97 // </ol>
98 // These access methods are given in order of preference. Some examples of
99 // these access methods are in the documentation for the
100 // <linkto class=Lattice>Lattice</linkto> class as well as below.
101 // <p>
102 // In nearly all cases you access the PagedArray by reading a "slice" of the
103 // PagedArray into a Casacore <linkto class=Array>Array</linkto>. Because the
104 // slice is stored in memory it is important that the slice you read is not
105 // too big compared to the physical memory on your computer. Otherwise your
106 // computer will page excessively and performance will be poor.
107 // <p>
108 // To overcome this you may be tempted to access the PagedArray a pixel at a
109 // time. This will use little memory but the overhead of accessing a large
110 // data set by separately reading each pixel from disk will also lead to poor
111 // performance.
112 // <p>
113 // In general the best way to access the data in PagedArrays is to use a
114 // LatticeIterator with a cursor size that "fits" nicely into memory. Not
115 // only do the LaticeIterator classes provide a relatively simple way to
116 // read/write all the data but they optimally set up the cache that is
117 // associated with each PagedArray.
118 // <p>
119 // If the LatticeIterator classes do not access the data the way you want
120 // you can use the getSlice and putSlice member functions. These functions
121 // do not set up the cache for you and improved performance may be obtained
122 // by tweaking the cache using the setCacheSizeFromPath member frunction.
123 //
124 // <ANCHOR NAME="PagedArray:Advanced"><h3>More Details</h3></ANCHOR>
125 // In order to utilise PagedArrays fully and understand many of the member
126 // functions and data access methods in this class, you need to be familiar
127 // with some of the concepts involved in the implementation of PagedArrays.
128 // <p>
129 // Each PagedArray is stored in one cell of a Table as an indirect Array
130 // (see the documentation for the <linkto module="Tables">Tables</linkto>
131 // module for more information). This means that multiple PagedArrays can be
132 // stored in one Table. To specify which PagedArray you are referring to in
133 // a given Table you need to specify the cell using its column name and row
134 // number during construction. If a cell is not specified the default column
135 // name (as given by the defaultColumnName function) and row number (as
136 // given by the defaultRowNumber function) are used. This ability to store
137 // multiple PagedArrays's is used in the PagedImage class where the image is
138 // stored in one cell and a mask is optionally stored in a another column in
139 // the same row.
140 // <p>
141 // There are currently a number of limitations when storing multiple
142 // PagedArrays in the same Table.
143 // <ul>
144 // <li> All the PagedArrays in the same column MUST have the same number of
145 // dimensions. The dimension used for any particular column is set when the
146 // first PagedArray in that column is constructed. If you want to put a
147 // say two-dimensional PagedArray into another row of a column that
148 // already contains a four-dimensional PagedArray you need to add two
149 // degenerate axes. In principle you could use the resize function, but see
150 // below for why this is not recommended. It is better to just ensure that
151 // all the PagedArrays have the same number of dimensions.
152 // <li> All the cells in a column that contains PagedArrays must have their
153 // shape defined. This becomes important if you are creating a PagedArray in
154 // say row five of a Table that currently only has one row. The PagedArray
155 // constructor will add another four rows to the Table, and put your
156 // PagedArray (with the shape you specify) in row five. For the three
157 // rows for which no shape was specified, the constructor will construct
158 // PagedArrays with only one element (and of an appropriate
159 // dimensionality). As you cannot resize these single element PagedArrays
160 // without difficulty (see below), it is recommended that you add
161 // PagedArrays to rows in your Table sequentially. It is necessary to have
162 // the constructor define the shape of all cells in the Table as it is an
163 // error to write a Table to disk with undefined cell shapes.
164 // </ul>
165 //
166 // Each PagedArray is stored on disk using the tiled cell storage manager
167 // (<linkto class=TiledCellStMan>TiledCellStMan</linkto>). This stores the
168 // data in tiles which are regular subsections of the PagedArray. For
169 // example a PagedArray of shape [1024,1024,4,128] may have a tile shape of
170 // [32,16,4,16]. The data in each tile is stored as a unit on the disk. This
171 // means that there is no preferred axis when accessing multi-dimensional
172 // data.
173 // <br>
174 // The tile shape can be specified when constructing a new PagedArray but
175 // not when reading an old one as it is intrinsic to the way the data is
176 // stored on disk. It is NOT recommended that you specify the tile shape
177 // unless you can control the lifetime of the PagedArray (this includes the
178 // time it spends on disk), or can guarantee the access pattern. For example
179 // if you know that a PagedArray of shape [512,512,4,32] will always be
180 // sliced plane by plane you may prefer to specify a tile shape of
181 // [512,64,1,1] rather than the default of [32,16,4,16].
182 // <br>
183 // Tiles can be cached by the tile storage manager so that it does not need
184 // to read the data from disk every time you are accessing the a pixel in a
185 // different tile. In order to cache the correct tiles you should tell the
186 // storage manager what section of the PagedArray you will be
187 // accessing. This is done using the setCacheSizeFromPath member
188 // function. Alternatively you can set the size of the cache using the
189 // setCacheSizeInTiles member function.
190 // <br>
191 // By default there is no limit on how much memory the tile cache can
192 // consume. This can be changed using the setMaximumCacheSize member
193 // function. The tiled storage manager always tries to cache enough tiles to
194 // ensure that each tile is read from disk only once, so setting the maximum
195 // cache size will trade off memory usage for disk I/O. Setting the cache
196 // size is illustrated in example 5 below.
197 // <br>
198 // The showCacheStatistics member function is provided to allow you to
199 // evaluate the performance of the tile cache.
200 // </synopsis>
201 
202 // <example>
203 // All the examples in this section are available in dPagedArray.cc
204 //
205 // <h4>Example 1:</h4>
206 // Create a PagedArray of Floats of shape [1024,1024,4,256] in a file
207 // called "myData_tmp.array" and initialize it to zero. This will create a
208 // directory on disk called "myData_tmp.array" that contains files that
209 // exceed 1024*1024*4*256*4 (= 4 GBytes) in size.
210 // <srcblock>
211 // const IPosition arrayShape(4,1024,1024,4,256);
212 // const String filename("myData_tmp.array");
213 // PagedArray<Float> diskArray(arrayShape, filename);
214 // cout << "Created a PagedArray of shape " << diskArray.shape()
215 // << " (" << diskArray.shape().product()/1024/1024*sizeof(Float)
216 // << " MBytes)" << endl
217 // << "in the table called " << diskArray.tableName() << endl;
218 // diskArray.set(0.0f);
219 // // Using the set function is an efficient way to initialize the PagedArray
220 // // as it uses a PagedArrIter internally. Note that the set function is
221 // // defined in the Lattice class that PagedArray is derived from.
222 // </srcblock>
223 //
224 // <h4>Example 2:</h4>
225 // Read the PagedArray produced in Example 1 and put a Gaussian profile into
226 // each spectral channel.
227 // <srcblock>
228 // PagedArray<Float> diskArray("myData_tmp.array");
229 // IPosition shape = diskArray.shape();
230 // // Construct a Gaussian Profile to be 10 channels wide and centred on
231 // // channel 16. Its height is 1.0.
232 // Gaussian1D<Float> g(1.0f, 16.0f, 10.0f);
233 // // Create a vector to cache a sampled version of this profile.
234 // Vector<Float> profile(shape(3));
235 // indgen(profile);
236 // profile.apply(g);
237 // // Now put this profile into every spectral channel in the paged array. This
238 // // is best done using an iterator.
239 // LatticeIterator<Float> iter(diskArray,
240 // TiledLineStepper(shape, diskArray.tileShape(), 3));
241 // for (iter.reset(); !iter.atEnd(); iter++) {
242 // iter.woCursor() = profile;
243 // }
244 // </srcblock>
245 //
246 // <h4>Example 3:</h4>
247 // Now multiply the I-polarization data by 10.0 in this PagedArray. The
248 // I-polarization data occupies 1 GByte of RAM which is too big to read
249 // into the memory of most computers. So an iterator is used to get suitable
250 // sized chunks.
251 // <srcblock>
252 // Table t("myData_tmp.array", Table::Update);
253 // PagedArray<Float> da(t);
254 // const IPosition latticeShape = da.shape();
255 // const nx = latticeShape(0);
256 // const ny = latticeShape(1);
257 // const npol = latticeShape(2);
258 // const nchan = latticeShape(3);
259 // IPosition cursorShape = da.niceCursorShape();
260 // cursorShape(2) = 1;
261 // LatticeStepper step(latticeShape, cursorShape);
262 // step.subSection(IPosition(4,0), IPosition(4,nx-1,ny-1,0,nchan-1));
263 // LatticeIterator<Float> iter(da, step);
264 // for (iter.reset(); !iter.atEnd(); iter++) {
265 // iter.rwCursor() *= 10.0f;
266 // }
267 // </srcblock>
268 //
269 // <h4>Example 4:</h4>
270 // Use a direct call to getSlice to access a small central region of the
271 // V-polarization in spectral channel 0 only. The region is small enough
272 // to not warrant constructing iterators and setting up
273 // LatticeNavigators. In this example the call to the getSlice function
274 // is unnecessary but is done for illustration purposes anyway.
275 // <srcblock>
276 // SetupNewTable maskSetup("mask_tmp.array", TableDesc(), Table::New);
277 // Table maskTable(maskSetup);
278 // PagedArray<Bool> maskArray(IPosition(4,1024,1024,4,256), maskTable);
279 // maskArray.set(False);
280 // COWPtr<Array<Bool> > maskPtr;
281 // maskArray.getSlice(maskPtr, IPosition(4,240,240,3,0),
282 // IPosition(4,32,32,1,1), IPosition(4,1));
283 // maskPtr.rwRef() = True;
284 // maskArray.putSlice(*maskPtr, IPosition(4,240,240,3,1));
285 // </srcblock>
286 //
287 // <h4>Example 5:</h4>
288 // In this example the data in the PagedArray will be accessed a row at
289 // a time while setting the cache size to different values. The comments
290 // illustrate the results when running on an Ultra 1/140 with 64MBytes
291 // of memory.
292 // <srcblock>
293 // PagedArray<Float> pa(IPosition(4,128,128,4,32));
294 // const IPosition latticeShape = pa.shape();
295 // cout << "The tile shape is:" << pa.tileShape() << endl;
296 // // The tile shape is:[32, 16, 4, 16]
297 //
298 // // Setup to access the PagedArray a row at a time
299 // const IPosition sliceShape(4,latticeShape(0), 1, 1, 1);
300 // const IPosition stride(4,1);
301 // Array<Float> row(sliceShape);
302 // IPosition start(4, 0);
303 //
304 // // Set the cache size to enough pixels for one tile only. This uses
305 // // 128kBytes of cache memory and takes 125 secs.
306 // pa.setCacheSizeInTiles (1);
307 // Timer clock;
308 // for (start(3) = 0; start(3) < latticeShape(3); start(3)++) {
309 // for (start(2) = 0; start(2) < latticeShape(2); start(2)++) {
310 // for (start(1) = 0; start(1) < latticeShape(1); start(1)++) {
311 // pa.getSlice(row, start, sliceShape, stride);
312 // }
313 // }
314 // }
315 // clock.show();
316 // pa.showCacheStatistics(cout);
317 // pa.clearCache();
318 //
319 // // Set the cache size to enough pixels for one row of tiles (ie. 4).
320 // // This uses 512 kBytes of cache memory and takes 10 secs.
321 // pa.setCacheSizeInTiles (4);
322 // clock.mark();
323 // for (start(3) = 0; start(3) < latticeShape(3); start(3)++) {
324 // for (start(2) = 0; start(2) < latticeShape(2); start(2)++) {
325 // for (start(1) = 0; start(1) < latticeShape(1); start(1)++) {
326 // pa.getSlice(row, start, sliceShape, stride);
327 // }
328 // }
329 // }
330 // clock.show();
331 // pa.showCacheStatistics(cout);
332 // pa.clearCache();
333 //
334 // // Set the cache size to enough pixels for one plane of tiles
335 // // (ie. 4*8). This uses 4 MBytes of cache memory and takes 2 secs.
336 // pa.setCacheSizeInTiles (4*8);
337 // clock.mark();
338 // for (start(3) = 0; start(3) < latticeShape(3); start(3)++) {
339 // for (start(2) = 0; start(2) < latticeShape(2); start(2)++) {
340 // for (start(1) = 0; start(1) < latticeShape(1); start(1)++) {
341 // pa.getSlice(row, start, sliceShape, stride);
342 // }
343 // }
344 // }
345 // clock.show();
346 // pa.showCacheStatistics(cout);
347 // pa.clearCache();
348 // </srcblock>
349 // </example>
350 
351 // <motivation>
352 // Arrays of data are sometimes much too large to hold in random access memory.
353 // PagedArrays, especially in combination with LatticeIterator,
354 // provide convenient access to such large data sets.
355 // </motivation>
356 
357 // <templating arg=T>
358 // <li> Due to storage in Tables, the templated type must be able to be
359 // stored in a Casacore Table. This restricts the template argument to all
360 // the common types Bool, Float, Double, Complex, String etc.) More details
361 // can be found in the RetypedArrayEngine class.
362 // </templating>
363 
364 // <todo asof="1997/04/14">
365 // <li> A better way of resizing PagedArrays
366 // </todo>
367 
368 // <linkfrom anchor="PagedArray" classes="Lattice ArrayLattice">
369 // <here>PagedArray</here> - a disk based Lattice.
370 // </linkfrom>
371 
372 
373 template <class T> class PagedArray : public Lattice<T>
374 {
375  //# Make members of parent class known.
376 public:
377  using Lattice<T>::ndim;
378 
379 public:
380  // The default constructor creates a PagedArray that is useless for just
381  // about everything, except that it can be assigned to with the assignment
382  // operator.
383  PagedArray();
384 
385  // Construct a new PagedArray with the specified shape. A new Table with
386  // the specified filename is constructed to hold the array. The Table will
387  // remain on disk after the PagedArray goes out of scope or is deleted.
388  PagedArray (const TiledShape& shape, const String& filename);
389 
390  // Construct a new PagedArray with the specified shape. A scratch Table is
391  // created in the current working directory to hold the array. This Table
392  // will be deleted automatically when the PagedArray goes out of scope or
393  // is deleted.
394  explicit PagedArray (const TiledShape& shape);
395 
396  // Construct a new PagedArray, with the specified shape, in the default
397  // row and column of the supplied Table.
398  PagedArray (const TiledShape& shape, Table& file);
399 
400  // Construct a new PagedArray, with the specified shape, in the specified
401  // row and column of the supplied Table.
402  PagedArray (const TiledShape& shape, Table& file,
403  const String& columnName, uInt rowNum);
404 
405  // Reconstruct from a pre-existing PagedArray in the default row and
406  // column of the supplied Table with the supplied filename.
407  explicit PagedArray (const String& filename);
408 
409  // Reconstruct from a pre-existing PagedArray in the default row and
410  // column of the supplied Table.
411  explicit PagedArray (Table& file);
412 
413  // Reconstruct from a pre-existing PagedArray in the specified row and
414  // column of the supplied Table.
415  PagedArray (Table& file, const String& columnName, uInt rowNum);
416 
417  // The copy constructor which uses reference semantics. Copying by value
418  // doesn't make sense, because it would require the creation of a
419  // temporary (but possibly huge) file on disk.
420  PagedArray (const PagedArray<T>& other);
421 
422  // The destructor flushes the PagedArrays contents to disk.
423  ~PagedArray();
424 
425  // The assignment operator with reference semantics. As with the copy
426  // constructor assigning by value does not make sense.
427  PagedArray<T>& operator= (const PagedArray<T>& other);
428 
429  // Make a copy of the object (reference semantics).
430  virtual Lattice<T>* clone() const;
431 
432  // A PagedArray is always persistent.
433  virtual Bool isPersistent() const;
434 
435  // A PagedArray is always paged to disk.
436  virtual Bool isPaged() const;
437 
438  // Is the PagedArray writable?
439  virtual Bool isWritable() const;
440 
441  // Returns the shape of the PagedArray.
442  virtual IPosition shape() const;
443 
444  // Return the current Table name. By default this includes the full path.
445  // The path preceeding the file name can be stripped off on request.
446  virtual String name (Bool stripPath=False) const;
447 
448  // Functions to resize the PagedArray. The old contents are lost. Usage of
449  // this function is NOT currently recommended (see the <linkto
450  // class="PagedArray:Advanced">More Details</linkto> section above).
451  void resize (const TiledShape& newShape);
452 
453  // Returns the current table name (ie. filename) of this PagedArray.
454  const String& tableName() const;
455 
456  // Return the current table object.
457  // <group>
458  Table& table();
459  const Table& table() const;
460  // </group>
461 
462  // Returns the current Table column name of this PagedArray.
463  const String& columnName() const;
464 
465  // Returns the default TableColumn name for a PagedArray.
466  static String defaultColumn();
467 
468  // Returns an accessor to the tiled storage manager.
469  const ROTiledStManAccessor& accessor() const;
470 
471  // Returns the current row number of this PagedArray.
472  uInt rowNumber() const;
473 
474  // Returns the default row number for a PagedArray.
475  static uInt defaultRow();
476 
477  // Returns the current tile shape for this PagedArray.
478  IPosition tileShape() const;
479 
480  // Returns the maximum recommended number of pixels for a cursor. This is
481  // the number of pixels in a tile.
482  virtual uInt advisedMaxPixels() const;
483 
484  // Set the maximum allowed cache size for all Arrays in this column of the
485  // Table. The actual value used may be smaller. A value of zero means
486  // that there is no maximum.
487  virtual void setMaximumCacheSize (uInt howManyPixels);
488 
489  // Return the maximum allowed cache size (in pixels) for all Arrays in
490  // this column of the Table. The actual cache size may be smaller. A
491  // value of zero means that no maximum is currently defined.
492  virtual uInt maximumCacheSize() const;
493 
494  // Set the actual cache size for this Array to be big enough for the
495  // indicated number of tiles. This cache is not shared with PagedArrays
496  // in other rows and is always clipped to be less than the maximum value
497  // set using the setMaximumCacheSize member function.
498  // Tiles are cached using a first in first out algorithm.
499  virtual void setCacheSizeInTiles (uInt howManyTiles);
500 
501  // Set the actual cache size for this Array to "fit" the indicated
502  // path. This cache is not shared with PagedArrays in other rows and is
503  // always less than the maximum value. The sliceShape is the cursor or
504  // slice that you will be requiring (with each call to
505  // {get,put}Slice). The windowStart and windowLength delimit the range of
506  // pixels that will ultimatly be accessed. The AxisPath is described in
507  // the documentation for the LatticeStepper class.
508  virtual void setCacheSizeFromPath (const IPosition& sliceShape,
509  const IPosition& windowStart,
510  const IPosition& windowLength,
511  const IPosition& axisPath);
512 
513  // Clears and frees up the tile cache. The maximum allowed cache size is
514  // unchanged from when <src>setMaximumCacheSize</src> was last called.
515  virtual void clearCache();
516 
517  // Generate a report on how the cache is doing. This is reset every
518  // time <src>clearCache</src> is called.
519  virtual void showCacheStatistics (ostream& os) const;
520 
521  // Return the value of the single element located at the argument
522  // IPosition.
523  // Note that <src>Lattice::operator()</src> can also be used.
524  virtual T getAt (const IPosition& where) const;
525 
526  // Put the value of a single element.
527  virtual void putAt (const T& value, const IPosition& where);
528 
529  // A function which checks for internal consistency. Returns False if
530  // something nasty has happened to the PagedArray. In that case
531  // it also throws an exception.
532  virtual Bool ok() const;
533 
534  // This function is used by the LatticeIterator class to generate an
535  // iterator of the correct type for a specified Lattice. Not recommended
536  // for general use.
537  virtual LatticeIterInterface<T>* makeIter (const LatticeNavigator& navigator,
538  Bool useRef) const;
539 
540  // Do the actual getting of an array of values.
541  virtual Bool doGetSlice (Array<T>& buffer, const Slicer& section);
542 
543  // Do the actual getting of an array of values.
544  virtual void doPutSlice (const Array<T>& sourceBuffer,
545  const IPosition& where,
546  const IPosition& stride);
547 
548  // Get the best cursor shape.
549  virtual IPosition doNiceCursorShape (uInt maxPixels) const;
550 
551  // Handle the (un)locking.
552  // <group>
553  virtual Bool lock (FileLocker::LockType, uInt nattempts);
554  virtual void unlock();
555  virtual Bool hasLock (FileLocker::LockType) const;
556  // </group>
557 
558  // Resynchronize the PagedArray object with the lattice file.
559  // This function is only useful if no read-locking is used, ie.
560  // if the table lock option is UserNoReadLocking or AutoNoReadLocking.
561  // In that cases the table system does not acquire a read-lock, thus
562  // does not synchronize itself automatically.
563  virtual void resync();
564 
565  // Flush the data (but do not unlock).
566  virtual void flush();
567 
568  // Temporarily close the lattice.
569  // It will be reopened automatically on the next access.
570  virtual void tempClose();
571 
572  // Explicitly reopen the temporarily closed lattice.
573  virtual void reopen();
574 
575 private:
576  // Set the data in the TableInfo file
577  void setTableType();
578  // make the ArrayColumn
579  void makeArray (const TiledShape& shape);
580  // Make a Table to hold this PagedArray
581  void makeTable (const String& filename, Table::TableOption option);
582  // The default comment for PagedArray Colums
583  static String defaultComment();
584  // Get the writable ArrayColumn object.
585  // It reopens the table for write if needed.
587  // Do the reopen of the table (if not open already).
588  // <group>
589  void doReopen() const;
590  void tempReopen() const;
591  // </group>
592 
593  mutable Table itsTable;
596  mutable Bool itsIsClosed;
603 };
604 
605 
606 template<class T>
608 {
609  if (itsIsClosed) {
610  doReopen();
611  }
612  if (!itsWritable) {
613  itsTable.reopenRW();
614  itsWritable = True;
615  }
616  return itsArray;
617 }
618 
619 template<class T>
621 {
622  doReopen();
623  return itsTable;
624 }
625 template<class T>
626 inline const Table& PagedArray<T>::table() const
627 {
628  doReopen();
629  return itsTable;
630 }
631 
632 template<class T>
633 inline const String& PagedArray<T>::columnName() const
634 {
635  return itsColumnName;
636 }
637 
638 template<class T>
640 {
641  return "PagedArray";
642 }
643 
644 template<class T>
646 {
647  return itsAccessor;
648 }
649 
650 template<class T>
652 {
653  return itsRowNumber;
654 }
655 
656 template<class T>
658 {
659  return 0;
660 }
661 
662 template<class T>
664 {
665  if (itsIsClosed) {
666  tempReopen();
667  }
668 }
669 
670 //# Declare extern templates for often used types.
671  extern template class PagedArray<Float>;
672  extern template class PagedArray<Complex>;
673 
674 
675 } //# NAMESPACE CASACORE - END
676 
677 #ifndef CASACORE_NO_AUTO_TEMPLATES
678 #include <casacore/lattices/Lattices/PagedArray.tcc>
679 #endif //# CASACORE_NO_AUTO_TEMPLATES
680 #endif
virtual void doPutSlice(const Array< T > &sourceBuffer, const IPosition &where, const IPosition &stride)
Do the actual getting of an array of values.
A Vector of integers, for indexing into Array&lt;T&gt; objects.
Definition: IPosition.h:118
virtual LatticeIterInterface< T > * makeIter(const LatticeNavigator &navigator, Bool useRef) const
This function is used by the LatticeIterator class to generate an iterator of the correct type for a ...
virtual void setCacheSizeFromPath(const IPosition &sliceShape, const IPosition &windowStart, const IPosition &windowLength, const IPosition &axisPath)
Set the actual cache size for this Array to &quot;fit&quot; the indicated path.
static uInt defaultRow()
Returns the default row number for a PagedArray.
Definition: PagedArray.h:657
virtual T getAt(const IPosition &where) const
Return the value of the single element located at the argument IPosition.
Main interface class to a read/write table.
Definition: Table.h:157
void makeArray(const TiledShape &shape)
make the ArrayColumn
void setTableType()
Set the data in the TableInfo file.
virtual Bool isWritable() const
Is the PagedArray writable?
static String defaultColumn()
Returns the default TableColumn name for a PagedArray.
Definition: PagedArray.h:639
virtual Bool ok() const
A function which checks for internal consistency.
virtual Bool doGetSlice(Array< T > &buffer, const Slicer &section)
Do the actual getting of an array of values.
virtual void unlock()
PagedArray()
The default constructor creates a PagedArray that is useless for just about everything, except that it can be assigned to with the assignment operator.
void makeTable(const String &filename, Table::TableOption option)
Make a Table to hold this PagedArray.
virtual Bool hasLock(FileLocker::LockType) const
void tempReopen() const
virtual IPosition shape() const
Returns the shape of the PagedArray.
uInt rowNumber() const
Returns the current row number of this PagedArray.
Definition: PagedArray.h:651
virtual void showCacheStatistics(ostream &os) const
Generate a report on how the cache is doing.
const String & columnName() const
Returns the current Table column name of this PagedArray.
Definition: PagedArray.h:633
virtual void setMaximumCacheSize(uInt howManyPixels)
Set the maximum allowed cache size for all Arrays in this column of the Table.
A base class for Lattice iterators.
Definition: ImageExpr.h:47
Define the shape and tile shape.
Definition: TiledShape.h:96
virtual uInt maximumCacheSize() const
Return the maximum allowed cache size (in pixels) for all Arrays in this column of the Table...
virtual Bool isPaged() const
A PagedArray is always paged to disk.
A templated, abstract base class for array-like objects.
Definition: Functional.h:37
virtual uInt advisedMaxPixels() const
Returns the maximum recommended number of pixels for a cursor.
virtual void tempClose()
Temporarily close the lattice.
virtual void setCacheSizeInTiles(uInt howManyTiles)
Set the actual cache size for this Array to be big enough for the indicated number of tiles...
const String & tableName() const
Returns the current table name (ie.
virtual IPosition doNiceCursorShape(uInt maxPixels) const
Get the best cursor shape.
Table & table()
Return the current table object.
Definition: PagedArray.h:620
PagedArray< T > & operator=(const PagedArray< T > &other)
The assignment operator with reference semantics.
ArrayColumn< T > & getRWArray()
Get the writable ArrayColumn object.
Definition: PagedArray.h:607
bool Bool
Define the standard types used by Casacore.
Definition: aipstype.h:42
Read and write access to an array table column with arbitrary data type.
Definition: CopyRecord.h:48
virtual void flush()
Flush the data (but do not unlock).
virtual String name(Bool stripPath=False) const
Return the current Table name.
virtual Bool isPersistent() const
A PagedArray is always persistent.
const Bool False
Definition: aipstype.h:44
Class to hold table lock options.
Definition: TableLock.h:68
ArrayColumn< T > itsArray
Definition: PagedArray.h:601
virtual void clearCache()
Clears and frees up the tile cache.
Specify which elements to extract from an n-dimensional array.
Definition: Slicer.h:288
IPosition tileShape() const
Returns the current tile shape for this PagedArray.
virtual void putAt(const T &value, const IPosition &where)
Put the value of a single element.
virtual Lattice< T > * clone() const
Make a copy of the object (reference semantics).
String: the storage and methods of handling collections of characters.
Definition: String.h:225
static String defaultComment()
The default comment for PagedArray Colums.
void doReopen() const
Do the reopen of the table (if not open already).
Definition: PagedArray.h:663
TableLock itsLockOpt
Definition: PagedArray.h:600
virtual void resync()
Resynchronize the PagedArray object with the lattice file.
LockType
Define the possible lock types.
Definition: FileLocker.h:95
Give access to some TiledStMan functions.
virtual void reopen()
Explicitly reopen the temporarily closed lattice.
const Bool True
Definition: aipstype.h:43
void resize(const TiledShape &newShape)
Functions to resize the PagedArray.
LatticeExprNode value(const LatticeExprNode &expr)
This function returns the value of the expression without a mask.
ROTiledStManAccessor itsAccessor
Definition: PagedArray.h:602
unsigned int uInt
Definition: aipstype.h:51
Abstract base class to steer lattice iterators.
const ROTiledStManAccessor & accessor() const
Returns an accessor to the tiled storage manager.
Definition: PagedArray.h:645
A Lattice that is read from or written to disk.
Definition: PagedArray.h:373
TableOption
Define the possible options how a table can be opened.
Definition: Table.h:172
~PagedArray()
The destructor flushes the PagedArrays contents to disk.
virtual Bool lock(FileLocker::LockType, uInt nattempts)
Handle the (un)locking.