Class to view a concatenation of tables as a single table. More...

#include <ConcatTable.h>

Inheritance diagram for casacore::ConcatTable:

Public Member Functions
	ConcatTable (const Block< Table > &tables, const Block< String > &subTables, const String &subDirName)
	Create a virtual table as the concatenation of the given tables. More...

	ConcatTable (const Block< String > &tableNames, const Block< String > &subTables, const String &subDirName, int option, const TableLock &lockOptions, const TSMOption &tsmOption)

	ConcatTable (AipsIO &, const String &name, rownr_t nrrow, int option, const TableLock &lockOptions, const TSMOption &tsmOption)
	Create a concat table out of a file (written by writeConcatTable). More...

virtual	~ConcatTable ()
	The destructor flushes (i.e. More...

virtual void	getPartNames (Block< String > &names, Bool recursive) const
	Get the names of the tables this table consists of. More...

virtual void	reopenRW ()
	Try to reopen the table (the underlying ones) for read/write access. More...

virtual Bool	asBigEndian () const
	Is the table stored in big or little endian format? It returns the endianness of the first underlying table. More...

virtual const StorageOption &	storageOption () const
	Get the storage option used for the table. More...

virtual Bool	isMultiUsed (Bool checkSubTable) const
	Is the table in use (i.e. More...

virtual const TableLock &	lockOptions () const
	Get the locking info. More...

virtual void	mergeLock (const TableLock &lockOptions)
	Merge the given lock info with the existing one. More...

virtual Bool	hasLock (FileLocker::LockType) const
	Has this process the read or write lock, thus can the table be read or written safely? More...

virtual Bool	lock (FileLocker::LockType, uInt nattempts)
	Try to lock the table for read or write access. More...

virtual void	unlock ()
	Unlock the table. More...

virtual void	flush (Bool fsync, Bool recursive)
	Flush the table, i.e. More...

virtual void	resync ()
	Resync the Table object with the table files. More...

virtual uInt	getModifyCounter () const
	Get the modify counter. More...

virtual Bool	isWritable () const
	Test if all underlying tables are opened as writable. More...

void	getConcat (AipsIO &, int option, const TableLock &lockOptions, const TSMOption &tsmOption)
	Read a concat table from a file. More...

virtual void	copy (const String &newName, int tableOption) const
	This is doing a shallow copy. More...

virtual void	deepCopy (const String &newName, const Record &dataManagerInfo, const StorageOption &, int tableOption, Bool, int endianFormat, Bool noRows) const
	Copy the table and all its subtables. More...

virtual int	tableType () const
	It returns the type of the parent table. More...

virtual TableDesc	actualTableDesc () const
	Get the actual table description. More...

virtual Record	dataManagerInfo () const
	Get the data manager info (of the first underlying table). More...

virtual TableRecord &	keywordSet ()
	Get readonly access to the table keyword set. More...

virtual TableRecord &	rwKeywordSet ()
	Get read/write access to the table keyword set. More...

virtual BaseColumn *	getColumn (uInt columnIndex) const
	Get a column object using its index. More...

virtual BaseColumn *	getColumn (const String &columnName) const
	Get a column object using its name. More...

virtual Bool	canRemoveRow () const
	Test if it is possible to remove a row from this table (no). More...

virtual void	removeRow (rownr_t rownr)
	Remove the given row. More...

virtual Bool	canRemoveColumn (const Vector< String > &columnNames) const
	Test if columns can be removed (no). More...

virtual void	addColumn (const ColumnDesc &columnDesc, Bool addToParent)
	Add one or more columns to the table. More...

virtual void	addColumn (const ColumnDesc &columnDesc, const String &dataManager, Bool byName, Bool addToParent)

virtual void	addColumn (const ColumnDesc &columnDesc, const DataManager &dataManager, Bool addToParent)

virtual void	addColumn (const TableDesc &tableDesc, const DataManager &dataManager, Bool addToParent)

virtual void	removeColumn (const Vector< String > &columnNames)
	Remove a column. More...

virtual Bool	canRenameColumn (const String &columnName) const
	Test if a column can be renamed (no). More...

virtual void	renameColumn (const String &newName, const String &oldName)
	Rename a column. More...

virtual void	renameHypercolumn (const String &newName, const String &oldName)
	Rename a hypercolumn. More...

virtual DataManager *	findDataManager (const String &name, Bool byColumn) const
	Find the data manager with the given name or for the given column. More...

const ConcatRows &	rows () const
	Get the rows object. More...

Block< BaseColumn * >	getRefColumns (const String &columnName)
	Get the column objects in the referenced tables. More...

Table	asTable ()
	Create a (temporary) Table object from it. More...

Public Member Functions inherited from casacore::BaseTable
	BaseTable (const String &tableName, int tableOption, rownr_t nrrow)
	Initialize the object. More...

	BaseTable (MPI_Comm mpiComm, const String &tableName, int tableOption, rownr_t nrrow)
	MPI version of the constructor. More...

void	BaseTableCommon (const String &tableName, int tableOption, rownr_t nrrow)
	Common code shared by the MPI constructor and non-MPI constructor. More...

virtual	~BaseTable ()

void	link ()
	Link to this BaseTable object (i.e. More...

virtual Bool	isNull () const
	Is the table a null table? By default it is not. More...

virtual void	setTableChanged ()
	Set the table to being changed. More...

void	doNotWrite ()
	Do not write the table (used in in case of exceptions). More...

Bool	isColumnWritable (const String &columnName) const
	Test if the given column is writable. More...

Bool	isColumnWritable (uInt columnIndex) const

Bool	isColumnStored (const String &columnName) const
	Test if the given column is stored (otherwise it is virtual). More...

Bool	isColumnStored (uInt columnIndex) const

const String &	tableName () const
	Get the table name. More...

virtual void	rename (const String &newName, int tableOption)
	Rename the table. More...

int	tableOption () const
	Get the table option. More...

void	markForDelete (Bool callback, const String &oldName)
	Mark the table for delete. More...

void	unmarkForDelete (Bool callback, const String &oldName)
	Unmark the table for delete. More...

Bool	isMarkedForDelete () const
	Test if the table is marked for delete. More...

const TableDesc &	tableDesc () const
	Get the table description. More...

void	showStructure (std::ostream &, Bool showDataMan, Bool showColumns, Bool showSubTables, Bool sortColumns, Bool cOrder)
	Show the table structure (implementation of Table::showStructure). More...

TableInfo &	tableInfo ()
	Get access to the TableInfo object. More...

virtual void	flushTableInfo ()
	Write the TableInfo object. More...

rownr_t	nrow () const
	Get number of rows. More...

virtual Bool	canAddRow () const
	Test if it is possible to add a row to this table. More...

virtual void	addRow (rownr_t nrrow=1, Bool initialize=True)
	Add one or more rows and possibly initialize them. More...

void	removeRow (const Vector< rownr_t > &rownrs)

void	removeRow (const Vector< uInt > &rownrs)

BaseTable *	select (const TableExprNode &, rownr_t maxRow, rownr_t offset)
	Select rows using the given expression (which can be null). More...

BaseTable *	select (rownr_t maxRow, rownr_t offset)
	Select maxRow rows and skip first offset rows. More...

BaseTable *	select (const Vector< rownr_t > &rownrs)
	Select rows using a vector of row numbers. More...

BaseTable *	select (const Block< Bool > &mask)
	Select rows using a mask block. More...

BaseTable *	project (const Block< String > &columnNames)
	Project the given columns (i.e. More...

BaseTable *	tabAnd (BaseTable *)
	Do logical operations on a table. More...

BaseTable *	tabOr (BaseTable *)
	union with another table More...

BaseTable *	tabSub (BaseTable *)
	subtract another table More...

BaseTable *	tabXor (BaseTable *)
	xor with another table More...

BaseTable *	tabNot ()
	take complement More...

BaseTable *	sort (const Block< String > &columnNames, const Block< CountedPtr< BaseCompare > > &compareObjects, const Block< Int > &sortOrder, int sortOption, std::shared_ptr< Vector< rownr_t >> sortIterBoundaries=nullptr, std::shared_ptr< Vector< size_t >> sortIterKeyIdxChange=nullptr)
	Sort a table on one or more columns of scalars. More...

BaseTableIterator *	makeIterator (const Block< String > &columnNames, const Block< CountedPtr< BaseCompare > > &, const Block< Int > &orders, int option, bool cacheIterationBoundaries=false)
	Create an iterator. More...

void	addColumns (const TableDesc &tableDesc, const Record &dmInfo, Bool addToParent)
	Add one or more columns to the table. More...

Bool	checkRemoveColumn (const Vector< String > &columnNames, Bool throwException) const
	Check if the set of columns can be removed. More...

virtual Vector< rownr_t >	rowNumbers () const
	Get a vector of row numbers. More...

virtual BaseTable *	root ()
	Get pointer to root table (i.e. More...

virtual Bool	rowOrder () const
	Tell if the table is in row order. More...

virtual Vector< rownr_t > *	rowStorage ()
	By the default the table cannot return the storage of rownrs. More...

virtual Bool	adjustRownrs (rownr_t nrrow, Vector< rownr_t > &rownrs, Bool determineOrder) const
	Adjust the row numbers to be the actual row numbers in the root table. More...

virtual BaseTable *	doSort (PtrBlock< BaseColumn * > &, const Block< CountedPtr< BaseCompare > > &, const Block< Int > &sortOrder, int sortOption, std::shared_ptr< Vector< rownr_t >> sortIterBoundaries, std::shared_ptr< Vector< size_t >> sortIterKeyIdxChange)
	Do the actual sort. More...

RefTable *	makeRefTable (Bool rowOrder, rownr_t initialNrrow)
	Create a RefTable object. More...

void	checkRowNumber (rownr_t rownr) const
	Check if the row number is valid. More...

int	traceId () const
	Get the table's trace-id. More...

Static Public Member Functions
static void	getLayout (TableDesc &desc, AipsIO &ios)
	Return the layout of a table (i.e. More...

Static Public Member Functions inherited from casacore::BaseTable
static void	unlink (BaseTable *)
	Unlink from a BaseTable. More...

static TableInfo	tableInfo (const String &tableName)
	Get the table info of the table with the given name. More...

Private Member Functions
	ConcatTable (const ConcatTable &)
	Copy constructor is forbidden, because copying a table requires some more knowledge (like table name of result). More...

ConcatTable &	operator= (const ConcatTable &)
	Assignment is forbidden, because copying a table requires some more knowledge (like table name of result). More...

void	showStructureExtra (std::ostream &) const
	Show the extra table structure info (names of used tables). More...

void	openTables (const Block< String > &tableNames, int option, const TableLock &lockOptions, const TSMOption &tsmOption)
	Open all tables in the required way. More...

void	initialize ()
	Initialize. More...

void	setup (BaseTable *btp, const Vector< String > &columnNames)
	Setup the main parts of the object. More...

void	addInfo ()
	Add lines containing the concatenated tables to the info. More...

void	makeConcatCol ()
	Create the ConcatColumn objects for all columns in the description. More...

void	handleSubTables ()
	Handle the subtales that have to be concatenated. More...

void	writeConcatTable (Bool fsync)
	Write a reference table. More...

void	checkAddColumn (const String &name, Bool addToParent)
	Check if the column can be added, thus does not exist yet. More...

void	addConcatCol (const ColumnDesc &cd)
	Add a column, with internal bookeeping (columns map). More...

void	addConcatCol (const TableDesc &tdesc)
	Add multiple columns, with internal bookeeping (columns map). More...

Private Attributes
Block< String >	subTableNames_p

String	subDirName_p

Block< Table >	tables_p

std::map< String, ConcatColumn * >	colMap_p

TableRecord	keywordSet_p

Bool	changed_p

ConcatRows	rows_p

Additional Inherited Members
Protected Member Functions inherited from casacore::BaseTable
void	scratchCallback (Bool isScratch, const String &oldName) const
	Do the callback for scratch tables (if callback is set). More...

Bool	makeTableDir ()
	Create the table directory when needed (and possible). More...

void	trueDeepCopy (const String &newName, const Record &dataManagerInfo, const StorageOption &, int tableOption, int endianFormat, Bool noRows) const
	Make a true deep copy of the table. More...

void	prepareCopyRename (const String &newName, int tableOption) const
	Prepare for copying or renaming a table. More...

virtual void	renameSubTables (const String &newName, const String &oldName)
	Rename the subtables (used by rename function). More...

void	throwIfTableExists ()
	Check if the table already exists. More...

Bool	openedForWrite () const
	Test if the table is opened for write. More...

void	writeStart (AipsIO &, Bool bigEndian)
	Start writing a table. More...

void	writeEnd (AipsIO &)
	End writing a table. More...

Bool	shouldNotWrite () const
	Should the table be written. More...

void	getTableInfo ()
	Read the TableInfo object. More...

Protected Attributes inherited from casacore::BaseTable
uInt	nrlink_p

rownr_t	nrrow_p

rownr_t	nrrowToAdd_p

CountedPtr< TableDesc >	tdescPtr_p

String	name_p

int	option_p

Bool	noWrite_p

Bool	delete_p

TableInfo	info_p

Bool	madeDir_p

int	itsTraceId

Detailed Description

Class to view a concatenation of tables as a single table.

Intended use:

Internal

Review Status

Reviewed By:: UNKNOWN

Prerequisite

Etymology

ConcatTable represents the concatenation of one or more tables.

Synopsis

ConcatTable is used to virtually concatenate one or more tables. Those tables must have the same description.

It acts to the user as a normal table. All gets and puts are handled by ConcatColumn which directs them to the referenced columns while (if needed) converting the given row number to the row number in the referenced tables. For that purpose ConcatTable keeps the number of rows in the referenced tables.
Note: Currently it cannot handle changes in the number of rows in the underlying tables;

It is possible to specify the keyword names of the subtables that have to be concatenated as well. The other subtables are assumed to be identical for all tables, so only the subtable of the first table is used.

The ConcatTable maintains its own keyword set, which is initially a copy of the keyword set of the first table. It replaces the keywords of the subtables to be concatenated. The keyword set is not persistent. One can add or change keywords, but these changes are not kept when the ConcatTable object is made persistent.

Motivation

Sometimes a very large MeasurementSet is split into multiple smaller ones using the time axis. Using ConcatTable they can still be viewed as a single MS. The SYSCAL subtable is split in time as well, thus it has to be possible to concatenate that one as well.
Note: An MS split in subband could be concatenated as well provided that at least the first part contains the full SPECTRAL_WINDOW subtable and that unique SPWids are used;

To Do

Maybe not allocating the row number vector for a projection. This saves space and time, but each rownr conversion will take a bit more time because it has to test if there is a vector.
Maybe maintain a Vector<String> telling on which columns the table is ordered. This may speed up selection, but it is hard to check if the order is changed by a put.
Allow to remove a row or column from the ConcatTable
Allow to rename a column in the ConcatTable
Maybe implement doSort one time for a more efficient sort. (now everything is handled by BaseTable).

Definition at line 118 of file ConcatTable.h.

Constructor & Destructor Documentation

casacore::ConcatTable::ConcatTable	(	const Block< Table > &	tables,
		const Block< String > &	subTables,
		const String &	subDirName
	)

Create a virtual table as the concatenation of the given tables.

It checks if the table descriptions of the tables are the same. Subtables with the given names will be concatenated as well. It is assumed that the other subtables are the same for all tables, so the ones of the first table are used.
The option can be Table::Old or Table::Update.
If a non-empty subdirectory name is given, the tables will be moved to that subdirectory when the concatenated table is written (by writeConcatTable).

casacore::ConcatTable::ConcatTable	(	const Block< String > &	tableNames,
		const Block< String > &	subTables,
		const String &	subDirName,
		int	option,
		const TableLock &	lockOptions,
		const TSMOption &	tsmOption
	)

casacore::ConcatTable::ConcatTable	(	AipsIO &	,
		const String &	name,
		rownr_t	nrrow,
		int	option,
		const TableLock &	lockOptions,
		const TSMOption &	tsmOption
	)

Create a concat table out of a file (written by writeConcatTable).

The referenced tables will also be opened (if not stored in the cache).

virtual casacore::ConcatTable::~ConcatTable ( )

virtual

The destructor flushes (i.e.

writes) the table if it is opened for output and not marked for delete.

casacore::ConcatTable::ConcatTable ( const ConcatTable & )

private

Copy constructor is forbidden, because copying a table requires some more knowledge (like table name of result).

Declaring it private, makes it unusable.

Member Function Documentation

virtual TableDesc casacore::ConcatTable::actualTableDesc ( ) const

virtual

Get the actual table description.

void casacore::ConcatTable::getConcat	(	AipsIO &	,
		int	option,
		const TableLock &	lockOptions,
		const TSMOption &	tsmOption
	)

Public Member Functions

Static Public Member Functions

Private Member Functions

Private Attributes

Additional Inherited Members