Value Members

1.  final def !=(arg0: Any): Boolean

   Definition Classes

   AnyRef → Any

2.  final def ##(): Int

   Definition Classes

   AnyRef → Any

3.  final def ==(arg0: Any): Boolean

   Definition Classes

   AnyRef → Any

4.  final def asInstanceOf[T0]: T0

   Definition Classes

   Any

5.  def clearActiveMetadataHandles(): Unit

   Clear all active MetadataHandles managed by this MetadataStoreManager

   Clear all active MetadataHandles managed by this MetadataStoreManager

   Definition Classes

   MetadataStoreManager

6.  def clearStats(): Unit

   Clears the stats for all active MetadataHandle instances.

   Clears the stats for all active MetadataHandle instances. Should be called before each query (or call clearActiveMetadataHandles) to make sure the aggregated stats are cleared

   Definition Classes

   MetadataStoreManager

7.  def clone(): AnyRef

   Attributes

   protected[java.lang]

   Definition Classes

   AnyRef

   Annotations

   @native() @throws( ... )

8.  def createMetadataHandle(sparkSession: SparkSession, tableIdentifier: String): MetadataHandle

   Creates a new instance MetadataHandle Used by getOrCreateMetadataHandle in order to create new instances

   Creates a new instance MetadataHandle Used by getOrCreateMetadataHandle in order to create new instances

   Definition Classes

   ParquetMetadataStoreManager → MetadataStoreManager

9.  final def eq(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

10.  def equals(arg0: Any): Boolean

    Definition Classes

    AnyRef → Any

11.  def finalize(): Unit

    Attributes

    protected[java.lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( classOf[java.lang.Throwable] )

12.  def getActiveMetadataHandles(): Map[String, MetadataHandle]

    Returns all active MetadataHandles managed by this MetadataStoreManager

    Returns all active MetadataHandles managed by this MetadataStoreManager

    Definition Classes

    MetadataStoreManager

13.  final def getClass(): Class[_]

    Definition Classes

    AnyRef → Any

    Annotations

    @native()

14.  def getDataSkippingFileFilterParams(tid: String, sparkSession: SparkSession, fileIndex: FileIndex): Map[String, String]

    Returns a map of parameters to be set by the io.xskipper.search.DataSkippingFileFilter on the MetadataHandle when using this MetadataStoreManager.

    Returns a map of parameters to be set by the io.xskipper.search.DataSkippingFileFilter on the MetadataHandle when using this MetadataStoreManager. This map can be used to pass specific parameters depending on the type of FileIndex that is being replaced. For example - in ParquetMetadataStoreManager we can specify that the metadata location should be inferred from the table/default db properties

    tid

    the table identifier for which the DataSkippingFileFilter will be created

    sparkSession

    the sparks session

    fileIndex

    the fileIndex that will be replaced by io.xskipper.search.DataSkippingFileFilter

    returns

    a map of parameters to be set on the MetadataStore when the DataSkippingFileFilter is created

    Definition Classes

    ParquetMetadataStoreManager → MetadataStoreManager

15.  def getLatestQueryAggregatedStats(): QueryIndexStatsResult

    Gets the aggregated latest query skipping stats for all active MetadataHandle instances.

    Gets the aggregated latest query skipping stats for all active MetadataHandle instances. In order to get reliable results it is assumed that either clearStats or clearActiveMetadataHandles was called before running the query.

    This is needed since the way we aggregate the skipping stats is by going over all active MetadataHandles of the MetadataStoreManager and aggregating their stats. When running multiple queries there could be a scenario in which the first query used dataset a and the second query didn't use it, therefore, when calling aggregatedStats for the second query the MetadataHandle for dataset a will be present as an active MetadataHandle therefore we need its stats to be cleared.

    returns

    QueryIndexStatsResult instance with the latest query aggregated stats

    Definition Classes

    MetadataStoreManager

16.  def getOrCreateMetadataHandle(sparkSession: SparkSession, tableIdentifier: String): MetadataHandle

    Gets an existing MetadataHandle or, if there is no existing one, creates a new one.

    Gets an existing MetadataHandle or, if there is no existing one, creates a new one.

    sparkSession

    the sparks session

    tableIdentifier

    the table identifier of the dataset for which the MetadataHandle instance is created

    returns

    a new instance of the MetadataHandle associated with this backend

    Definition Classes

    MetadataStoreManager

17.  def getParams(sparkSession: SparkSession): Map[String, String]
    returns

    a map representing relevant MetadataStoreManager parameters

    Definition Classes

    ParquetMetadataStoreManager → MetadataStoreManager

18.  def getType: MetadataStoreManagerType
    returns

    the MetadataStoreManagerType associated with this backend

    Definition Classes

    ParquetMetadataStoreManager → MetadataStoreManager

19.  def hashCode(): Int

    Definition Classes

    AnyRef → Any

    Annotations

    @native()

20.  def init(): Unit

    A unit function for setting up the MetadataStoreManager.

    A unit function for setting up the MetadataStoreManager. The setup is called once when the MetadataStoreManager is registered

    Definition Classes

    ParquetMetadataStoreManager → MetadataStoreManager

21.  def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean

    Attributes

    protected

    Definition Classes

    Logging

22.  def initializeLogIfNecessary(isInterpreter: Boolean): Unit

    Attributes

    protected

    Definition Classes

    Logging

23.  final def isInstanceOf[T0]: Boolean

    Definition Classes

    Any

24.  def isTraceEnabled(): Boolean

    Attributes

    protected

    Definition Classes

    Logging

25.  def listIndexedDatasets(sparkSession: SparkSession): Map[String, (MetadataVersionStatus.MetadataVersionStatus, Seq[Index])]
    returns

    Map of indexed datasets to their respective md version status and indexes

    Definition Classes

    ParquetMetadataStoreManager → MetadataStoreManager

26.  def log: Logger

    Attributes

    protected

    Definition Classes

    Logging

27.  def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Attributes

    protected

    Definition Classes

    Logging

28.  def logDebug(msg: ⇒ String): Unit

    Attributes

    protected

    Definition Classes

    Logging

29.  def logError(msg: ⇒ String, throwable: Throwable): Unit

    Attributes

    protected

    Definition Classes

    Logging

30.  def logError(msg: ⇒ String): Unit

    Attributes

    protected

    Definition Classes

    Logging

31.  def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Attributes

    protected

    Definition Classes

    Logging

32.  def logInfo(msg: ⇒ String): Unit

    Attributes

    protected

    Definition Classes

    Logging

33.  def logName: String

    Attributes

    protected

    Definition Classes

    Logging

34.  def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Attributes

    protected

    Definition Classes

    Logging

35.  def logTrace(msg: ⇒ String): Unit

    Attributes

    protected

    Definition Classes

    Logging

36.  def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Attributes

    protected

    Definition Classes

    Logging

37.  def logWarning(msg: ⇒ String): Unit

    Attributes

    protected

    Definition Classes

    Logging

38.  val metadataHandlesInstanceMap: ConcurrentHashMap[String, MetadataHandle]

    Definition Classes

    MetadataStoreManager

39.  final def ne(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

40.  final def notify(): Unit

    Definition Classes

    AnyRef

    Annotations

    @native()

41.  final def notifyAll(): Unit

    Definition Classes

    AnyRef

    Annotations

    @native()

42.  final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes

    AnyRef

43.  def toString(): String

    Definition Classes

    AnyRef → Any

44.  final def wait(): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

45.  final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

46.  final def wait(arg0: Long): Unit

    Definition Classes

    AnyRef

    Annotations

    @native() @throws( ... )