Object/Class

io.xskipper.metadatastore.parquet

ParquetMetadataPath

Related Docs: class ParquetMetadataPath | package parquet

Permalink

object ParquetMetadataPath extends Logging with Serializable

Linear Supertypes
Serializable, Serializable, Logging, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. ParquetMetadataPath
  2. Serializable
  3. Serializable
  4. Logging
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  6. def convertMetaDataLocationToPath(mdLocation: String): Path

    Permalink

    Wrapper for stringToPath to return backend specific error message

    Wrapper for stringToPath to return backend specific error message

    mdLocation

    string to convert to org.apache.hadoop.fs.Path

    returns

    metadata hadoop fs path

    Exceptions thrown

    ParquetMetaDataStoreException if invalid path

  7. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  8. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  9. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  10. def getBaseMetadataPath(sparkSession: SparkSession, metaDataLocation: String, metadataType: String): Path

    Permalink

    returns

    Metadata base location

    Exceptions thrown

    ParquetMetaDataStoreException if invalid metaDataLocation or invalid metadataType

  11. def getBasePathFromDB(sparkSession: SparkSession, db: String): String

    Permalink

    Given a db name extracts the base path from the database parameters

  12. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  13. def getMetadataPath(basePath: String, uri: String): Path

    Permalink

    Constructs the metadata path for a given base path and a uri by concatenating sha256 digest of the uri to the base path

    Constructs the metadata path for a given base path and a uri by concatenating sha256 digest of the uri to the base path

    basePath

    the base path

    uri

    the uri of the dataset (can be either a path or table identifier)

    returns

    the path of the metadata

  14. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  15. def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  16. def initializeLogIfNecessary(isInterpreter: Boolean): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  17. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  18. def isTraceEnabled(): Boolean

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  19. def log: Logger

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  20. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  21. def logDebug(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  22. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  23. def logError(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  24. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  25. def logInfo(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  26. def logName: String

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  27. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  28. def logTrace(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  29. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  30. def logWarning(msg: ⇒ String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    Logging
  31. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  32. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  33. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  34. def resolveMetadataPath(sparkSession: SparkSession, uri: String, metaDataLocation: String, type: String): Path

    Permalink

    Resolves the metadata path according to the given parameters

    Resolves the metadata path according to the given parameters

    sparkSession

    SparkSession instance for accessing the catalog if needed

    uri

    the URI of the dataset

    metaDataLocation

    depending on the type can be: type = EXPLICIT_METADATA_LOCATION => an explicit location of the metadata type = EXPLICIT_BASE_PATH_LOCATION => an explicit definition of the base path of the metadata type = HIVE_TABLE_NAME => the name of the hive table that contains the table properties the exact path of the metadata. Note that if the parameter contains only the table name then the current database is used type = HIVE_DB_NAME => the name of the hive database that contains in the db properties the base path of the metadata

    returns

    the path of the metadata

  35. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  36. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  37. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  38. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  39. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from Logging

Inherited from AnyRef

Inherited from Any

Ungrouped