SQOOP-IMPORT-ALL-(1)

NAME
sqoop-import-all-tables - Import a set of table from an RDBMS to HDFS

SYNOPSIS

sqoop-import-all-tables <generic-options> <tool-options>

sqoop import-all-tables <generic-options> <tool-options>

DESCRIPTION

The import-all-tables tool imports a set of tables from an RDBMS to HDFS. Data from each table is stored in a separate directory in HDFS.

For the import-all-tables tool to be useful, the following conditions must be met:

    • Each table must have a single-column primary key.

    • You must intend to import all columns of each table.

    • You must not intend to use non-default splitting column, nor impose any conditions via a WHERE clause.

OPTIONS

The --connect option is required.

  Database connection and common options

--connect (jdbc-uri)
    Specify JDBC connect string (required)

--connection-manager (class-name)
    Specify connection manager class name (optional)

--driver (class-name)
    Manually specify JDBC driver class to use

--connection-param-file (filename)
    Optional properties file that provides connection parameters

--hadoop-home (dir)
    Override $HADOOP_HOME

--help
    Print usage instructions

--password (password)
    Set authentication password (Note: This is very insecure. You should use -P instead.)

-P
    Prompt for user password

--username (username)
    Set authentication username

--verbose
    Print more information while working

  Import control options

--as-avrodatafile
    Imports data to Avro Data Files

--as-sequencefile
    Imports data to SequenceFiles

--as-textfile
    Imports data as plain text (default)

--direct
    Use direct import fast path (mysql only)

--direct-split-size (n)
    Split the input stream every n bytes when importing in direct mode.

--inline-lob-limit (n)
    Set the maximum size for an inline LOB

-m, --num-mappers (n)
    Use n map tasks to import in parallel

--warehouse-dir (dir)
    Tables are uploaded to the HDFS path /warehouse/dir/(tablename)/

--compress, -z
    Uses gzip (or, alternatively, the codec specified by --compression-codec, if set) to compress data as it is written to HDFS

--compression-codec (codec)
    Uses the Hadoop codec class to compress data as it is written to HDFS.

  Output line formatting options

The delimiters used to separate fields and records can be specified on the command line, as can a quoting character and an escape character (for quoting delimiters inside a values). Data imported with --as-textfile will be formatted according to these parameters. Classes generated by Sqoop will encode this information, so using toString() from a data record stored --as-sequencefile will reproduce your specified formatting.

The (char) argument for each argument in this section can be specified either as a normal character (e.g., --fields-terminated-by ,) or via an escape sequence. Arguments of the form \0xhhh will be interpreted as a hexidecimal representation of a character with hex number hhh. Arguments of the form \0ooo will be treated as an octal representation of a character represented by octal number ooo. The special escapes \n, \r, \", \b, \t, and \\ act as they do inside Java strings. \0 will be treated as NUL. This will insert NUL characters between fields or lines (if used for --fields-terminated-by or --lines-terminated-by), or will disable enclosing/escaping if used for one of the --enclosed-by, --optionally-enclosed-by, or --escaped-by arguments.

The default delimiters are , for fields, \n for records, no quote character, and no escape character. Note that this can lead to ambiguous/unparsible records if you import database records containing commas or newlines in the field data. For unambiguous parsing, both must be enabled, e.g., via --mysql-delimiters.

--fields-terminated-by (char)
    Sets the field separator character

--lines-terminated-by (char)
    Sets the end-of-line character

--optionally-enclosed-by (char)
    Sets a field-enclosing character which may be used if a value contains delimiter characters.

--enclosed-by (char)
    Sets a field-enclosing character which will be used for all fields.

--escaped-by (char)
    Sets the escape character

--mysql-delimiters
    Uses MySQL’s default delimiter set:

    fields: , lines: \n escaped-by: \ optionally-enclosed-by: '

  Input line parsing options

Record classes generated by Sqoop include both a toString() method that formats output records, and a parse() method that interprets text based on an input delimiter set. The input delimiters default to the same ones chosen for output delimiters, but you can override these settings to support converting from one set of delimiters to another.

--input-fields-terminated-by (char)
    Sets the input field separator

--input-lines-terminated-by (char)
    Sets the input end-of-line char

--input-optionally-enclosed-by (char)
    Sets an input field-enclosing character

--input-enclosed-by (char)
    Sets a required input field encloser

--input-escaped-by (char)
    Sets the input escape character

  Hive options

--hive-home (dir)
    Override $HIVE_HOME

--hive-import
    If set, then import the table into Hive

--hive-overwrite
    Overwrites existing data in the hive table if it exists.

--create-hive-table
    If set, then the job will fail if the target hive table exits. By default this property is false.

--hive-table (table-name)
    When used with --hive-import, overrides the destination table name

--map-column-hive (mapping)
    Override default mapping for SQL types into Hive types for configured columns

  Code generation options

--bindir (dir)
    Output directory for compiled objects

--class-name (name)
    Sets the name of the class to generate. By default, classes are named after the table they represent. Using this parameters ignores --package-name.

--jar-file (filename)
    Disable code generation; use specified jar

--outdir (dir)
    Output directory for generated code

--package-name (package)
    Puts auto-generated classes in the named Java package

  Database-specific options

Additional arguments may be passed to the database manager after a lone -- on the command-line.

In MySQL direct mode, additional arguments are passed directly to mysqldump.

ENVIRONMENT

See sqoop(1)

AUTHOR

Sqoop Team
    Author.