SQOOP-EXPORT(1)

NAME
sqoop-export - Export records from HDFS to a table in an RDBMS

SYNOPSIS

sqoop-export <generic-options> <tool-options>

sqoop export <generic-options> <tool-options>

DESCRIPTION

The export tool exports a set of files from HDFS back to an RDBMS. The target table must already exist in the database. The input files are read and parsed into a set of records according to the user-specified delimiters.

The default operation is to transform these into a set of INSERT statements that inject the records into the database. In "update mode," Sqoop will generate UPDATE statements that replace existing records in the database.

OPTIONS

The --connect, --export-dir, and --table options are required.

  Database connection and common options

--connect (jdbc-uri)
    Specify JDBC connect string (required)

--connection-manager (class-name)
    Specify connection manager class name (optional)

--driver (class-name)
    Manually specify JDBC driver class to use

--connection-param-file (filename)
    Optional properties file that provides connection parameters

--hadoop-home (dir)
    Override $HADOOP_HOME

--help
    Print usage instructions

--password (password)
    Set authentication password (Note: This is very insecure. You should use -P instead.)

-P
    Prompt for user password

--username (username)
    Set authentication username

--verbose
    Print more information while working

  Export control options

--direct
    Use direct import fast path (MySQL)

--export-dir (dir)
    HDFS source path for the export

-m, --num-mappers (n)
    Use n map tasks to import in parallel

--table (table-name)
    The table to read (required)

--update-key (col-names)
    Anchor column to use for updates. More than one column names may be specified as a comma separated list of column names.

--update-mode (mode)
    Specify how updates are performed when new rows are found with non-matching keys in database. By default, "mode" is updateonly, in which case new rows are silently ignored. Alternatively, "mode" can be allowinsert, in which case new rows are inserted instead.

--input-null-string
    The string to be interpreted as null for string columns

--input-null-non-string
    The string to be interpreted as null for non-string columns

--staging-table
    The table in which data will be staged before being inserted in destination table.

--clear-staging-table
    Will result in deletion of any data that exists in the staging table.

--batch
    Use batch mode for underlying statement execution. This is useful, for example, for those databases that do not support multirow insert in a single statement yet.

  Input line parsing options

Record classes generated by Sqoop include both a toString() method that formats output records, and a parse() method that interprets text based on an input delimiter set. The input delimiters default to the same ones chosen for output delimiters, but you can override these settings to support converting from one set of delimiters to another.

--input-fields-terminated-by (char)
    Sets the input field separator

--input-lines-terminated-by (char)
    Sets the input end-of-line char

--input-optionally-enclosed-by (char)
    Sets an input field-enclosing character

--input-enclosed-by (char)
    Sets a required input field encloser

--input-escaped-by (char)
    Sets the input escape character

  Output line formatting options

The delimiters used to separate fields and records can be specified on the command line, as can a quoting character and an escape character (for quoting delimiters inside a values). Data imported with --as-textfile will be formatted according to these parameters. Classes generated by Sqoop will encode this information, so using toString() from a data record stored --as-sequencefile will reproduce your specified formatting.

The (char) argument for each argument in this section can be specified either as a normal character (e.g., --fields-terminated-by ,) or via an escape sequence. Arguments of the form \0xhhh will be interpreted as a hexidecimal representation of a character with hex number hhh. Arguments of the form \0ooo will be treated as an octal representation of a character represented by octal number ooo. The special escapes \n, \r, \", \b, \t, and \\ act as they do inside Java strings. \0 will be treated as NUL. This will insert NUL characters between fields or lines (if used for --fields-terminated-by or --lines-terminated-by), or will disable enclosing/escaping if used for one of the --enclosed-by, --optionally-enclosed-by, or --escaped-by arguments.

The default delimiters are , for fields, \n for records, no quote character, and no escape character. Note that this can lead to ambiguous/unparsible records if you import database records containing commas or newlines in the field data. For unambiguous parsing, both must be enabled, e.g., via --mysql-delimiters.

--fields-terminated-by (char)
    Sets the field separator character

--lines-terminated-by (char)
    Sets the end-of-line character

--optionally-enclosed-by (char)
    Sets a field-enclosing character which may be used if a value contains delimiter characters.

--enclosed-by (char)
    Sets a field-enclosing character which will be used for all fields.

--escaped-by (char)
    Sets the escape character

--mysql-delimiters
    Uses MySQL’s default delimiter set:

    fields: , lines: \n escaped-by: \ optionally-enclosed-by: '

  Code generation options

--bindir (dir)
    Output directory for compiled objects

--class-name (name)
    Sets the name of the class to generate. By default, classes are named after the table they represent. Using this parameters ignores --package-name.

--jar-file (filename)
    Disable code generation; use specified jar

--outdir (dir)
    Output directory for generated code

--package-name (package)
    Puts auto-generated classes in the named Java package

--map-column-java (mapping)
    Override default mapping from SQL type to Java type for configured columns

  Database-specific options

Additional arguments may be passed to the database manager after a lone -- on the command-line.

In MySQL direct mode, additional arguments are passed directly to mysqldump.

Note: When using MySQL direct mode, the MySQL bulk utilities mysqldump and mysqlimport should be available on the task nodes and present in the shell path of the task process.

ENVIRONMENT

See sqoop(1)

AUTHOR

Sqoop Team
    Author.