How to import data from text file to mysql database

I have a 350MB file named text_file.txt containing this tab delimited data:

345868230   1646198120  1531283146  Keyword_1531283146  1.55    252910000
745345566   1646198120  1539847239  another_1531276364  2.75    987831000
...

MySQL Database name: Xml_Date

Database table: PerformanceReport

I have already created the table with all the destination fields.

I want to import this text file data into a MySQL. I googled and found some commands like LOAD DATA INFILE and quite confused on how to use it.

How can I import this text file data?

Contents hide

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

It should be as simple as…

LOAD DATA INFILE '/tmp/mydata.txt' INTO TABLE PerformanceReport;

By default LOAD DATA INFILE uses tab delimited, one row per line, so should take it in just fine.

Method 2

Walkthrough on using MySQL’s LOAD DATA command:

Create your table:

CREATE TABLE foo(myid INT, mymessage VARCHAR(255), mydecimal DECIMAL(8,4));

Create your tab delimited file (note there are tabs between the columns):

1   Heart disease kills     1.2
2   one out of every two    2.3
3   people in America.      4.5

Use the load data command:
```
LOAD DATA LOCAL INFILE '/tmp/foo.txt' 
INTO TABLE foo COLUMNS TERMINATED BY 't';
```
If you get a warning that this command can’t be run, then you have to enable the --local-infile=1 parameter described here: How can I correct MySQL Load Error

The rows get inserted:

Query OK, 3 rows affected (0.00 sec)
Records: 3  Deleted: 0  Skipped: 0  Warnings: 0

Check if it worked:

mysql> select * from foo;
+------+----------------------+-----------+
| myid | mymessage            | mydecimal |
+------+----------------------+-----------+
|    1 | Heart disease kills  |    1.2000 |
|    2 | one out of every two |    2.3000 |
|    3 | people in America.   |    4.5000 |
+------+----------------------+-----------+
3 rows in set (0.00 sec)

How to specify which columns to load your text file columns into:

Like this:

LOAD DATA LOCAL INFILE '/tmp/foo.txt' INTO TABLE foo
FIELDS TERMINATED BY 't' LINES TERMINATED BY 'n'
(@col1,@col2,@col3) set <a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="96fbeffff2abd6f5f9faa7">[email protected]</a>,<a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="deb3a7babbbdb7b3bfb2e39ebdb1b2ed">[email protected]</a>;

The file contents get put into variables @col1, @col2, @col3. myid gets column 1, and mydecimal gets column 3. If this were run, it would omit the second row:

mysql> select * from foo;
+------+-----------+-----------+
| myid | mymessage | mydecimal |
+------+-----------+-----------+
|    1 | NULL      |    1.2000 |
|    2 | NULL      |    2.3000 |
|    3 | NULL      |    4.5000 |
+------+-----------+-----------+
3 rows in set (0.00 sec)

Method 3

If your table is separated by others than tabs, you should specify it like…

LOAD DATA LOCAL 
    INFILE '/tmp/mydata.txt' INTO TABLE PerformanceReport 
    COLUMNS TERMINATED BY 't'  ## This should be your delimiter
    OPTIONALLY ENCLOSED BY '"'; ## ...and if text is enclosed, specify here

Method 4

The LOAD DATA INFILE statement reads rows from a text file into a table at a very high speed.

LOAD DATA INFILE '/tmp/test.txt' 
INTO TABLE test
FIELDS TERMINATED BY ','
LINES STARTING BY 'xxx';

If the data file looks like this:

xxx"abc",1
something xxx"def",2
"ghi",3

The resulting rows will be (“abc”,1) and (“def”,2). The third row in the file is skipped because it does not contain the prefix.

LOAD DATA INFILE 'data.txt'
INTO TABLE tbl_name
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY 'rn'

You can also load data files by using the mysqlimport utility; it operates by sending a LOAD DATA INFILE statement to the server

mysqlimport -u root -ptmppassword --local test employee.txt

test.employee: Records: 3  Deleted: 0  Skipped: 0  Warnings: 0

Method 5

You should set the option:

local-infile=1

into your [mysql] entry of my.cnf file or call mysql client with the –local-infile option:

mysql --local-infile -uroot -pyourpwd yourdbname

You have to be sure that the same parameter is defined into your [mysqld] section too to enable the “local infile” feature server side.

It’s a security restriction.

LOAD DATA LOCAL INFILE '/softwares/data/data.csv' INTO TABLE tableName;

Method 6

LOAD DATA INFILE '/home/userlap/data2/worldcitiespop.txt' INTO TABLE cc FIELDS TERMINATED BY ','LINES TERMINATED BY 'r n' IGNORE 1 LINES;

IGNORE 1 LINES to skip over an initial header line containing column names
FIELDS TERMINATED BY ‘,’ is to read the comma-delimited file
If you have generated the text file on a Windows system, you might have to use LINES TERMINATED BY ‘rn’ to read the file properly, because Windows programs typically use two characters as a line terminator. Some programs, such as WordPad, might use r as a line terminator when writing files. To read such files, use LINES TERMINATED BY ‘r’.

Method 7

For me just adding the “LOCAL” Keyword did the trick, please see the attached image for easier solution.

My attached image contains both use cases:

(a) Where I was getting this error.
(b) Where error was resolved by just adding “Local” keyword.

Method 8

Make Sure your Local-Infile variable is set to True (ON)

mysql> show global variables like 'local_infile';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| local_infile  | OFF   |
+---------------+-------+
1 row in set (0.04 sec)

mysql> set global local_infile=true;
Query OK, 0 rows affected (0.01 sec)

Find the correct path to store the txt files for loading in SQL tables

mysql>  SELECT @@GLOBAL.secure_file_priv;
+------------------------------------------------+
| @@GLOBAL.secure_file_priv                      |
+------------------------------------------------+
| C:ProgramDataMySQLMySQL Server 8.0Uploads |
+------------------------------------------------+
1 row in set (0.00 sec)

Load using data infile from the path (Use backward slashes in path)

mysql>  load data infile 'C:/ProgramData/MySQL/MySQL Server
8.0/Uploads/text_file.txt' into table TABLE_NAME fields terminated by 't' lines terminated by 'n';

Method 9

1. if it’s tab delimited txt file:

LOAD DATA LOCAL INFILE ‘D:/MySQL/event.txt’ INTO TABLE event

LINES TERMINATED BY ‘rn’;

2. otherwise:

LOAD DATA LOCAL INFILE ‘D:/MySQL/event.txt’ INTO TABLE event

FIELDS TERMINATED BY ‘x’ (here x could be comma ‘,’, tab ‘t’, semicolon ‘;’, space ‘ ‘)

LINES TERMINATED BY ‘rn’;

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0

0 0 votes

Article Rating