Recently I came across the task where I need to import large csv records into mysql database. It’s quite tough because your phpmyadmin will not allow you to import large file. That csv contains around 10 million records. First thing I tried to open that file, but it shows “File not loaded completely”. The reason you get this error because excel can only handle 65,536 lines per sheet and it will not open remaining records. I even tried to import it directly via phpmyadmin but maximum time limit exceeded, then next thing I increased php max execution time but it was taking much time to process that file.
So you can solve this problem simply by splitting the large csv file into smaller fragments. By doing that script will consume time limit.
You can simply download FXFisherman’s CSV Spliter program and run the program in your windows system. Installation no longer required.
Once you run you will see this screen below-
Now you may browse your csv file which you want to split, specify how many numbers of lines you want to have per csv and also another way you can simply specify max number of pieces. Now simply click “Split Now!”.
It will start splitting your csv into number of pieces. When process is completed you should see screen like below.
No comments:
Post a Comment