CSV Importer timing out

[baynature]

I have the CSV Importer working great on a small sample set, but when I try to do a larger set (430 Events), it only imports 13 records. At some point, I get a 502 Bad Gateway error on the site while doing the import. I’m doing this on our WPEngine staging server, our production site is baynature.org

[Julie Kuehl]

Hello baynature,

A 502 Gateway error is something that your hosting provider will have to help you with. I’m sure the folks at WP Engine can provide an assist.

If that’s not the case, let us know!

— Julie

[baynature]

Hi Julie,

Here’s the response I got from WP Engine:

“Thank you for your patience! Unfortunately, the import is timing max script execution limit, which is 60 seconds. This a limitation that cannot be changed, so my recommendation would be to reach out to the developer of the plugin and inquire if there is a way to batch import your .csv file.”

Well, the CSV importer is the batch import but I guess it’s timing out because of the size of the file (about 430 records). What do you think?

[Julie Kuehl]

Well buggers. It seems odd that number of records should cause a problem, but there it is. Given that constraint, could you break the CSV file apart and import it in sections, maybe 100 at at time?

— Julie

[baynature]

Thanks for your suggestion – I was thinking the same thing. It seems to work if I break it down to 50 at a time, which seems like an awfully small batch.

[baynature]

Actually, 50 at a time worked for a couple of imports, but now I am getting the 502 error even on that. So… any other ideas?

[Julie Kuehl]

Hi baynature,

 

First off, MY APOLOGIES for the delay in getting back to you. I’ve failed to meet out 24-hour promise and while there are lots of reasons, there’s no excuse. Please forgive me.

 

Better yet, let’s get you taken care of. Any idea how large the file is (in KB, MB, GB) that you are trying to import? When you spoke with WP Engine did they by chance provide any logs?

 

Could you possibly send me a copy of the CSV file, maybe both the full one and the small one, you are trying to import so I can give it a try?

 

— Julie

[baynature]

Hi Julie,

No worries! The holiday season can do that. I’ve asked WPEngine if there were error logs I can access, so I’ll get back to you on that. I’ve parsed the file down to a 288k file of 50 records, but I don’t seem to have an option to attach it here.

Thanks,
Laura

[Julie Kuehl]

Well, if WP Engine is unwilling to up the limit, about the only thing you can do is keep breaking it down into smaller chunks.

 

Another option might be to look for another hosting provider that would be more willing to work with you on that execution limit setting, which might also make your site load faster too. But that’s your decision.

 

If you’d like to email a copy of your CSV file(s) to pro (at) tri.be I can give them a try here. Please mention this thread and my name in your message.

 

 

– Julie

[baynature]

Hi Julie,
I tried to send an email to pro (at) tri.be, but I just get an email back saying “Unfortunately, our forum does not allow reply by email”. Is there another place I can email the CSV file?

I recently got word back from WP Engine as follows. I’m attaching the file to see if you will have the same conclusion. I don’t have any problem with the files I’m using, so I’m really curious what you find.

WPEngine:
Looks like there may be something wrong with either the CSV file or the plugin. I was messing around with file (I don’t have MS Excel, only free apps) and it seems the formatting is messed up or it is encoded improperly. I tried to duplicate it but I got an error trying to save it as a .CSV file. I got to the next page of the import with out having to manually change any settings in the import options. After these issue I really think the file is corrupted. I suggest making sure it is saved as plain text UTF-8 and try saving the file as a new .CSV file.

[Julie Kuehl]

Hi baynature,

Sorry that I’ve not gotten back to you on this. We’ve some new support tools and they don’t seem to like this thread. That being said we’re looking at adding an extra feature in our upcoming maintenance release that will essentially automatically batch the data to respond to timeout issues like we were doing manually. I’m hoping this will help. However, if the file is corrupted, that’s a different issue. Did you try the steps that WP Engine suggested?

 

— Julie

[baynature]

I did, and I don’t think the file is actually corrupted. I’m able to import all 400 records or so on my local environment with no problems, so it’s something about their set up.

When do you think the feature to resolve timeout issues might be implemented? I’m trying to decide if it’s worth my going back to my original idea of manually importing directly into the database or waiting until that feature is in place.

Thanks,
Laura

[Julie Kuehl]

While I can’t give you an exact date, I do believe that it will be addressed in an upcoming maintenance release. If you don’t need it immediately, it could be worth waiting for.

 

 

— Julie

[baynature]

Okay, I guess I’ll sit tight until then. Thanks for your help.

[Leah]

Hi baynature,

I’m happy to report that we have fixed this in our upcoming version 3.4. Keep an eye on your Plugins page for an update message! Thanks for your patience while we worked on this.

Best,
Leah

[Author]

Posts