Urgent, Help Needed with File Downloads

Posts: 25
Joined: 09/24/2008

I have exhausted all search options.

I have Ubercart 5.x-1.0-beta1 running on a Drupal site that sells file audio and video downloads. There are a couple of problems here:

Users are not getting their email with the file download links. I have Workflow-ng set up to change the order status from Pending to Completed and the email is NOT being sent when the status is changed. Only if I go into the actual order, change the status to something like "Processing", then back to "Complete" does the user get access to the files and the email with links.

Second major problem is that users' downloads are failing 100-200 megabytes into a 600+ megabyte download. I am getting this error message in the Drupal log:

Lost connection to MySQL server during query query: UPDATE uc_file_users SET `accessed` = 1, `addresses` = 'a:1:{i:0;s:14:\"97.100.148.163\";}', `key` = '69c283e515500b2b34a2665b922d8bf1' WHERE `fid` = 14 AND `uid` = 180 AND `key` = '7ea6817fc10a82ba7f2897b97da79509' in /home/18879/domains/supernovarecords.net/html/site/includes/database.mysql.inc on line 172.

I have increased max_execution_time to 1200, and memory_limit is 99M in my php.ini file

Any tips would be greatly appreciated. The site is live and customers are getting impatient.

Posts: 5625
Joined: 08/07/2007
AdministratorHead Code Monkey - I eat bugs.

Unfortunately, it sounds like your Workflow-ng problem was one fixed in a later version and torgosPizza (who manages the file download sales for http://www.rifftrax.com) just gave us a patch for better code for file downloads that will be in the 1.5 release. Any chance you can update your site?

Posts: 25
Joined: 09/24/2008

Ryan,

I will update to the latest version of ubercart.

Any clue on the download failing?

Posts: 1377
Joined: 08/14/2007
Bug FinderEarly adopter... addicted to alphas.Getting busy with the Ubercode.

Is your database server not the same server as the one hosting your Drupal installation? I can't say I've ever seen that error before.. why you'd be losing the connection to the database during that query, which is relatively small, I think.

--

"Pain don't hurt." - Dalton

Mike Nelson's RiffTrax! www.rifftrax.com

Posts: 25
Joined: 09/24/2008

torgosPizza wrote:
Is your database server not the same server as the one hosting your Drupal installation? I can't say I've ever seen that error before.. why you'd be losing the connection to the database during that query, which is relatively small, I think.

I am using MediaTemple's Grid Service, and after a 5 minute search, I cannot answer with absolute certainty that they are the same server / machine.

The files being downloaded are on the same server under a sub-domain.

Posts: 1377
Joined: 08/14/2007
Bug FinderEarly adopter... addicted to alphas.Getting busy with the Ubercode.

I don't have any experience with them. I know UC users who do, but not sure if they use File Downloads or not. I might consider running the issue by mt's support guys, tell them what you are doing and what the error is, and perhaps they can narrow down the issue. I don't think it's a PHP limit per se, but probably something to do with MySQL hanging for some reason. Perhaps tables are being locked and staying locked when they shouldn't be - do you have a lot of users connecting at once?

--

"Pain don't hurt." - Dalton

Mike Nelson's RiffTrax! www.rifftrax.com

Posts: 25
Joined: 09/24/2008

torgosPizza wrote:
I don't have any experience with them. I know UC users who do, but not sure if they use File Downloads or not. I might consider running the issue by mt's support guys, tell them what you are doing and what the error is, and perhaps they can narrow down the issue. I don't think it's a PHP limit per se, but probably something to do with MySQL hanging for some reason. Perhaps tables are being locked and staying locked when they shouldn't be - do you have a lot of users connecting at once?

I asked MT's tech support and this is what i got:

MySQL and the hosting servers are different. MySQL holds the information specific to your site while the files are being ran from the hosted server. The server connects to the database server and performs queries which are operations to either read, write, or modify information in the database.

Actual downloads are kept on the hosting server so it is very unlikely that the large downloads are affected by any issues with the MySQL server.

Obviously they included some basic stuff in that response.

Posts: 1377
Joined: 08/14/2007
Bug FinderEarly adopter... addicted to alphas.Getting busy with the Ubercode.

Yeah, seems strange. I would update to the latest bazaar version of the uc_file.module if you can. I don't think that'll have an effect either way on the "lost database connection" error you experienced. There's the chance that this is an issue separate from the downloads failing, but they could be related somehow.

--

"Pain don't hurt." - Dalton

Mike Nelson's RiffTrax! www.rifftrax.com

Posts: 25
Joined: 09/24/2008

Guys,

Thanks for the continued support.

I upgraded to the latest version of ubercart, and I am still having the problem with workflow-ng. It changes the status to "Complete" but doesn't allow the user to download the file, or send off the email with links.

Also users are still complaining about their downloads dying.

-Mike

Posts: 1377
Joined: 08/14/2007
Bug FinderEarly adopter... addicted to alphas.Getting busy with the Ubercode.

One thing I can mention: if the File Downloads and everything is setup correctly, but the users still are not getting downloads, check your SKU fields. I helped an issue with someone a while back where the same thing was happening - orders were being completed but no Downloads events were getting fired. It turns out the user had spaces in their SKUs, and/or their SKU fields were quite long. You'll need to download the uc_file_retro queue (File Downloads Retro) I wrote, which will allow you to edit your product's SKU, and then edit and "re-commit" the File Download feature.

Sorry if that's a confusing statement. But basically once you edit the SKU field in your product, the file downloads will break, and so you'll need to edit the Features again and re-submit them for the new model.

Also, did this just happen recently? Or did it not work to begin with?

Finally, talk to mt again and tell them that you know the difference between a db and a file server. Ask them to actually check the mysql and apache error logs to see if there is any clue as to why the connection between the two servers would be dropping during a simple UPDATE.

--

"Pain don't hurt." - Dalton

Mike Nelson's RiffTrax! www.rifftrax.com

Posts: 25
Joined: 09/24/2008

Now that I think of it, i DID change some of the SKU's on download products.

Thanks for the lead... I'll look into it.

Posts: 25
Joined: 09/24/2008

rosswog wrote:
Now that I think of it, i DID change some of the SKU's on download products.

Thanks for the lead... I'll look into it.

actually i'm wrong. I changed a file name, not the SKU.

for reference, here's my longest SKU:

SNRD002iPhoneCellular

Let me know if this is a problem

Posts: 25
Joined: 09/24/2008

OK I just made a new observation... I tried downloading 2 of our biggest files separately via FTP to see what would happen.

Both attempts failed at exactly 10:00PM and 11:00PM, respectively.

What could this mean?

also, no errors were logged in drupal, or apache.

Posts: 1377
Joined: 08/14/2007
Bug FinderEarly adopter... addicted to alphas.Getting busy with the Ubercode.

The SKU looks fine to me, but I know that when you change filenames you should resubmit the file feature (in the product edit form) to make sure users can download it going forward.

And regarding the FTP issue - you're saying you went into your site's files location, via FTP, and the files stopped even there? That sounds like a more serious issue. How are you connecting? FTP or SFTP? How big are these files we're talking about?

That sounds like another issue you should mention to mt. Perhaps they have limits on how large file transfers can be? I mean, I've never heard of that, but anything is possible, I suppose.

Re: 10 and 11, do you have Cron running every hour?

--

"Pain don't hurt." - Dalton

Mike Nelson's RiffTrax! www.rifftrax.com

Posts: 25
Joined: 09/24/2008

torgosPizza wrote:
The SKU looks fine to me, but I know that when you change filenames you should resubmit the file feature (in the product edit form) to make sure users can download it going forward.

And regarding the FTP issue - you're saying you went into your site's files location, via FTP, and the files stopped even there? That sounds like a more serious issue. How are you connecting? FTP or SFTP? How big are these files we're talking about?

That sounds like another issue you should mention to mt. Perhaps they have limits on how large file transfers can be? I mean, I've never heard of that, but anything is possible, I suppose.

Re: 10 and 11, do you have Cron running every hour?


I logged into my server via FTP and started the download, it worked all the way through. (Sorry, my last post was incorrect... i'm getting worn out!)

I logged into my site as admin / user 1 (access to everything) and tried downloading from a users file list. This is where the downloads stopped on the turn of the hour.

The files are about 600 megabytes each.

I don't have cron running every hour, actually just today i ran it manually for the first time in a long time. Is this critical?

Posts: 1377
Joined: 08/14/2007
Bug FinderEarly adopter... addicted to alphas.Getting busy with the Ubercode.

I'm wondering if mt is running a Cron on your server or something else that you're not aware of? 600 megs is fairly large, but our site has files that are bigger, and we don't have this type of issue.

Did you manage to get ahold of support again? They should really investigate this issue, since on a dedicated server things appear to work without issue, but on the grid they break for some unknown reason. Well, the reason appears to be a dropped connection with the db. To me it sounds like a problem with mt's grid service, at least so far.

--

"Pain don't hurt." - Dalton

Mike Nelson's RiffTrax! www.rifftrax.com

Posts: 25
Joined: 09/24/2008

I am still waiting for their response.

I also noticed that downloads were failing on the 15 minute mark of the hour, i.e. 12:15AM, 10:15AM

I know that on my last test, I started it at about 11:50, it went past 12:00AM, to 12:15AM and stopped.

Posts: 1377
Joined: 08/14/2007
Bug FinderEarly adopter... addicted to alphas.Getting busy with the Ubercode.

Yeah to me, that is pointing to a more serious problem. I'm subscribed to this thread so I'll see any updates you post. Please keep us informed, and if they say that "it's not us, it's Ubercart" then let us know. I doubt that'll be the case, but who knows what they'll say.

--

"Pain don't hurt." - Dalton

Mike Nelson's RiffTrax! www.rifftrax.com

Posts: 25
Joined: 09/24/2008

A couple of months later and still facing this problem.

I was able to break up the 600MB file into smaller pieces, and that worked for a while.

Now my client wants to offer a 1.8GB video download and the problem is rearing it's ugly head again.

I am able to download the file directly from my mediatemple server via HTTP and FTP.

When downloading through UberCart and Drupal, the file transfers are failing at peculiar times, i.e. 10:30, 11:30, 12:00, 2:30.

I'm trying to find out what processes could be going on behind the scenes to create this problem.

At this point, I'm willing to pay $30 an hour for someone to help me figure this out.