Home » Forums » Ubercart » Ideas and Suggestions

File Downloads using content delivery network (CDN)

Submitted by torgosPizza on Tue, 02/26/2008 - 12:45
torgosPizza

Posts: 1315
 Joined: 08/14/2007
Bug FinderEarly adopter... addicted to alphas.Getting busy with the Ubercode.

This is mainly directed at Shawn, but I'm looking into a way that we can utilize an external Content Delivery Network, or CDN, to leverage our downloads but maintain the functionality of the Downloads module.

The way this CDN will work is by making the download come from a CNAME, a setting in DNS that allows for a subdomain of cdn.example.com, for instance. We will be the origin server, meaning we will only upload files to our site, and the CDN comes into play on a per-request basis.

I'm still hashing out the details, but I was wondering if anyone had any experience with CDNs in the past, and what they think the best way to integrate this type of system would be. I'm not sure what it would take to change the actual file download headers to come from another domain... and hopefully there won't be any cross-domain issues, or should I say cross-server issues. The main hurdle I see in the near future is the fact that our downloads are kept above / outside the web root to prevent people from just being able to download our content without buying it. I'm sure this CDN has a way around that; I'm talking to them this afternoon so I'll post a reply with any developments we reach. (We're going to be scheduling a full-scale test in the near future as well).

Let me know what you think.

--

"Pain don't hurt." - Dalton

Mike Nelson's RiffTrax! www.rifftrax.com
permalink #1 Tue, 02/26/2008 - 14:17
Shawn Conn

Posts: 332
 Joined: 08/07/2007
Administrator

I have no familiarity with CDNs. In fact, before this post, I didn't even know what they were. From the brief overviews I've read it sounds like you basically get a URL from a CDN and it handles the overhead of load balancing and whatnot. Assuming you had a table that mapped current file ids to URLs on your CDN you could modify _file_download_transfer to stuff a "Location: " into the HTTP headers that would just redirect the download to the appropriate CDN URL. That sounds like it could be grossly simplifying the job, but without more specifics that's all I can tell you.

--

-Shawn Conn: If the Name Don't Rhyme It Ain't Mine

permalink #2 Tue, 02/26/2008 - 15:00
torgosPizza

Posts: 1315
 Joined: 08/14/2007
Bug FinderEarly adopter... addicted to alphas.Getting busy with the Ubercode.

That does sound like it'll work. At least that way the module is still controlling access rights, but it's just coming from a different location rather than the local server.

I was thinking about how the Amazon S3 contrib worked things out - I'll have to take a look. But maybe an "Advanced" option in the uc_file setup to be able to use a content delivery network. I would hate to have to keep updating the core module whenever a new version is downloaded from bazaar.

Thanks for pointing me in what I think is the right direction.

--

"Pain don't hurt." - Dalton

Mike Nelson's RiffTrax! www.rifftrax.com

permalink #3 Wed, 02/27/2008 - 13:45
spuky
Posts: 37
 Joined: 01/31/2008

For me this sounds like this will only work if the CDN is giving you onetime URL's to your prestored content. Other wiese the URL is out there and can't be controlled.

DNS based CDNs are mostly for getting hi demanded content near to the user.. take for example youtube the have an video up on their frontpage (and if they were a smaller company with smal servers) they would hire a cdn for their domain and as soon as I wan't that vid for the first time it is mirrored to a location nearto me. And if now my neighbor is going to the same site they say we got a copy of this allready in his area he can just download that. So my understanding is that for your situation where you only wan't paying customers to download your content i guess it will not work...

But it could at least hold all your free content that people start watching when they come to the site which I guess will be more traffic than the actual sold stuff...

permalink #4 Wed, 02/27/2008 - 13:50
torgosPizza

Posts: 1315
 Joined: 08/14/2007
Bug FinderEarly adopter... addicted to alphas.Getting busy with the Ubercode.

Good point. I'll have to ask the CDN about security, if there is a unique hash or some kind of value we can pass via URL that validates it on both ends. Might be interesting to see if that's something they have done before, otherwise you're right, it'll be near impossible to control it once the URL gets out there. Thanks for reminding me of that.

--

"Pain don't hurt." - Dalton

Mike Nelson's RiffTrax! www.rifftrax.com

permalink #5 Mon, 03/03/2008 - 17:31
agilpwc
Posts: 73
 Joined: 12/10/2007

If I understand you correctly the Amazon S3 mods I made to uc_file fulfill all your requirements.

S3 is pretty awesome. It provides ACL or access control and expiring urls. So I set all the files private. Then one someone buys a download and they goto download page and url are generated that expire after X seconds.

Also Amazon provides for subdomains. If you name your Amazon bucket for example video.mysite.com and change your cname to redirect video.mysite.com to amazonsws.com then it gets sorted out correctly and it looks like the download is coming from video.mysite.com not amazon.

It is all pretty slick.

I've been keeping up with uc_file updates, but haven't been uploading them since no one seemed to be using it.

permalink #6 Mon, 03/03/2008 - 17:44
torgosPizza

Posts: 1315
 Joined: 08/14/2007
Bug FinderEarly adopter... addicted to alphas.Getting busy with the Ubercode.

That's pretty cool - what does S3 entail? To be honest I haven't looked much. I'm sure there's a cost involved, and you don't have to share. But I would be interested in finding out what it would cost to host all of our content with them. It sounds like, based on what you're saying, that it would fulfill all our needs and then some, especially when combined with Ubercart.

Thanks for any detail you can offer me. (If you want to keep it private you can PM me, I promise not to share any sensitive info.)

--

"Pain don't hurt." - Dalton

Mike Nelson's RiffTrax! www.rifftrax.com

permalink #7 Mon, 03/03/2008 - 17:50
agilpwc
Posts: 73
 Joined: 12/10/2007

http://www.amazon.com/gp/browse.html?node=16427261

Bascically between 15 and 18 cents a gigabyte.

permalink #8 Mon, 03/03/2008 - 18:09
torgosPizza

Posts: 1315
 Joined: 08/14/2007
Bug FinderEarly adopter... addicted to alphas.Getting busy with the Ubercode.

That is pretty sweet. And I love how your module works seamlessly with uc_file - you don't even realize there's a difference. Definitely post the new code when you can!

--

"Pain don't hurt." - Dalton

Mike Nelson's RiffTrax! www.rifftrax.com