File products - downloadable products in core

Posts: 1139
Joined: 08/14/2007
Bug FinderEarly adopter... addicted to alphas.Getting busy with the Ubercode.

Per Ryan's suggestion, I'm posting a new thread regarding the goal of integrating CPiLL's File Product module into the Ubercart Core. CPiLL has given me permission to take over development of his module, since I've added at least one feature and changed some of the functionality of his original code. See the contrib thread here: http://www.ubercart.org/contrib/145

Here's a list of features I'd like to include in the module going forward, and my thoughts / status on them.

- File Products are based on SKU / model, not on purchase or fpid (relationship in database). I have customized my version of the File Product to allow files to be updated independently of the product / model in question. The user is able to visit "My Files" to view an up-to-the-minute list of all files associated with every product they have purchased. I also incorporated a jquery "show-and-hide" script to allow for a collapsible, accordion-like list, to prevent clutter.

- Anti-leech checking - IP and uid based. I have coded a function that tracks each IP address the user has downloaded files from. When a threshold is surpassed, it should trigger a warning in watchdog, logout the user, and change their password - then send an email to the email address on file, making user aware a potential leeching or hacking attempt was logged on their account. Should flag the administrator for review. (Can be disabled if needed, and threshold configurable in the store admin). I have this mostly completed.

- Option: Number of clicks / days before expiration. As an alternative to the "unlimited downloads / anti-leeching" scheme, allow a certain number of times for the file to be downloaded, or number of days before their product expires, whichever comes first. The major downside with the Number of clicks method is you can't use download accelerators or managers, as it causes the link to expire prematurely.

- Add-to-user function. Would like the ability to add a product to a user (for instance if they "win" a downloadable file as a result of a contest, or request). This might be more suitable for Ubercart core - being able to skip the process of adding an order to a user, and then adding products to that order. This would shorten the process considerably. (This might just be better left to adding an order to a customer, as it is now).

- Send links via email when a new purchase is made. Self-explanatory; check to see if downloadable products were purchased. If so, add the links to the user's order complete email. Can either be a link to mysite.com/my_files or to the file/*/download URL.

That's all I have for now. As I've mentioned I'm pretty well on my way to having the major features completed for use on my project site. But I'm more than happy to help code and brainstorm ideas as the process of integrating this awesome module into core continues.

If you'd like to see my code, let me know and I can paste it in here, along with my database structures and all the other necessary info (screenshots, JS, etc.)

Erik

--

"Pain don't hurt." - Dalton

Mike Nelson's RiffTrax! www.rifftrax.com

Posts: 128
Joined: 08/12/2007
Bug FinderGetting busy with the Ubercode.

I prefer the number of clicks/days before expiration option. As stated in the previous thread. I don't really like the use of my_files in relation to purchasing products, to complicated for the user, and requires registration.

I noticed you didn't address anonymous purchasing, and the location of the files -> keeping them out of the web access.

Idan

Posts: 1139
Joined: 08/14/2007
Bug FinderEarly adopter... addicted to alphas.Getting busy with the Ubercode.

Personally I'm not a fan of anonymous purchasing, but it's something we could include as an option (turning it on or off). I feel that it's best to be able to keep track of purchases as much as possible. The best solution, IMO, is to allow someone to purchase the product without logging in; but once they have bought the product, issue them a random password (and allow them to login using their email address and said password). Logintoboggan works great for allowing login via email address.

And I'm not sure what you mean by "location of the files -> keeping them out of the web access". Right now it is possible (using the current File Products module) to keep the downloadable content outside the web root, thereby disallowing a casual user (or malicious hacker) from gaining easy access to the files. I would strongly advise against allowing access to the content from within the public web root, simply for that reason.

--

"Pain don't hurt." - Dalton

Mike Nelson's RiffTrax! www.rifftrax.com

Posts: 14
Joined: 08/29/2007

Erik,

Thanks for all of your contributions to this important functionality.

You mentioned in your note that many of these features you've been working on are already up and running on your project site. Is there a URL where we can access this site and try it out?

Seeing what you've done in action might help jog some ideas for needed features, improvements, or bug fixes...

Thanks,
Ben

Posts: 1139
Joined: 08/14/2007
Bug FinderEarly adopter... addicted to alphas.Getting busy with the Ubercode.

Ben,

Thanks for the kind words. Most of the thanks go to CpiLL for giving us an awesome module to begin with.

There is a site but it's a development / staging area at the moment.. I have it pretty well locked off. So I guess "up and running on the project site" is a bit misleading, since the project site itself isn't live to the public.

I can work on getting a live site enabled, I just have to get a domain and set it up on our dedicated box.

E

--

"Pain don't hurt." - Dalton

Mike Nelson's RiffTrax! www.rifftrax.com

Posts: 331
Joined: 08/07/2007
Administrator

I've been thinking about the posts made regarding file_products into core. So far I've come up with two things:

limits on downloads need to be flexible:
It looks like everyone has different needs/wants with how downloads are restricted. Instead of providing a few options for restrictions (by IP address, # of downloads, a set expiration date, etc.), it seems the thing to do is provide a hook that would be called to allow or deny the file download when the file is requested. This way someone could provide their own logic in a separate module for how they want to restrict downloads based on all the different variables (time since purchase, anonymous users, # of time downloaded and so on...).

file management needs to be flexible:
Like with limits on downloads, the way files are managed (uploaded, edited, etc.) is going vary a lot between applications. Music files might need preview versions for customers, images files might need to be watermarked, PDFs might need to be password protected, software will need to have a serial key, and so on. All of this too much for a general purpose file product module. Hooks should be set in place to operate on a file when it is uploaded, to provide new operations in the file manager, and so forth.

I'll try to come up with more hooks as I start writing code.

--

-Shawn Conn: If the Name Don't Rhyme It Ain't Mine

Posts: 1139
Joined: 08/14/2007
Bug FinderEarly adopter... addicted to alphas.Getting busy with the Ubercode.

I agree, the core of the module should allow for flexibility - but I am also unsure as to whether or not it should just be left at hooks for future modules.

For instance, I would like to keep a lot of the flexibility I have already coded, and not have to write an additional module (or split the two apart) to provide the anti-leeching mechanism. I think perhaps brainstorming all of the different "core" features, such as time limit, allowable clicks, allowable IP addresses, etc., is fairly broad enough to be usable by everyone requiring downloadable content - enough so, I think, to have them as selectable options in the File Product core.

--

"Pain don't hurt." - Dalton

Mike Nelson's RiffTrax! www.rifftrax.com

Posts: 4368
Joined: 08/07/2007
AdministratorHead Code Monkey - I eat bugs.

I think I'd lean more toward torgos' idea of keeping as much as possible in a single module. I don't have any thoughts on what that would include, though. Do you need me to go ahead and move the file product contrib over to you so you can update the download w/ your latest version?

Posts: 331
Joined: 08/07/2007
Administrator

That would be nice. As is, it breaks while testing since order statuses have changed since the 1.6 release of uc_file_product.

--

-Shawn Conn: If the Name Don't Rhyme It Ain't Mine

Posts: 1139
Joined: 08/14/2007
Bug FinderEarly adopter... addicted to alphas.Getting busy with the Ubercode.

Let me know what you guys would like from me at this stage. I talked to my supervisor and we're thinking about opening up our currently-in-development shopping cart site up so that the other developers here can take a look. A lot of it is broken (I'm still working on theming it). It's either that or create a completely blank slate with different products, etc.; although for ease of use I'd like to keep everything more or less mirrored in both areas.

For now though I can start pasting snippets of code (or attaching them) with regard to the changes I've made. I can also update the current File Product module with the change for order statuses in it. I don't think uploading MY custom version of the module is a good idea yet, since it's not yet feature-complete as far as my needs go.

Let me know what you think.

--

"Pain don't hurt." - Dalton

Mike Nelson's RiffTrax! www.rifftrax.com

Posts: 4368
Joined: 08/07/2007
AdministratorHead Code Monkey - I eat bugs.

I gotcha. I'll go ahead and set you as the contrib author so you can put in the fix for order statuses. That's kind of essential. Sticking out tongue

Posts: 10
Joined: 10/09/2007
Bug Finder

I'm really excited that so much progress is happening on the file product module. This is definitely something we'll be taking advantage of.

I like the idea of the middle-road... Have a complete working out-of-the-box solution for the most common virtual product strategies like torgos has explained, but keep the inner workings of the module as flexible & hook based as possible so developers can customize everything every step of the way. But hey isn't that whole philosophy behind Drupal?

From what I've read the planned features for the default file product settings sound like exactly what we would need to start selling virtual products through the site. We'd also like to allow users to feature purchased products on their profile, purchase products as gifts for other users, all sorts of cool music sharing stuff so I'm really looking forward to playing with these new core features to see how it works.

Just wanted to introduce myself and let you all know I'm a big fan of any file product progress. Smiling

Posts: 1139
Joined: 08/14/2007
Bug FinderEarly adopter... addicted to alphas.Getting busy with the Ubercode.

Sara,

It's a pleasure to meet you. I work for Legend Films and RiffTrax here in San Diego - our core business is downloadable products. So, I have a lot of ideas and can definitely aid in the process of moving File Products along.

If you have any questions or suggestions, feel free to drop me a line.

--

"Pain don't hurt." - Dalton

Mike Nelson's RiffTrax! www.rifftrax.com

Posts: 116
Joined: 08/08/2007
Brain Stormer


I agree, the core of the module should allow for flexibility - but I am also unsure as to whether or not it should just be left at hooks for future modules.

For instance, I would like to keep a lot of the flexibility I have already coded, and not have to write an additional module (or split the two apart) to provide the anti-leeching mechanism. I think perhaps brainstorming all of the different "core" features, such as time limit, allowable clicks, allowable IP addresses, etc., is fairly broad enough to be usable by everyone requiring downloadable content - enough so, I think, to have them as selectable options in the File Product core.

I understand the need for a single module that does some percentage of most requested feature set. it's probably simpler initially. but is it really simpler in the long run? i like shawn's idea about hooks.

i mean would it really be that hard to modify what you've done to use hooks versus how you have it hard coded right now?

i'm thinking you could keep your code the same and add a few hooks where it's likely that multiple functionality in a given area might be desired. in those hooks you might want to switch / case on an option setting which then calls your original functions depending on the store owners needs. right now, you simply make your most-desired functionality (probably a function call in the case block) the default behavior.

in this sense, my description of a hook is not necessarily a traditional drupal hook where every module is traversed to invoke the hook and add some functionality. It's more or less a wrapper function that checks for a configuration option setting and then simply calls a function based on the setting. the function can be supplied by an include file. in otherwords, your base module will provide a framework which is easier to enhance and dis
|en able functionality.

just a suggestion. i personally like the idea of extensibility in an area that is going to be heavily used and require a lot of tweaking. fwiw, i believe you edited CPill's original module to get the functionality you wanted. what about the next guy/gal, etc.

you can already tell by the interest here file downloads are highly in demand. anyway, i don't have time to code the thing so you'll eventually make your own decisions but thought i'd cast my vote in terms of hooks Smiling

Posts: 1139
Joined: 08/14/2007
Bug FinderEarly adopter... addicted to alphas.Getting busy with the Ubercode.

Oh, I'm not against adding in hooks - don't get me wrong - I'm just saying there should already be some inherent functionality, with regard to flexible security and file management. We can make generic hooks that people can use if they need a specific function - but I also like the idea of someone requesting a feature, and if it's something everyone likes, adding the code into the core of the module as well. Smiling

--

"Pain don't hurt." - Dalton

Mike Nelson's RiffTrax! www.rifftrax.com

Posts: 116
Joined: 08/08/2007
Brain Stormer

sounds good. i guess we're in agreement. rock on Smiling..

Posts: 14
Joined: 10/07/2007

Hi Eric

These days there are so much hype around the band Radio Head and there concept where the customer decide the price. Maybe that's not gonna last for long, but i really like the concept on the magna music store, https://magnatune.com, where you can select what price you wanna pay for the product. /normal, genius, very nice ...)
https://magnatune.com/buy/buy_dl_cc?artist=Junoon&album=Azadi&genre=New+...

Is that possible already, or could that be possible in your future file module?

All the rest you write about sounds great! I like the IP tjecking !

/August

Posts: 331
Joined: 08/07/2007
Administrator

I guess the thing to do is both; implement the functionality Erik mentioned (IP limits, download limits, time limits...) and also throw in a hook that gets called after all those other checks are made. This gives us the flexibility for anyone to throw in their needed file download limits and the usability of just enabling the module, configuring limits, and being set to go. If anyone needs only a custom built file limit solution, they can turn off all the built in option and use the hook.

--

-Shawn Conn: If the Name Don't Rhyme It Ain't Mine

Posts: 1139
Joined: 08/14/2007
Bug FinderEarly adopter... addicted to alphas.Getting busy with the Ubercode.

Sounds like a plan to me.

By the way, you can see the changes I've made on my test site:

http://torgo168.rifftrax.com

Login: testuser
Pass: testing124 (yes, 124)

You can view the File Products module (using show-hide with jquery) under My Downloads (on the left). I'm not running any payment gateways or anything, so you can just submit an order and it'll go through, but won't be processed. (So I'd recommend not doing that) Smiling

--

"Pain don't hurt." - Dalton

Mike Nelson's RiffTrax! www.rifftrax.com

Posts: 116
Joined: 08/08/2007
Brain Stormer

torgo,

is the code with your "complete" mods available for download yet?

i know you worked in and posted 1 change to cpill's original code under his original thread. but i don't think that has any of the enhancements you presented at the top of this thread even though i think you have most of it already working on your test site. not sure you if you followed that Smiling.

if it's not posted yet, any idea on a rough eta for posting and where it will posted? i guess if it's going to wind up in core then we just need to bzr update.

fwiw, one thing i've noticed is that sometimes i'll see a thread that's listed as being 'recent' in my tracker meaning i've posted to it and someone else has updated it recently. however, when i go to inspect the thread, the dates are often very old (not recent at all). i think this is probably occurring because someone "edits" a previous post but the original CREATION dates don't change. drupal tries to keep the post context based on the creation date but marks the thread as "recent" based on MOD time. as a result, it's hard to figure out which post about a given thread made it different / recent. I think this is often the case where someone changes the link in an existing (older) contrib module post to update a contrib module download link. It's hard to be fully certain that the module download has actually been updated or not. at least for me it is Eye-wink

i see this same behavior in "alpha ## battle plan" type threads a lot.

not sure if what i'm saying makes sense but thought i'd bring it up since you're going to be taking over this module and it's probably going to be pretty popular. hence a method of posting updates would be best served if as confusion free as possible. of course if this is rolled into core than you simply have to bzr update.. but it's also possible that you will let people bang on the code as a contrib module before making it a permanent part of core.

thanks bud